Version 134 (modified by 5 years ago) ( diff ) | ,
---|
Amazon EC2
Table of Contents
- 1. Create AWS Account
- 2. Create Instance
- 1: Log in to the Management Console
- 2: Select a Region
- 3: Launch Instance
- 4: Choose an Amazon Machine Image (AMI)
- 5: Choose an Instance Type
- 6: Configure Instance Details
- 7: Add Storage, Add Tags
- 8: Configure Security Group
- 9: Create KeyPair
- 10: Configure Security Group
- 11: Associate Elastic IP
- 12: Gain SSH access
- 13: Configure Email to use a Smart Host
- 3. Install & Configure Sahana
- Optional Instance Adjustments
- CLI Tools
- Building AMIs for easier deployment
- Troubleshooting
Amazon's Cloud provides a flexible platform to deploy Eden scalably.
The costs aren't fixed & can be difficult to predict, despite their calculator, but are competitive, especially in Singapore, which is a good base for the Asia Pacific region. Users who are using the free tier: Remember - after 750 hours, your trial will end and the credit card on file will be charged based on the rates shown in EC2. You can prevent these charges by closing the AWS account from the AWS account management page.
1. Create AWS Account
If you haven't already, create an Amazon AWS account through their site.
2. Create Instance
1: Log in to the Management Console
2: Select a Region
Amazon supports multiple Regions in order to provide a service closest to your users.
- Namespaces of Instances, Volumes & Snapshots are unique only within a Region.
- Within each Region, there are a couple of Availability Zones to allow spreading the risk across different facilities.
- Volumes are located within a specific Availability Zone
- Bandwidth transfers are free within an Availability Zone
3: Launch Instance
This is a blue button in the EC2 Dashboard
4: Choose an Amazon Machine Image (AMI)
- Recommend using the AWS Marketplace and selecting the Debian GNU/Linux 9 (Stretch) image
- alternatively it is possible to use Community AMIs, use latest debian-stretch-amd64-hvm image, which supports the new T2.micro free tier instances
- In time we may provide pre-built "Sahana Eden" AMIs (some old unmaintained ones may be available in some regions)
5: Choose an Instance Type
- https://aws.amazon.com/ec2/instance-types/
- The free starter 'T2.micro' instance is flexible as it can run both 32-bit & 64-bit Operating Systems....it is suitable for prototyping, development, QA and smaller scale production services. Note that for User Training it can be good to increase capacity as this typically has more users accessing the system concurrently than in normal operations.
- For production-level performance of larger deployments, we recommend a balance of processor & RAM, so the M5.large would be our current recommendation, usually purchased as a 1 year reservation
6: Configure Instance Details
Default settings are fine, except you should open the 'Advanced Details' at the bottom & paste the contents of the user-data.sh file into the User Data section:
If you leave this unedited then this will install a Setup tool which will allow you to configure your Sahana Eden instance through a GUI
if you wish to bypass the GUI then you can delete the last 2 lines & edit the last line to select the template that you wish to run, to set your site's Public DNS, and the email address used to send mails From:
bash bootstrap.sh mytemplate myhostname.mydomain sender@domain
7: Add Storage, Add Tags
Default settings are fine.
8: Configure Security Group
'Select existing' to avoid the confusion caused by having multiple groups.
9: Create KeyPair
Ensure that you keep the generated private key safe...save as private.pem
. You will need this file to log into your instance.
10: Configure Security Group
NETWORK & SECURITY > Security Groups
You will need to set the following Inbound Rules:
- HTTP | TCP | 80
- HTTPS | TCP | 443
- SSH | TCP | 22
Restricting the source will add further security, but obviously also restricts your ability to administer
11: Associate Elastic IP
Each time you start an instance up, it will be assigned a new IP ('Public DNS') although this can be overcome using an Elastic IP:
- NETWORK & SECURITY > Elastic IPs
- Allocate New Address
- Associate Address. Set the instance to your new instance
NB If you have a free EC2 instance, be sure to release your Elastic IP if you shut down your instance. IPv4 addresses are a "scarce resource" so Amazon will charge you for wasting one if you keep it assigned to your instance while you are not using it.
If you are not using a Smart Host, remember to set up Reverse DNS for your Elastic IP to be able to send emails reliably:
12: Gain SSH access
In order to get the public key (needed by SecureCRT for instance) then you need to login using CLI & retrieve it (username 'admin' for the AWS MarketPlace Debian, username 'root' for some other Images):
ssh -l admin -i private.pem <hostname> cat ~/.ssh/authorized_keys
On Windows, you can use Cygwin to get a CLI SSH client.
SecureCRT needs the private key storing as <filename> & the public as <filename.pub> (all on one line)
Recovering From a Lost Keypair
If you lose your keypair then you need to:
- Create a new keypair in the AWS console & download the generated private key
- Stop the instance
- Create an AMI from this instance
- Wait for the AMI to be ready
- Launch a new instance using this AMI
- Re-associate the Public IP
- Delete the old instance
- Deregister the AMI
- Delete the snapshot used to create the AMI
Thanks to: http://itkbcentral.blogspot.co.uk/2011/07/replace-lost-key-pair-existing-aws-ec2.html
13: Configure Email to use a Smart Host
Whilst you can configure your system to send email directly, this is fraught with problems as your IP will often get black listed purely for being an Amazon IP.
It is suggested that you use a Smart Hose such as SendGrid or Amazon's own Simple Email Service (SES). We go through the latter process here since it is free for the 1st 62,000 mails/month which is sufficient for most of our use cases.
- In your Amazon Console, navigate to SES (Services | Customer Engagement | Simple Email Service)
- Verify your Outbound Email Sender
- Domain, if possible: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domains.html
- Email Address, otherwise: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html
- Get SMTP Credentials (USERNAME & PASSWORD for the below config): https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html
- Lookup your SMTP endpoint for your AWS region: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-connect.html
- Configure your MTA, which we assume to be Exim4:
cd /etc/exim4 # We start with the default upstream config rather than the Debian one gunzip /usr/share/doc/exim4-base/examples/example.conf.gz cp example.conf exim4.conf # Configure guided by https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-exim.html vi /etc/exim4/exim4.conf primary_hostname = myhost.mydomain # set options to empty value to avoid warnings keep_environment = tls_advertise_hosts = begin routers # Use SMTP endpoint for your region send_via_ses: driver = manualroute domains = ! +local_domains transport = ses_smtp route_list = * email-smtp.us-west-2.amazonaws.com; begin transports ses_smtp: driver = smtp port = 587 hosts_require_auth = * hosts_require_tls = * begin authenticators # Use your USERNAME & PASSWORD ses_login: driver = plaintext public_name = LOGIN client_send = : USERNAME: PASSWORD # Test config: exim -C /etc/exim4/exim4.conf -bV service exim4 restart # Test: exim -v recipient@domain From: me@myhost.mydomain Subject: Testing SES CLI Test .
- Request a Sending Limit Increase using the AWS Console (button in Sending Statistics)
3. Install & Configure Sahana
This should happen automatically through the user-data.sh script that you used earlier. You can check the progress of this using:
tail -f -n 45 /var/log/user-data.log
If you are not using the user-data.sh then you can use the older Manual scripts
See Admin Guide - especially read how to set the sender & approver emails
Optional Instance Adjustments
Add Swap partition
You can add a swap partition in order to improve performance further:
- Create Volume in AWS Console (e.g. 4Gb)
- Attach as /dev/sdf
sudo su - swapoff -a mkswap /dev/xvdf swapon -a # Make persistent across reboots cat << EOF >> "/etc/fstab" /dev/xvdf swap swap defaults 0 0 EOF rm -f /mnt/swapfile
Add Storage
If you need an additional disk for Storage then configure a new volume in the AWS console. There are several different types available: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
For the database (normally on the boot volume) you want at least a General Purpose SSD (gp2). For additional storage (Prepop, Logs, Uploads, etc) then a Cold HDD (sc1) provides a cheaper per-Gb option, but needs a minimum of 500Gb provisioned.
Attach as /dev/sdf, then in Linux console:
sudo su - fdisk /dev/xvdf n (accept defaults) w mkfs.ext4 /dev/xvdf tune2fs -m 0 /dev/xvdf # Remove 5% reservation for reserved blocks mkdir /data cat << EOF >> "/etc/fstab" /dev/xvdf /data ext4 defaults,barrier=0 1 1 EOF mount /data
Grow Storage
Can increase the size of an existing disk. This can be done dynamically (no need to go offline):
- In AWS console, locate the Volume & Create a Snapshot (just in case you wish to rollback)
- Modify Volume, set the new size
- Login to server to resize the filesystem (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html):
# Identify the volumes present & their labels file -s /dev/xvd* # See old filesystem sizes df -h # See new volume sizes & check if a partition needs resizing lsblk # *If* an xvdf1 partition needs extending (no need if the filesystem is just inside a disk xvdf) growpart /dev/xvdf 1 # Extend the filesystem resize2fs /dev/xvdf1 # Confirm that this has taken effect df -h
- Delete the backup Snapshot once you've confirmed that your data is unharmed
Disk Striping
For DB I/O performance increase can stripe multiple EBS
- monitoring data is available to see if this is the issue
CLI Tools
You can do this using the AWS EC2 Console or else you can do it via the CLI To use any of the AWS CLI tools on your own machine to remotely manage instances, then you need to generate a unique X.509 Certificate per account. This can be done from the 'Security Credentials' page within your account.
CLI Management
There are extensive CLI tools available to manipulate your instances.
- Java CLI for Windows/Linux
- Python: http://libcloud.apache.org
CLI Script
Edit the settings as-indicated as you proceed through the script
# Settings for Instance set EC2_URL=https://ec2.us-east-1.amazonaws.com set ZONE=us-east-1c set DEV=i-950895f1 set OLD=vol-31f5a35d # Stop Host ec2stop %DEV% # Create a snapshot ec2-create-snapshot %OLD% # Record the snapshot ID set SNAPSHOT=snap-63f89d08 # Create new volume from snapshot ec2-create-volume -z %ZONE% --size 4 --snapshot %SNAPSHOT% # Record the new Volume ID set NEW=vol-a9c2a3c4 # Attach new volume as secondary ec2-attach-volume -i %DEV% %NEW% -d /dev/sdb1 # Delete Snapshot (if no data in yet) ec2-delete-snapshot %SNAPSHOT% # Start Host ec2start %DEV% # Re-attach the Public IP # Login mkdir /mnt/data echo '/dev/xvdb1 /mnt/data ext3 defaults,noatime 0 0' >> /etc/fstab mount /mnt/data resize2fs /dev/xvdb1 umount /mnt/data shutdown -h now # Unattach volumes ec2-detach-volume -i %DEV% %OLD% ec2-detach-volume -i %DEV% %NEW% # Attach volume as boot ec2-attach-volume -i %DEV% %NEW% -d /dev/sda1 # Attach old volume for /var/log ec2-attach-volume -i %DEV% %OLD% -d /dev/sdb1 # OR Delete old volume #ec2-delete-volume %OLD% # Start Host ec2start %DEV% # Re-attach the Public IP # Login df -h # Use the old partition for /var/log (to avoid DoS) vi /etc/fstab /dev/xvdb1 /var/log ext3 noatime 0 0 mv /var/log /var/log_old mkdir /var/log mount /var/log mv /var/log_old/* /var/log rm -rf /var/log/bin/ rm -rf /var/log/boot/ rm -rf /var/log/dev/ rm -rf /var/log/etc/ rm -rf /var/log/home/ rm -rf /var/log/initrd.img rm -rf /var/log/lib/ rm -rf /var/log/mnt/ rm -rf /var/log/media/ rm -rf /var/log/opt/ rm -rf /var/log/proc/ rm -rf /var/log/root/ rm -rf /var/log/sbin/ rm -rf /var/log/selinux/ rm -rf /var/log/srv/ rm -rf /var/log/tmp/ rm -rf /var/log/usr/ rm -rf /var/log/var/ rm -rf /var/log/vmlinuz rm -rf /var/log_old
Building AMIs for easier deployment
See: InstallationGuidelines/Amazon/AMI
Troubleshooting
To troubleshoot any errors in installation of EC2 visit its documentation. If you encounter problems installing eden on the EC2 instance, you can contact us via IRC or the mailing list.
Attachments (14)
- ami-built.png (156.9 KB ) - added by 11 years ago.
- ami-permissions.2.png (176.1 KB ) - added by 11 years ago.
-
orig-ami-built.png
(172.9 KB
) - added by 11 years ago.
lifeeth's original ami-built.png from 2 years ago
-
orig-create-image.png
(271.0 KB
) - added by 11 years ago.
lifeeth's original create-image.png from 2 years ago
-
orig-ami-permissions.png
(215.7 KB
) - added by 11 years ago.
lifeeth's original ami-permissions.png from 2 years ago
-
orig-create-image-config.png
(270.8 KB
) - added by 11 years ago.
lifeeth's original create-image-config.png from 2 years ago
- create-image-config.png (184.8 KB ) - added by 11 years ago.
- create-image.png (211.0 KB ) - added by 11 years ago.
- Dashboard.png (103.7 KB ) - added by 11 years ago.
- SelectAMI.png (94.4 KB ) - added by 11 years ago.
- SecurityGroup.png (70.1 KB ) - added by 10 years ago.
- UserData.png (59.3 KB ) - added by 10 years ago.
-
select_debian.png
(68.2 KB
) - added by 8 years ago.
selct debian OS
-
select_region.png
(12.8 KB
) - added by 8 years ago.
select region