/dev/websphere

I'd been doing this anyway for some scale up testing but with the story on infoq about EC2, I figured I'd do a post. The trick with amazon ec2 cost wise is limit the amount of data transfered in and out of  the virtual machines as the bandwidth is the killer component of the cost. When I first did ObjectGrid testing on EC2, I'd start the virtual machines, scp (secure copy) up objectgrid and an ibm jvm to each one which is costing me money per mega byte. This obviously not so smart so heres the right way to do it.

First, create a virtual machine using your favorite prebuild image. I used fedora 4 + mysql + apache. I then uploaded objectgrid and the IBM java 6 JVM. The virtual machine has two disks in it. One has the operating system on it and the other (/mnt) is for data. It's important to unzip these in the root directory and not in /mnt. The /mnt disk isn't part of the image so anything there won't be available in the cloned virtual machines using your new image. Once they are up there then upload any scripts you need or application jars, spring jars what ever. Put everything in a directory on the main file system. Don't use /mnt for anything related to your new image.

Test it and now we need to tell Amazon to create a new private image of this. This new image includes the existing amazon prebuilt image AND the new stuff you uploaded. The first thing to do is to get your amazon account number, key id and secret key. Keep them handy.

First, create a directory called ami from /mnt. Run the following command

ec2-bundle-vol -d /mnt/ami -k /mnt/PrivateKey.pem -c /mnt/509Certificate.pem -u AccountId

You can see I also uploaded my private and 509 certificate pem files to /mnt also. Don't put them in / as then anyone with your image has them. /mnt is not included in the image that this makes. This creates the image after around ten minutes in the /mnt/ami directory and splits it in to chunks for upload to Amazon S3 where the image has to be stored.

Next use this command to upload this new image:

ec2-upload-bundle -b bnewport-ami-og-image -m /mnt/ami/image.manifest.xml -a KeyId -s SecretKey --retry

This uploads the image to an S3 bucket called bnewport-ami-og-image. Use a different bucket name for each image you want to make. I needed to use the -retry parameter as the upload would almost always fail on one of the pieces. The retry parameter fixes this. Now, you have made an image and you have uploaded it to S3. Now, you need to tell Amazon EC2 about it using this command which needs to be run from your laptop.

ec2-register bnewport-ami-og-image/image.manifest.xml

Once you have done this then you can see the image when you do a 'ec2-describe-images' command. You can then start the private image using the 'ec2-run-instance' command and use the parameters to start as many instances as you can afford to do. I start a catalog server on two of the instances and then start my application JVMs on all the machines using the host names for the two catalog servers. I wrote shell scripts for my mac to execute ssh commands to start/stop all the instances using the host names of the running instances that I extract from ec2-describe-instances and awk.

Thats it, this is a pretty cool way to do scale testing on an application. Start em up when you need them and then stop them when you are done.