Share Virtual Machines

Here are the Top 12 referrers to our blog over the past 3 months, the numbers of referrals are in parentheses.

1. http://pro-linux.de/berichte/ext4/ext4.html (765)
2. http://networksecuritytoolkit.org/nst/index.html (566)
3. http://dabcc.com/article.aspx?id=9653 (149)
4. http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/ (111)
5. http://kakku.wordpress.com/2008/06/23/virtualbox-shrink-your-vdi-images-space-occupied-disk-size/ (101)
6. http://stumbleupon.com/refer.php?url=http://sharevm.wordpress.com/2009/01/19/most-popular-vmware-virtual-appliances-for-it-administrators/ (84)
7. http://techblog.41concepts.com/2008/03/31/shrink-your-windows-disk-image-on-wmware-fusion-mac/ (67)
8. http://thedarkmaster.wordpress.com/2007/03/12/vmware-virtual-machine-to-virtual-box-conversion-how-to/ (66)
9. http://blogs.msdn.com/heaths/archive/2005/07/30/445621.aspx (66)
10. http://prefetch.net/blog/index.php/2007/01/21/determining-file-fragmentation-on-ext3-file-systems/ (61)
11. http://virtualgeek.typepad.com/virtual_geek/2009/01/updated-homebrew-esx-hardware-list.html (52)
12. http://blog.rightscale.com/2009/01/09/amazon-launches-ec2-console/ (53)

Thank you for the referrals. Hope the content is meaningful for our readers

I received the email annoucement from Amazon ec2 earlier today:

You can read the detailed announcement here and also download the AWS Toolkit for Eclipse.

Amazon’s Elastic Computing Cloud (ec2) is offering over 1500 AMI’s (virtual appliances). It’s recent foray into Europe highlights its aggressive pursuit of a land grab strategy. So, what has its competition in the US been up to? Every major web hosting vendor has announced a cloud initiative:

• The Rackspace cloud is called Mosso. Its prices compare favorably with Amazon’s. Its pricing calculator and spreadsheets can be found here.
• Savvis has a VMWare ESX based offering for Windows and Linux
• AT&T is also offering VMWare ESX based hosting Windows, Linux and Solaris x86. They intend to offer Citrix XenServer and Microsoft Hyper-V in the future. AT&T offers SLA’s for cloud services, a first.
• Terremark has also launced an Enterprise Cloud, which incidentally is a utility computing cloud sans virtualization
• IBM’s cloud initiative has its products available as AMI’s on ec2.
• Sungard has not yet publicly announced its cloud plans.
• Verizon is slated to announce its cloud initiative in Summer 2009

Amazon’s ec2 is nearly three years old now (Mar 2006) and competition has started appearing on the horizon just over the past year. If you look at the timescale of announcements,

• Rackspace’s Mosso is just over a year old (Feb 2008),
• Terremark’s cloud is nine months old (June 2008),
• AT&T’s cloud is seven months old (Aug 2008),
• IBM’s AMI launch on ec2 and Savvis’ cloud is three weeks old (Feb 2009).

Since ec2 was first to market, the Rackspace, AT&T and Savvis cloud offerings have a “me too” feel to them. However, unlike other vendors, Rackspace has published pricing on the Web and it appears to be very competitive with ec2. 

Unlike Amazon, Savvis and AT&T are going the buy versus build route to get fast time to market. They are initially launching their service using VMWare technology put together using Professional Services instead of following Amazon’s approach of building a proprietary infrastructure using Open Source software as its foundation. In fact, this may be the preferred route amongst the upper echelons of cloud service providers. There is optimism that providing cloud services is a growth business. I am noticing startups like Enomaly (“Build your own private elastic cloud”) and VMOps (“launch ec2 today”, aka public cloud) offering cloud infrastrcture products.

In conclusion, credible competition is emerging and there are real alternatives to ec2 available today. However, given the state of the economy, I think the market will begin to form by 2010 and should reach critical mass by 2011-2012.

 Ubuntu JeOS

• How to develop virtual appliances using Ubuntu JeOS

VMWare Studio

• Studio: A Free Virtual Appliance Authoring Tool With Robust Management Features
• Videos: Learn how to build a VMWare virtual appliance
• Create your own Linux appliances

Parallels

• How to build a Parallel’s virtual appliances

rPath rBuilder

• rBuilder Appliance creator
• Best practices for building virtual appliances

Rightscale

RightImages provide ready-to-go base operating systems with core cloud software.

ServerTemplates also allow you to designate any number of scripts that you want to run at boot time, upon demand, or when an event is triggered.

RightScripts allow you to specify packages that you want to install before a script is executed and even allow you to upload and attach files directly to scripts.

Amazon Machine Image (AMI)

• Use an existing AMI
• Create your own AMI from scratch

Here are the Top 12 referrers to our blog over the past 3 months, the numbers of referrals are in parentheses.

1. http://pro-linux.de/berichte/ext4/ext4.html (546)
2. http://dabcc.com/article.aspx?id=9653 (342)
3. http://networksecuritytoolkit.org/nst/index.html (110)
4. http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/ (59)
5. http://communities.vmware.com/thread/189804?tstart=0 (49)
6. http://techblog.41concepts.com/2008/03/31/shrink-your-windows-disk-image-on-wmware-fusion-mac/ (42)
7. http://blog.rightscale.com/2009/01/09/amazon-launches-ec2-console/ (37)
8. http://wordpress.com/tag/vhd/ (33)
9. http://wordpress.com/tag/vmdk/ (32)
10. http://virtualgeek.typepad.com/virtual_geek/2009/01/updated-homebrew-esx-hardware-list.html (32)
11. http://blogs.msdn.com/heaths/archive/2005/07/30/445621.aspx (32)
12. http://kakku.wordpress.com/2008/06/23/virtualbox-shrink-your-vdi-images-space-occupied-disk-size/ (31)

Thank you for the referrals. Hope the content is meaningful for our readers.

Mike Culver, technology evangelist for Amazon Web Services announced the availability of a web-based management console for ec2, the Elastic Compute Cloud.

The key features are summarized by James Urquhart and by Saad Ali Abassi.

Alan Williamson offers a visual tour of the new UI.

As an AWS customer, I feel heartened by this investment on Amazon’s part because it emphasizes their commitment to commoditize AWS and make its services widely usable. The AWS services will become cheaper as usage ramps up on the commoditization curve and makes “cloud computing accessible for the masses“

Charles Babcock has written an insightful article  on the InformationWeek virtualization blog that highlights how virtualization is a foundational pillar for cloud computing. I have paraphrased it below:

“The cloud infrastructure is an abstracted, fabric-based infrastructure that enables dynamic movement, growth and protection of services that are billed based on usage.” …  ”In the cloud, bits, bytes, and cycles are transparent.”

… Smith and Pieravi’s presentation cites three stages of virtualization ….

Stage one is server consolidation through virtualization in the data center, along with virtualization of software development and testing. This stage already has taken place at many enterprises,

… in 2008 we entered stage two, where implementation of virtualization leads to flexible resource management, juggling virtual machine operation through the day for dynamic load balancing, high availability, and disaster recovery.

Stage three will begin in 2012, and … will take advantage of the virtualization skills gained in stages one and two. By knowing how to build and store virtual machines or design them from templates, IT managers will build virtual machines and send them over the wire to run in the cloud. The cloud might be either inside or outside the enterprise, it doesn’t matter, as long as the application and its operating system arrives in the correct virtual machine format for the target cloud.

In other words, VMware’s Life Cycle Manager and VMotion, Vizioncore’s VConverter, Fortisphere’s Virtual Essentials, and other tools for generating, converting, and moving virtual machines around are the precursors to exporting virtual machines into the cloud.

Developers like me are sharing VM’s today using USB keys or drives between neighboring machines. We use  Windows network shares and nfs-mounted disks when we want to share VM’s between members of our team.  In the production environment, my IT colleagues use several other tools as have been documented in our blog earlier:

• scp, VSS for Windows VHD backup, disaster recovery
• rsync vm, vhd for backup, disaster recovery, ec2
• Using a USB drive to store & run virtual machines
• How to transfer Virtual Images between machines?

Watch this space for the launch of sharevm’s tools in early 2009 that will accelrate the convergence of virtualization and cloud computing.

After having gone through the learning curve of setting up an application on Amazon’s ec2, I found it interesting that Jumpbox has released 38 open source applications, including SugarCRM, mySQL, Drupal, etc, as virtual appliances available on the ec2 cloud. This is an interesting play because it leverages virtualization to offer software as a service in the cloud and is of great benefit to SMB’s who can use these services right away, when they don’t have the IT infrastructure (staff, hardware, data center, etc.) to host it themselves.

Smart move, Jumpbox, you are a hip startup – the convergence of virtualization, SaaS in the cloud should help raise your market valuation.

1. Open ElasticFox; Open tab AMI and Instances
2. Select an AMI Instance to suit your needs and launch it as described here
3. You will be shown the Launch New Instance dialog box
4. Select the Instance Type as small or large
5. Select the key pair you want to use. There might be just one in the dropdown if you are the only user, otherwise there can be one per member of your development team.
6. Select the Availability Zone in the same zone as your primary services and data. We chose us-east-1c because that is the zone where all our VM’s are currently running.
7. Select the appropriate Security Group, I chose pauls-sandbox, which was created as described here.
8. Press the Launch button to launch an instance of this AMI
9. The AMI instance is shown as running
10. Right click on the running AMI instance and select Connect to Instance. This should bring up a terminal window

You have got to remember at all times that Amazon EC2 is stateless, it will not remember things between one AMI instance to the next unless you have allocated persistent storage in Amazon’s Simple Storage Service (S3) and maintain state there, e.g., .bashrc, etc for logins.

1. Create a new volume in the Elastic Block Store (EBS) from the Volumes and Snapshots tab in ElasticFox
2. Attach this newly created volume as /dev/sdh with the AMI instance that you have just started
3. This causes the volume to be associated with the running instance

If you are a member of a development team that will be sharing code and data residing on shared storage in the cloud, you should create your own security group to isolate your VM and testing from others in your group. I learnt this the hard way when our development team was operating in the same security group and I accidentally terminated Harvey’s VM while he was rewriting the makefile. He lost all his changes.

1. Access the Security Groups tab in ElasticFox.
2. Create a new group and add new group permissions. I added a new group called pauls-sandbox and added group permissions for

• SSH (port 22),
• HTTP (port 80),
• HTTPS (port 443) and
• ICMP (port -1) for pings