Share Virtual Machines

Our readers are primarily asking questions like:

• How can I free disk space, on Windows, and on ext4, ext3 on Ubuntu and Linux, within virtual disks like vmdk, vhd and vdi?
• Where can I find the best virtual appliances/ Top 10 virtual appliances?
• How can I convert from one virtual disk (vmdk to vhd, or vdi to vhd) to another?
• Who are the competitors for ec2?

An analysis of the search terms shows interesting clusters:

If I abstract it out, our readers are primarily interested in learning how to free disk storage and where to find the best / Top 10 vmware, Xen and Windows virtual applainces.

Thank you. I appreciate your interest in this blog.

I have reviewed several appliances in the Secure Content and Threat Management and Identity, Access and Vulnerability Managament categories of the VMware Appliance Marketplace to identify the Top 20 appliances. Here are the steps I followed for selecting the appliances listed below:

• I have relied on the Average customer Rating, expressed as a 5 star, or a 4 star, etc., rating (you may wish to review my analysis of VMware’s ratings)
• I discarded all virtual appliances that solely package OS distributions, primarily, ubuntu, fedora, etc. My rationale is that an OS by itself provides low business value to an IT Administrator. While an IT administrator can use these just as if they were using a ghost image, these virtual appliances neither package applications in a usable form, nor simplify the task of installing and configuring the applications that provide business value. Besides, the base OS virtual appliances are available in a category by themselves
• I also discarded several appliances rated 4 star or less, which are present in the directory but have either broken or stubbed out download links. They seem to have been retained in the directory to beef up the appliance count, however, they are not useful to the community.

Which virtual appliance do you use the most and why do you like it?

I have reviewed several appliances in the IT Administration and Systems Infrastructure categories of the VMware Appliance Marketplace to identify the Top Ten and more appliances. Here are the steps I followed for selecting the appliances listed below:

• I have relied on the Average customer Rating, expressed as a 5 star, or a 4 star, etc., rating (you may wish to review my analysis of VMware’s ratings)
• I discarded all virtual appliances that solely package OS distributions, primarily, ubuntu, fedora, etc. My rationale is that an OS by itself provides low business value to an IT Administrator. While an IT administrator can use these just as if they were using a ghost image, these virtual appliances neither package applications in a usable form, nor simplify the task of installing and configuring the applications that provide business value. Besides, the base OS virtual appliances are available in a category by themselves
• I also discarded several appliances rated 4.5 star or less, which are present in the directory but have either broken or stubbed out download links. They seem to have been retained in the directory to beef up the appliance count, however, they are not useful to the community.

Which virtual appliance do you use the most and why do you like it?

During the past week, I was analyzing the recently revamped VMware Appliance Marketplace. My analysis is summarized below:

Salient Statistics

• 9% are top-rated, i.e., have a Star Rating of 5
• 41% have a Star Rating of between 3 and 4, i.e., this cluster may be providing the most value to its users, even though it is not top-rated.
• There is a steep drop-off in the number of appliances that have a Star Rating below 3
• 31% are unrated, which seems to indicate that a significant majority has been submitted recently
• Only 3% (40/1227) have Reviews – this is a very small number. The number of reviews have to grow substantially to indicate that a vibrant community, like Amazon.com reviews, has formed here

Transparency

I feel there is a greater need for transparency. What do the Star Ratings mean?

• Are they correlated with the number of downloads for that appliance?
• What is the impact of reviews on these ratings?

Beware Star Ratings alone! The following appliance rated 4.5 has 4 reviews, read them and you will know what I mean.

As a member of VMware’s Technology Network community, I will urge VMware to provide greater transparency around how these ratings are computed/awarded. It will greatly help the users understand the ratings system.

What do you think?

Data

I had observed in my earlier article that DropBox performs de-duplication in the cloud. This would mean that de-duplication is not performed at the client. In order to test my hypothesis, I performed the following experiment:

I first looked at the size of the DropBox folder on Windows and found it to be 1,723,871,232 bytes.

Next, in the DropBox client, I opened the DropBox folder and simply duplicated the contents of the Public folder by copying the 1.68MB file and pasting it as its copy. I looked at the size of the folder once again and it had doubled to 3,446,513,664 bytes.

If DropBox had been performing dedup at the client, then it should have detected the duplicate blocks between the parent and its copy at source and the folder should not have grown in size at all. As a result, my conclusion is that DropBox dedup’s only in the cloud but not at the client.

Wait, there’s more:

I repeated the same experiment on the Mac after deleting the duplicate file. Here’s what I started out with:

Last login: Thu Sep 17 15:30:58 on ttys000
mace1s:~ paule1s$ du -k DropBox
1152 DropBox/Photos/Sample Album
1516 DropBox/Photos
1682636 DropBox/Public
368 DropBox/sharevm
1684880 DropBox

Notice that the total size of the folder (the last line of the listing above) is 1.68GB.

Next, in the DropBox client, I opened the DropBox folder and simply duplicated the contents of the Public folder by copying the 1.68MB file and pasting it as its copy. I looked at the size of the folder once again and saw:

mace1s:~ paule1s$ du -k DropBox
1152 DropBox/Photos/Sample Album
1516 DropBox/Photos
2600140 DropBox/Public
368 DropBox/sharevm
2602384 DropBox

This is very interesting. I had expected the storage requirements to double to 3,369,760 however, they grew by approx. 1GB. What happened to the remaining 682MB? Did the DropBox client truncate the file? If so, why?

Readers, can you shed some light?

1) Software as a Service (SAAS)
Salesforce.com (sales application)
wush.net (svn)

2) Platform as a Service (PAAS)
a) Java
Google App Engine
b) Rails
Joyent
Heroku
EngineYard
c) Python
Google App Engine
http://wiki.python.org/moin/PythonHosting
d) .net
Azure

3) Infrastructure as a Service (IAAS)
Amazon EC2

I found Dropbox, a nifty service for storing files online, keeping their copies on several of your own computers in sync, or sharing some of them with your friends.

• You download the Dropbox client (supported on Windows XP and Vista (32 and 64-bit), Mac OS X Tiger and Leopard, as well as Ubuntu 7.10+ and Fedora Core 9+)
• 2GB of free storage provided with it
• You can then drag and drop files that you want to store online or share into the Dropbox.
• Dropbox maintains a snapshot of files
• If any of the files get updated, it sends only blocks that have changed
• It also offers the ability to undelete and restore files from the copies that are stored online.
• You can create Public folders for sharing, files in Public folders have URL’s that you can share with your friends.

While the company seems to be consumer-focused, the service is usable for dull and boring corporate stuff, like instantaneous automatic backups of files that change and also enables disaster recovery.

Someone has used Dropbox for syncing and sharing VM’s. This is an interesting use case, however, readers should pay heed to the transfer times as image sizes grow

Here are the Top 12 referrers to our blog over the past 3 months, the numbers of referrals are in parentheses.

1. http://pro-linux.de/berichte/ext4/ext4.html (765)
2. http://networksecuritytoolkit.org/nst/index.html (566)
3. http://dabcc.com/article.aspx?id=9653 (149)
4. http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/ (111)
5. http://kakku.wordpress.com/2008/06/23/virtualbox-shrink-your-vdi-images-space-occupied-disk-size/ (101)
6. http://stumbleupon.com/refer.php?url=http://sharevm.wordpress.com/2009/01/19/most-popular-vmware-virtual-appliances-for-it-administrators/ (84)
7. http://techblog.41concepts.com/2008/03/31/shrink-your-windows-disk-image-on-wmware-fusion-mac/ (67)
8. http://thedarkmaster.wordpress.com/2007/03/12/vmware-virtual-machine-to-virtual-box-conversion-how-to/ (66)
9. http://blogs.msdn.com/heaths/archive/2005/07/30/445621.aspx (66)
10. http://prefetch.net/blog/index.php/2007/01/21/determining-file-fragmentation-on-ext3-file-systems/ (61)
11. http://virtualgeek.typepad.com/virtual_geek/2009/01/updated-homebrew-esx-hardware-list.html (52)
12. http://blog.rightscale.com/2009/01/09/amazon-launches-ec2-console/ (53)

Thank you for the referrals. Hope the content is meaningful for our readers

I received the email annoucement from Amazon ec2 earlier today:

You can read the detailed announcement here and also download the AWS Toolkit for Eclipse.

Amazon’s Elastic Computing Cloud (ec2) is offering over 1500 AMI’s (virtual appliances). It’s recent foray into Europe highlights its aggressive pursuit of a land grab strategy. So, what has its competition in the US been up to? Every major web hosting vendor has announced a cloud initiative:

• The Rackspace cloud is called Mosso. Its prices compare favorably with Amazon’s. Its pricing calculator and spreadsheets can be found here.
• Savvis has a VMWare ESX based offering for Windows and Linux
• AT&T is also offering VMWare ESX based hosting Windows, Linux and Solaris x86. They intend to offer Citrix XenServer and Microsoft Hyper-V in the future. AT&T offers SLA’s for cloud services, a first.
• Terremark has also launced an Enterprise Cloud, which incidentally is a utility computing cloud sans virtualization
• IBM’s cloud initiative has its products available as AMI’s on ec2.
• Sungard has not yet publicly announced its cloud plans.
• Verizon is slated to announce its cloud initiative in Summer 2009

Amazon’s ec2 is nearly three years old now (Mar 2006) and competition has started appearing on the horizon just over the past year. If you look at the timescale of announcements,

• Rackspace’s Mosso is just over a year old (Feb 2008),
• Terremark’s cloud is nine months old (June 2008),
• AT&T’s cloud is seven months old (Aug 2008),
• IBM’s AMI launch on ec2 and Savvis’ cloud is three weeks old (Feb 2009).

Since ec2 was first to market, the Rackspace, AT&T and Savvis cloud offerings have a “me too” feel to them. However, unlike other vendors, Rackspace has published pricing on the Web and it appears to be very competitive with ec2. 

Unlike Amazon, Savvis and AT&T are going the buy versus build route to get fast time to market. They are initially launching their service using VMWare technology put together using Professional Services instead of following Amazon’s approach of building a proprietary infrastructure using Open Source software as its foundation. In fact, this may be the preferred route amongst the upper echelons of cloud service providers. There is optimism that providing cloud services is a growth business. I am noticing startups like Enomaly (“Build your own private elastic cloud”) and VMOps (“launch ec2 today”, aka public cloud) offering cloud infrastrcture products.

In conclusion, credible competition is emerging and there are real alternatives to ec2 available today. However, given the state of the economy, I think the market will begin to form by 2010 and should reach critical mass by 2011-2012.