shareVM- Share insights about using VM's

This is a summary of Kris Midgely’s (Founder and CTO, Unidesk) interview by Brian Madden

Unidesk is a PC Lifecycle Management company planning to provide

• Virtual Desktop Management
• Personalization
• Storage reduction

with no agent on the desktop.

Supports VMware ESX today. Intends to support Citrix XenServer and Microsoft Hyper-V,  VMware Workstation, VMware Fusion, Citrix XenClient. and application virtualization technologies such as, VMware ThinApp, Microsoft App-V, etc.

CacheCloud

CacheCloud: is a content delivery network (think Akamai) for pushing out VDI gold images to different data centers, laptops/desktops in branch offices or machines that connect intermittently. Cloud consists of  a large number of virtual appliances, called CachePoints, running one per blade or laptop. Each CachePoint stores user personalization locally as well as replicates it out. CachePoint appliances are made of Linux, have virtualized storage that supports

• thin provisioning
• replication
• versioning

Windows and app code is shared, user personalization is unique. This makes scanning for AV really fast since there is only image of code

Block-level replication of deltas, file-level replication for compositing.  Personalization data can be written from several individual CachePoints to a  NAS/SAN in the data center which enables legal discovery of changes to data, which was not possible until today.

Composite Virtualization

Composite Virtualization understands the abstract layers, Windows’, apps and user data and knows how to merge them together (composite) in real time to create a bootable C: device and provide a rich desktop experience. Virtualizes each desktop into  layers

• exe, com objects and dlls are apps
• Registry – configuration 
• everything else is data 

It will support encryption in the future: Shared keys for windows and apps code, personal keys for private data

Composting engine sits on top of the device driver and form the individual layers by merging individual IO streams with the namespace knowledge it maintains.

A virtualization storage layer implemented as a NTFS file system filter driver provides a high performance block IO device that talks to the CacheCloud. It loads early in the boot cycle. Once it is loaded, it loads a vmdk disk image which contains Just Enough Windows pre-composited to provide a bootable C drive. The latter can be served from the Cache Cloud.

It Snapshots the system automatically by auto detecting application installs/uninstalls, ActiveX control downloads. An admin can get a timeline view of user-installed software to reconstruct a hosed machine easily from the CacheCloud. Lets you recover system state while retaining your data.

Availability

Currently in Beta with 22 customers spanning Financial Institutions, Higher Ed and the Government.

Distribution through a channel strategy, working with Top Channel providers for VMware, Citrix, Microsoft. Can replace WAN acceleration, Backup and DR and Persistent Personalization products.

Here are the Top 12 referrers to our blog over the past 3 months, the numbers of referrals are in parentheses.

1. http://pro-linux.de/berichte/ext4/ext4.html (546)
2. http://dabcc.com/article.aspx?id=9653 (342)
3. http://networksecuritytoolkit.org/nst/index.html (110)
4. http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/ (59)
5. http://communities.vmware.com/thread/189804?tstart=0 (49)
6. http://techblog.41concepts.com/2008/03/31/shrink-your-windows-disk-image-on-wmware-fusion-mac/ (42)
7. http://blog.rightscale.com/2009/01/09/amazon-launches-ec2-console/ (37)
8. http://wordpress.com/tag/vhd/ (33)
9. http://wordpress.com/tag/vmdk/ (32)
10. http://virtualgeek.typepad.com/virtual_geek/2009/01/updated-homebrew-esx-hardware-list.html (32)
11. http://blogs.msdn.com/heaths/archive/2005/07/30/445621.aspx (32)
12. http://kakku.wordpress.com/2008/06/23/virtualbox-shrink-your-vdi-images-space-occupied-disk-size/ (31)

Thank you for the referrals. Hope the content is meaningful for our readers.

• How to convert a big VMWare VMDK to a Hyper-V VHD?
• WinImage VMDK to VHD converter for Microsoft Virtual PC and Virtual Server
• VirtualToolkit VMDK to VHD converter for Xen
• VHD to VMDK: There does not seem to be a demand for this.

I went through this process of discovery over the past three weeks and felt I should share my learning with you.

What is virtualization?

Virtualization is the technology that allows you to run several different OS’es, each of which is independently running on a dedicated (virtual) machine, concurrently on the same physical machine.

Wikipedia describes different types of virtualization

How is virtualization realized?

A hypervisor is the software layer that virtualizes the underlying host OS and hardware  to allow multiple operating systems to run concurrently on the same physical machine.  You can read about the hypervisor in depth here.

Which are the primary virtualization technologies in use commercially?

There are principally three arms vendors for virtual machines, VMWare, Microsoft and Xen. Their  technologies differ in subtle but important ways.

VMWare implements full virtualization. The VMWare hypervisor simulates the underlying x86 hardware completely so that Windows and Linux variants designed for the x86 architectcure can run unmodified within individual VMWare virtual machines. 

Microsoft and Xen implement paravirtualization. The hypervisor exposes a software interface to virtual machines that is similar but not identical to that of the underlying OS/hardware.  The calls supported by this interface are called hypercalls, analogous to the OS syscalls. Paravirtualization simplifies the design and implementation of the hypervisor and virtual machines that run on them can achieve performance closer to non-virtualized hardware. However, the downside is that operating systems must be explicitly ported to run on top of such a hypervisor.

Microsoft’s  Hyper-V hypervisor, introduced in Windows 2008,  supports Windows 2008, 2003, 2000 and SuSe Linux Enterprise Server 10  as guests,

How are these systems architected?

VMWare:

VMWare ESX

Solution Architecture

Micorosoft Hyper-V:

http://msdn.microsoft.com/en-us/library/cc768520.aspx

Xen:

How do the VM’s differ?

VMware uses the vmdk file format

The set of files necessary for VMWare VM

Microsoft and Xen use the vhd file format.

Experience with running out of storage

• Need to run lots of Virtual Machines on a laptop – but need more storage?

Background about sparse files, Windows NTFS and VHD

• NTFS Compression, sparse files and operations on them
• UNIX sparse files
• Be Careful with VHDs and Windows XP Compressed Folders
• Why do VHD’s grow?
• What is a VHD file, what is a disk image?

VHD Pre-Compactor & Compactor

When you create a new VM you can pre-allocate all the storage in one shot or let the storage grow on demand. In the latter case, the VM uses a sparse file

• Compact your Microsoft Virtual PC 2004 VHD Files
• Compressing Microsoft Virtual Server 2005 VHD files
• Virtualization with Microsoft Virtual Server 2005 by Roger Dittner, David Rule, Jr., Ken Majors
• How to compress Microsoft Virtual PC 2007 hard disks
• Compressing Microsoft Virtual PC 2007 Images
• Compressing Microsoft Windows Server 2008 Hyper-V disk images

VMDK Wipe & Shrink

• Wipe & Shrink
• Enable compression on individual .vmdk files
• Issues

3rd Party (Not from MSFT & VMW)

• Raxco Software
  • PerfectDisk for VmWare
  • for ESX
• Parallels Compressor
  • Desktop Version
  • Server Version

Rsync: for transferring files between machines

• Rsync Virtual Machine Backups
• Virtualize a server with minimum downtime

Research

• IZO: Applications of Large-Window Compressions to Virtual Machine Management (Mark et al. IBM Almaden, USENIX, Nov 2008)
• CZIP [ppt][paper]
• redundancy elimination within large collections of files
• Data Domain SISL
• Hyperfactor (IBM Data Depulication)
• Network Applicance

Experience with running out of storage

• Need to run lots of Virtual Machines on a laptop – but need more storage?

Background about sparse files, Windows NTFS and VHD

• NTFS Compression, sparse files and operations on them
• UNIX sparse files
• Be Careful with VHDs and Windows XP Compressed Folders
• Why do VHD’s grow?
• What is a VHD file, what is a disk image?

VHD Pre-Compactor & Compactor

When you create a new VM you can pre-allocate all the storage in one shot or let the storage grow on demand. In the latter case, the VM uses a sparse file

• Compact your Microsoft Virtual PC 2004 VHD Files
• Compressing Microsoft Virtual Server 2005 VHD files
• Virtualization with Microsoft Virtual Server 2005 by Roger Dittner, David Rule, Jr., Ken Majors
• How to compress Microsoft Virtual PC 2007 hard disks
• Compressing Microsoft Virtual PC 2007 Images
• Compressing Microsoft Windows Server 2008 Hyper-V disk images

VMDK Wipe & Shrink

• Wipe & Shrink
• Enable compression on individual .vmdk files
• Issues

3rd Party (Not from MSFT & VMW)

• Raxco Software
  • PerfectDisk for VmWare
  • for ESX
• Parallels Compressor
  • Desktop Version
  • Server Version

Rsync: for transferring files between machines

• Rsync Virtual Machine Backups
• Virtualize a server with minimum downtime

Research

• IZO: Applications of Large-Window Compressions to Virtual Machine Management (Mark et al. IBM Almaden, USENIX, Nov 2008)
• CZIP [ppt][paper]
• redundancy elimination within large collections of files
• Data Domain SISL
• Hyperfactor (IBM Data Depulication)
• Network Applicance