«

»

Apr 20 2011

VDI Base Image: The Missing Step

How far can you go when creating the perfect base image for your VDI deployment? I haven’t stopped searching for the Holy Grail, however I have recently figured out that a very critical step in the base image creation was missing, forgotten or had not been discussed yet or enough.

However, before I give you the cooking recipe let’s dig into the semantics of the problem.

During tests with the Linked Cloning technology on VMware View 4.6 (although the example below applies to any version) I noticed that the Replica disks and Linked Clones were using more storage space than they should, or had been provisioned with.

Observing the size of .vmdk’s I noticed that Replica disk sizes were much bigger than Windows NTFS used space was showing.

I won’t explain how Linked Cloning works in this article, but feel free to read my article VMware View 4.5 Linked Cloning explained for a better understanding. If you haven’t catch-up on my previous articles – the size of a replica disk is equal to the disk space consumed in the Parent SOE.

So, if the above is true why does the Replica size doesn’t match the NTFS used space?

 

Provisioned VM Size: 31.99GB (33,554,430)
Replica Size: 19.10GB (20,033,540)

NTFS Provisioned: 31.99GB
NTFS Used: 14.4GB
NTFS Available: 17.5GB

(Real life example above)

To find out the reason of this disparity I filled up the Windows NTFS partition with data until nothing else would fit, not even a 1KB file. At this point Windows file system had 0KB free space. When I looked at the Delta it was 17.52GB, and that matches with the NTFS free space from above example. Therefore:

 

Replica (19.10GB) + Delta (17.52GB) = 36.62GB

 

Discounting additional files like swap and logs, there was an additional 4.63GB if compared to the total provisioned for the VM that was 31.66GB. In Summary, the replica is 4.63GB bigger than the real data in the Windows NTFS partition.

 

Why?

In normal case, a full clone VM’s .vmdk file is always large than the actual NTFS usage. This is because that once a .vmdk file (in Thin format) grows up, it does not shrink back when NTFS released some space, unless the user shrinked the .vmdk files explicitly.

If a replica was created before such "shrinking" was performed, then the size of the copied disk of the replica will be larger than the actual NTFS used space. When a linked clone is created from such a replica attempt to write data into these NTFS free space, the update will be written into the delta disk. These writes could simply have been executed by Windows swap files or Logs.

 

Consequences

This may not seem like a big issue at first, however in certain situations it could use storage space that was not initially provisioned, costing projects additional money that was not originally accounted for. I used my VDI Calculator to simulate a scenario with 2,000 VMs and a maximum of 2 concurrent snapshots per desktop pool (only 4 pools).

 

Replica Size: 8GB
Total Storage for Replicas: 2.20TB

Replica Size: 8GB + 4GB (overhead):12GB
Total Storage for Replicas: 3.30TB

 

Just making sure that Parent VMs have been properly shrunk you could be saving few gigabytes or even terabytes depending on the size and design of the VDI solution.

In a scenario with more desktop pools and more LUNs these numbers could be even bigger.

** The calculations above only reflect deployments NOT using Dedicated Replica Datastores. When using Dedicated Replica Datastore the difference was from 70MB to 100MB.

 

Solutions

To avoid datastore space to be wasted on unshrinked free space copied by the Replicas disks, always shrink the Parent VM’s virtual disks before taking the snapshot to initiate the linked cloning process.

One of the methods to shrink a .vmdk disk is using vmware-vdiskmanager.exe utility. VMware Virtual Disk Manager is a utility in VMware Workstation that allows you to create, manage and modify virtual disk files from the command line or within scripts.

To shrink a virtual disk, it must be located on a Windows host. Before you can shrink the virtual disk, you need download the virtual disk for shrinking. Then use a command like the following:

vmware-vdiskmanager -k myDisk.vmdk

The easy way is to use VMware Tools installed in the parent VM. Shrinking a virtual disk reclaims unused space in the virtual disk. It means that, if there is empty space in the disk, this process reduces the amount of space the virtual disk occupies on the host drive.

To shrink a virtual disk:

1. Launch the control panel.

Windows guest — double-click the VMware Tools icon in the system tray, or choose Start > Settings > Control Panel, then double-click VMware Tools.

2. Click the Shrink tab.

clip_image002

3. Select the virtual disks you want to shrink, then click Prepare to Shrink.

Be aware that you cannot shrink a virtual disk if

  • You preallocated disk space when you created the disk. Preallocating disk space is the default option for both typical and custom virtual machine creation paths.
  • The virtual machine has any snapshots.
  • The virtual machine contains physical disks.
  • The virtual disk is not an independent disk in persistent mode.
  • The virtual disk is stored on a CD-ROM.

Similar Posts:

Permanent link to this article: http://myvirtualcloud.net/?p=1877

14 comments

4 pings

Skip to comment form

  1. Mike Holgate

    Andre,
    You have an uncanny ability to hit on subjects that just happen to be discussions/issues within my current circle!! Another great article that will be very helpful for my project!

    Mike

  2. Corné

    André,
    Well done on getting to the crux of it. I remember the filling up and megabyte maths.
    Good article!
    Corné

  3. VMMikeC

    very interesting article Andre. I’m using linked-clones but never really setup my VMs at Independent(and persistent) disks. I’m in the process of recreating my ParentVM. Maybe this is a good time to start setting that disk option?

    I run a small environment, < 100 clones

  4. VMMikeC

    I guess the more that I think about it. If the disk is set to independent and persistent, snapshots are unavailable, so there’s no way to protect the parentVM from lets say a bad MS update or software install

  5. Andre Leibovici

    @Corné
    Thanks. Keep in touch and let me know you make to VMWorld this year.

  6. Andre Leibovici

    @Mike Holgate
    I swear it’s not on purpose :-)
    Thanks for the feedback and support.

  7. Alastair Cooke

    You can reclaim the free but not zeroed space in your Master VM using SDelete inside the VM, this lets you keep the VM on ESX where it needs to be to be the master for the View Pool.

    I’ve written it up here: http://www.demitasse.co.nz/wordpress2/2011/04/bloated-replicas-or-deleted-isnt-zeroed/

  8. Andre Leibovici

    @Alastair Cooke
    Thanks for that Alastair. I thought about SDelete and even discusssed with few VMware colleagues, however I didn’t include in the article because I didn’t have time to test it with view composer. Thanks for validating the use of SDelete. This is the best way to proceed and cleanup the Parent VM.

    One thing concerns me: what If you are using Disposable disks? Would be good then to get rid of the page file before creating the replica.

  9. Andrew

    You also cannot shrink the disk if it is thin provisioned.

    I tried using the sdelete option mentioned and it increased the vmdk to the provisioned size. In my case it went from 16GB to 30GB

  10. Andre Leibovici

    @Andrew
    It is good practice not to use Thin Provisioned disks for Parent VMs. Instead create them eagerzerothick. Atter you use SDelete you need to delete the file created and then run the shrink process.

  11. Andrew

    I have done some testing and have found that although the master image is thick provisioned, once the shrink process has been run using VMware Tools, I needed to clone the master to another VM and use the thin provisioned option to reclaim space. If you do not do this and snapshot the original master, the replica uses the same disk format as the master i.e. thick so no space savings are seen. I had this issue using the VMware tools and sdelete process although I did find that using sdelete it still created a thin provisioned disk using the full disk allocated so I did not see any reclaimed space.

  12. Andre Leibovici

    @Andrew
    Thank you for the very usefull information and time spent in R&D.

  13. Andrew

    The issue I have encountered is related to VAAI and similar block sizes for datastores.

    Please refer to Alastair Cooke’s blog which was a result of our findings.

    http://www.demitasse.co.nz/wordpress2/2011/08/vaai-consideration-with-view-composer/

  14. Andre Leibovici

    @Andrew
    I have seen it before.
    I have a workaround for that without having to change block sizes and will writing about that soon.

  1. Bloated Replicas, or “deleted isn’t zeroed” | Demitasse

    […] posted recently about an issue that arises with VMware View Composer pools where a replica, to which a group of desktop VMs are anchored, has a larger than expected storage […]

  2. VDI Base Images | Wall's View

    […] has a great article over on myvirtualcloud.net (http://myvirtualcloud.net/?p=1877) about creating the “perfect” base image for VDI.  Good read with links to other […]

  3. - Cliff Davies

    […] and customization. I recommend reading Mastering VDI Templates updated for Windows7 and PCoIP and VDI Base Image: The Missing Step for additional information on how to profile your guest […]

  4. myvirtualcloud.net » Should I use storage DRAM for VDI deployments?

    […] and customization. I recommend reading Mastering VDI Templates updated for Windows7 and PCoIP and VDI Base Image: The Missing Step for additional information on how to profile your guest […]

Leave a Reply