«

»

Mar 01 2013

Clearing up Space-Efficient Virtual Disk questions

Horizon View 5.2 make use of the new Space-Efficient Sparse Virtual Disks available in vSphere 5.1. I previously wrote about it in Space-Efficient Sparse Virtual Disks and VMware View and announced the new Horizon View 5.2 feature at What’s New in VMware Horizon View 5.2 (Beyond Marketing).

 

The new SE sparse disk implements a space reclaim feature to reclaim blocks that were previously used but now are unused on the guest OS. These are blocks that were previously written but currently are unaddressed in a file system/database due to file deletions, temporary files, and so on.
There are two steps involved in the space reclamation feature: The first step is the wipe operation that frees up a contiguous area of free space in the virtual machine disk (VMDK); the second step is the shrink, which unmaps or truncates that area of free space to enable the physical storage to be returned to the free pool.

Wipe
• Initiate a call to VMware Tools to scan the guest OS file system.
• Mark the unused blocks as free.
• Run the SCSI UNMAP command in the guest to instruct the virtual SCSI layer in the VMkernel to mark the blocks as free in the SE sparse disk.

Shrink
• SCSI device – VMware® ESXi™ issues an SCSI UNMAP command to the array.
• NFS device – ESXi issues an RPC call to TRUNCATE the file.

The wipe operation is initiated via an API call to VMware Tools. VMware Tools initiates a scan of the guest OS to find stranded space and mark the file system blocks as free. The first SCSI UNMAP operation is then run from within the guest OS, instructing the VMkernel as to which blocks can be reclaimed. The VMkernel captures these SCSI UNMAP commands and does not pass them through to the array. When the VMkernel detects which blocks are free, it uses its virtual SCSI layer to reorganize the SE sparse disk by moving blocks from the end of the disk to unallocated blocks at its beginning. This creates a contiguous area of free space within the VMDK. The shrink operation then sends either an SCSI UNMAP command (for SCSI disks) or an RPC TRUNCATE command (for NFS) to the array to free the space.”

 

Last week during VMware Partner-Exchange in my session EUC1222 that I co-presented with Robert Baesman (@rbaesman) we also briefly discussed the new disk format. Few questions came up from the audience and I would like to also address them here.

 

  • This new feature requires vSphere 5.1 as well as hardware version 9 for the VMs.
  • The Space Efficient format is only for for linked clone pools; and only for the OS disk.
  • Horizon View automatically chooses the space efficient format when possible and there is no official way to change this behavior.
  • The Space Efficient Format is not yet supported for Windows 8.
  • The space reclamation process is kicked off automatically by Horizon View when the reclaimable space for the VM exceeds the threshold set by the administrator.
  • The space reclamation statistics (reclaimed amount, reclaimed time) are displayed in the Pool Summary, and Desktop Summary.
  • SESparse disks ARE COMPATIBILE with the Horizon View Storage Accelerator technology (Understanding CBRC) from View 5.1 / vSphere 5.

 

If you want to test the new Space-Efficient Sparse Virtual Disks feature in Horizon View 5.2 here is a quick walk-trough to configure and run execute the space reclamation process.

 

  1. Ensure vSphere 5.1 is installed.
  2. Create a Windows XP or Windows 7 VM in vSphere using the latest virtual hardware version, install the Tools & View Agent, and snapshot it to serve as a base image for a View Composer based View Desktop.
  3. Install and Configure the View Connection Server appropriately  and ensure that space reclamation is enabled for vCenter.
  4. Set up a space reclamation blackout time in Horizon View.
  5. Create a new Automated, Dedicated, Horizon View Composer linked clone based pool and provision & entitle 1 or more desktops in the pool.
  6. Ensure that space reclamation is enabled for the desktop pool.
  7. Run a workload in a desktop in the pool that creates and deletes lots of data in the Windows file system, or manually copy and delete large files.
  8. Observe growth of the VM’s disk over time in VC.  You can see this in VM in VC under Summary -> Resources -> Used Storage.  Make a note of how much it has grown.
  9. Re-observe the VM’s disk consumption shrink following a reclamation.  You can see this event occur by either:
    (a) Waiting up to 1 hour for the automated process to kick in
    (b) Kicking the reclamation process of manually using the vdmadmin command

 

 

This article was first published by Andre Leibovici (@andreleibovici) at myvirtualcloud.net.

1 comment

2 pings

  1. VMware for Beginners

    Hi Andre – this is very good stuff! Keeping the datastore clean is a routine task that should be covered in an ops plan or during a regular health check. Orphan files, old snapshots and zombie VMs can waste good vdisk space.

  1. How to change Horizon View SESparse to VMFSSparse » myvirtualcloud.net

    […] my article Clearing up Space-Efficient Virtual Disk questions I explained the most important requirements and constraints when the new SESparse VMDK disk format […]

  2. Welcome to vSphere-land! » Storage Links

    […] User Sizing Example (EMC) Space-Efficient Sparse Virtual Disks and VMware View (My Virtual Cloud) Clearing up Space-Efficient Virtual Disk questions (My Virtual Cloud) Managing storage for virtual desktops (Storage Magazine) VDI and Storage: Deep […]

Leave a Reply