VMware VMDK shrink and storage thin provisioning 101

How do you free up storage space in a virtualized IT environment? Get answers on VMware VMDK shrinks and thin provisioning of virtual machines.

Most of us don't spend time thinking about storage -- until we're running out of it. The same can be true in the virtualized computing environment. If your virtual closet can't hold one more pair of shoes, it might be time to think about VMware VMDK shrink and thin provisioning.

Working with the VMware virtual machine disk file (.vmdk) format specification is an important part of operating VMware-based virtual machines (VMs). A given VM's entire constituent data resides in this single disk file.

VMDK shrink is the equivalent of a thorough closet cleaning, while storage thin provisioning helps keep virtual machines from grabbing more storage space than they need in the first place.

How can I know how much storage a VM really needs? I have limited storage available for the virtual infrastructure.

While made with the best intentions, the estimate of how much storage a new VM needs cannot be 100% accurate. When a VM is provisioned, the admin can create a thick or thin disk. Thick provisioning allots a set quantity of storage for that VM, while thin provisioning only allocates storage space to a VM over time when it is actually required. VMDK thin provisioning is useful for virtual operations that are short on storage space.

Will storage thin provisioning benefit my storage hardware at the expense of VM performance?

You may have heard that thin-provisioned VMs can't perform as well as those with fixed data sizes, but this is a myth. More important than the input/output operations per second (IOPS) volume on a VM is how the admin handles that VM. Performance concerns are quick fixes; a large block size, for instance, reduces the IOPS generated by the VMDK as it grows. Beware of these seemingly innocent VM tasks that can balloon a thin disk and wreak havoc on the IT infrastructure. VMware admins also need to keep an eye on capacity alerts, because thin-provisioned VMs grow over time.

Will thin-provisioned VMs shrink automatically just as they would grow automatically?

While thin provisioning adds storage as needed, it does not work in reverse, removing unneeded storage for the VM. Windows deletes a file by removing the references/pointers in the file allocation table, not by removing the actual data, and "non-zero" values are left sitting on storage space. Some storage providers have developed automated methods to combat this, but you may need to perform a VMDK shrink depending on your setup. Typically, VMDK shrink is performed prior to exporting the disk.

I know a given VM is not using all its available storage, but the VMDK shows limited space remaining. How is this space being wasted?

Space is not automatically freed up when files are deleted from the VMDK. If you are performing many read/write and read/delete events, the VM is likely sitting on excessive empty but non-zeroed space. You may need to shrink a VMDK to prevent wasted storage space or to reduce a VM to its true storage footprint before a conversion/export.

What's the best way to shrink a VMDK?

Two popular tools are available for VMDK shrink: SDelete and Shrink Disks, a VMware Tools offering. VMware Tools is a free set of drivers, services and user processes that's installed in a guest OS. VMware Tools' shrink functionality cannot be scripted, so the operator needs to run this process. To run a VMDK shrink, VMware Tools stuffs the virtual disk with zero-filled files. If you have to interrupt or stop the shrink for any reason, these zero-filled files are left behind.

SDelete, a command-line tool that Mark Russinovich created and Microsoft offers for free can be scripted and performs a VMDK shrink differently than VMware Tools. SDelete random-fills a hard drive to delete data securely. It allocates the largest file and cached file it can, hogging free space on the disk to perform a secure cleanse without affecting any operating files.

A third option was suggested by blogger vNoob: a PowerCLI script. Using this method, a script connects to the data store, grabs and copies the VMDK header file locally, changes the file and copies it back up. After this script is complete, a cold storage migration is needed for it to take effect. Be sure to read over vNoob's disclaimers before trying this method and test it on a clone first.

Do I still need storage lifecycle planning with a virtualized environment?

Storage lifecycles are not as painful with virtualized environments, but they're still important to ensure adequate performance and availability for each logical unit number. Because storage provisioning is simpler, IT admins may face storage sprawl. They should remember that virtual storage is still based on physical storage hardware and is not infinite.

Dig Deeper on Creating and upgrading VMware servers and VMs