Determining the right size for your VMware Virtual Machine File System (VMFS) data stores is important, because...
once you size a data store, it's difficult to change it. Correctly determining data store size depends on several factors that you may not have considered. It's not just about the projected total size of the number of virtual disk files you want to put on a single logical unit number (LUN). You also need to account for the companion files that make up a virtual machine (VM).
When you perform certain actions with your virtual machines -- such as suspending, powering on and creating snapshots -- you create companion files. Don't put too many virtual machines on a single volume, as this can undermine performance because of input/output (I/O) contention and LUN locking.
In this article, I outline all the factors to consider when determining VMFS data store size and then give a step-by-step formula for determining an appropriate data store size for your virtual infrastructure.
Determining VMs per LUN
Using multiple LUNs to extend a VMFS data store is not recommended. Instead, I recommend creating a single LUN to use for your VMFS volume, which is why determining the proper size beforehand is important. So what's the magic number of VMs for a single LUN? Well, there isn't one, but a general rule of thumb is 14 to 16 VMs per LUN ,which varies depending on how much disk I/O you VMs generate and how often you use snapshots. If you have VMs with mostly low disk I/O, such as Web and application servers, you could put more VMs on a single LUN. Similarly, if you have VMs that have high disk I/O, such as email and database servers, put fewer VMs on a single LUN. Another argument for placing fewer VMs on a LUN is when you use snapshots frequently and let them run for any length of time. Growing snapshots cause a host VM to briefly claim exclusive access to the LUN when the VMkernel applies changes to the VMFS metadata, also known as SCSI reservations. This occurs so that multiple hosts cannot concurrently write to the metadata, which would corrupt VMFS. Once the update completes the lock is released. Snapshots grow in 16 MB increments; every time a snapshot grows it causes a SCSI reservation. Some additional less frequent operations that cause SCSI reservations are the following:
- moving a VM with VMotion;
- creating a new VM or deploying a VM from a template;
- powering a VM on or off;
- creating a template; and
- creating or deleting a file (snapshots included).
Determining LUN size
Once you decide how many VMs to place on your data store, then determine the size of the LUN that you will use. It's not a good idea, however, to add up the sizes of the virtual disks; many other files will reside on your data stores. You'll find a list of these files and their projected sizes below. Let's first review these files that will exist for each VM and their projected sizes:
- The .vswp fileis a virtual swap file that equals the size of the memory assigned to a VM, minus any memory reservations assigned to a VM, which by default is zero. For example, if you assign a VM 4 GB of memory, a 4 GB .vswp file is created when a VM is powered on and deleted when it is powered off. If you create a memory reservation equal to 3 GB, only a 1 GB .vswp file is created. Likewise, if you create a 4 GB reservation (which in most cases is not recommended), a zero byte .vswp file is created.
- The .vmss file is created only when a VM is suspended and equals the size of the amount of memory allocated to a VM.
- The .vmsn file is used to store snapshot states (including memory if you choose to include it) when a snapshot is created, and whose size is equal to the amount of memory assigned to a VM. If you choose not to store snapshot memory states, this file will be very small (less then 1 MB).
- The –delta.vmdk files are snapshot data files that will start as 16 MB and grow in 16 MB increments as disk changes are made to a VM. These files cannot exceed the size of the original disk file. Their rate of growth depends on the number of changes that are made to the original disk, which will depend largely on the application that runs on a VM. Applications that are fairly static such as Web and application servers usually do not have a lot of data change and therefore should have fairly small, slow-growing snapshots. Email and database servers, however, are likely to have many disk writes, and their snapshots grow quickly.
- The rest are miscellaneous files that are usually small and do not take up much room on VMFS volumes. These files include the .nvram file (BIOS), .vmx file (config), .vmsd file (snapshot metadata) and .log files. For these files, 50 MB per VM is typically enough. You can also control the number and size of log files with advanced VM parameters.
To wrap it all up, here are the steps to determine VMFS disk size.
- Add up the virtual disk size of all the virtual machines you plan on placing in a data store.
- The second step involves separate steps. Add up the total amount of memory allocated to the VMs (step A), then add up the total size of memory reservations that you will assign to a VM (step B) (remember the default for memory reservation is zero). Now subtract the total size of the memory reservations from the total amount of memory allocated to the VM to get the amount of space that you need for the .vswp files (step A minus step B). Alternately, you can configure your host to store .vswp files on a local data store. If you choose to do so, you don't need to include this number when you add up your totals.
- Add about 50 MB per VM for miscellaneous files.
- If you plan on suspending VMs, calculate approximate disk space size needed by multiplying the maximum number of VMs you expect to suspend concurrently by the largest amount of memory assigned to a single VM.
- The next step also involves a few steps. Now calculate how much space you need for snapshots. This is a rough estimate based on a few factors, and I recommend estimating on the high side. Start by approximating the maximum number of snapshots that you will run concurrently (step A), calculate the average size of all your virtual disks in gibabytes (step B). Now use a percentage multiplier based on how long you plan on keeping the snapshots and how quickly you expect them to grow (use 20% as a low number, 40% as a medium number and 60% as a large number) (step C). Then multiply (A * B) * C) to calculate the number of gigabytes in disk space that you should leave free for snapshots.
If you plan on including the memory state with snapshots then multiply (step A) times the largest amount of memory assigned to a single VM to calculate the additional disk space needed.
- Finally, I recommend allocating extra space for unexpected events and operations to ensure that you do not run out of space on your data store. When you have multiple snapshots running on a VM and you delete them all at once, the extra space comes in handy. That's because extra space is needed to commit (or delete) the snapshots back into the original disk. Add another 25 GB for this.
To visualize this, use the spreadsheet that is available from my website VMware-land.com. A screenshot is available below.
To conclude, these numbers are general estimates. Snapshots are the main factor that increases or decreases the amount of disk space your environment needs, and it's hard to predict how large a snapshot will grow. These guidelines should assist you in sizing your data stores properly, but remember, it is best to err on the side of caution – in this case, with more disk space. Once you create a VMFS volume, you cannot increase its size without using methods that are not recommended.
ABOUT THE AUTHOR: Eric Siebert is a 25-year IT veteran who specializes in Windows and VMware system administration. He is a guru-status moderator on the VMware community VMTN forums and maintains VMware-land.com, a VI3 information site. He is also the author of the upcoming book VI3 Implementation and Administration , which is due out in June 2009 from Pearson Publishing. Siebert is also a regular on VMware's weekly VMTN Roundtable podcast.