VMware admins should know the components of virtual machines. Understanding the files that make up a virtual machine can help admins decide what files are unnecessary and clean them out, and ease other management tasks.
Once you understand virtual machines (VMs) from a hardware perspective, you can study the components that make up a VM on an ESX/ESXi host. These are the various VMware file types associated with a VM, located in the VM's directory on the host (represented in the illustration below).
VMware virtual machine files are organized in the Virtual Machine File System (VMFS). If you look at the list of files associated with the VM -- use an SCP-based tool or follow VMware's recommendations -- you'll notice that most of the files start with the actual name of the VM, followed by different file extensions that denote the file type. You may not see all of the possible file types in the VMFS until your VM is in a certain state. For example, the .vswp file is only present when the VM is powered on and the .vmss file is only present when a VM is suspended. Below is a typical VM directory listing (using WinSCP).
So what are all these VMware file types and what are they used for? Here's each file type in detail.
The .nvram file. This small file contains the BIOS that is used when the VM boots. It is similar to a physical server that has a BIOS chip that lets you set hardware configuration options. A VM also has a virtual BIOS that is contained in the NVRAM file. The BIOS can be accessed when a VM first starts up by pressing the F2 key. Whatever changes are made to the hardware configuration of the VM are then saved in the NVRAM file. This file is in binary format and if deleted it will be automatically re-created when a VM is powered on.
The .vmx file. This file contains all of the configuration information and hardware settings of the virtual machine. Whenever you edit the settings of a virtual machine, all of that information is stored in text format in this file. This file can contain a wide variety of information about the VM, including its specific hardware configuration (i.e., RAM size, network interface card info, hard drive info and serial/parallel port info), advanced power and resource settings, VMware tools options, and power management options. While you can edit this file directly to make changes to a VM's configuration, don't do this unless you know what you are doing. If you do make changes directly to this file, make a backup copy first.
VMDK files. All virtual disks are made up of two files, a large data file equal to the size of the virtual disk and a small text disk descriptor file, which describes the size and geometry of the virtual disk file. The descriptor file also contains a pointer to the large data file as well as information on the virtual disks drive sectors, heads, cylinders and disk adapter type. In most cases these files will have the same name as the data file that it is associated with (i.e., myvm_1.vmdk and myvm_1-flat.vmdk). You can match the descriptor file to the data file by checking the Extent Description field in this file to see which -flat, -rdm or -delta file is linked to it.
The different types of virtual disk data files that can be used with VMware virtual machines are:
The -flat.vmdk file
This is the default large virtual disk data file that is created when you add a virtual hard drive to your VM that is not an RDM. When using thick disks, this file will be approximately the same size as what you specify when you create your virtual hard drive. One of these files is created for each virtual hard drive that a VM has configured, as shown in the examples below.
The -delta.vmdk file
These virtual disk data files are only used when making snapshots. When a snapshot is created, all writes to the original -flat.vmdk are halted and it becomes read-only; changes to the virtual disk are then written to these -delta files instead. The initial size of these files is 16 MB and they are grown as needed in 16 MB increments as changes are made to the VM's virtual hard disk. Because these files are a bitmap of the changes made to a virtual disk, a single -delta.vmdk file cannot exceed the size of the original -flat.vmdk file. A delta file will be created for each snapshot that you create for a VM and their file names will be incremented numerically (i.e., myvm-000001-delta.vmdk, myvm-000002-delta.vmdk). When the snapshot is deleted, these files are automatically deleted after they are merged back into the original flat.vmdk file.
The -rdm.vmdk file
This is the mapping file for the raw device mapping (RDM) format that manages mapping data for the RDM device. The mapping file is presented to the ESX host as an ordinary disk file, available for the usual file system operations. However, to the VM, the storage virtualization layer presents the mapped device as a virtual SCSI device. The metadata in the mapping file includes the location of the mapped device (name resolution) and the locking state of the mapped device. If you do a directory listing, you will see that these files will appear to take up the same amount of disk space on the VMFS volume as the actual size of the LUN that it is mapped to, but in reality they just appear that way and their size is very small. One of these files is created for each RDM that is created on a VM.
The .vswp file. When you power on a VM, a memory swap file is created that can be used in lieu of physical host memory if an ESX host exhausts all of its physical memory because it is overcommitted. These files are created equal in size to the amount of memory assigned to a VM, minus any memory reservations (default is 0) that a VM may have set on it (i.e., a 4 GB VM with a 1 GB reservation will have a 3 GB VSWP file created). These files are always created for virtual machines but only used if a host exhausts all of its physical memory. As virtual machine memory that is read/written to disk is not as fast as physical host RAM, your VMs will have degraded performance if they do start using this file. These files can take up quite a large amount of disk space on your VMFS volumes, so ensure that you have adequate space available for them, as a VM will not power on if there is not enough room to create this file. These files are deleted when a VM is powered off or suspended.
Virtual machines will lock the .vswp, -flat.vmdk and -delta.vmdk, .vmx and .log files during runtime.
The .vmss file. This file is used when virtual machines are suspended and is used to preserve the memory contents of the VM so it can start up again where it left off. This file will be approximately the same size as the amount of RAM that is assigned to a VM (even empty memory contents are written). When a VM is brought out of a suspend state, the contents of this file are written back into the physical memory of a host server, however the file is not automatically deleted until a VM is powered off (an OS reboot won't work). If a previous suspend file exists when a VM is suspended again, this file is re-used instead of deleted and re-created. If this file is deleted while the VM is suspended, then the VM will start normally and not from a suspended state.
The .vmsd file. This file is used with snapshots to store metadata and other information about each snapshot that is active on a VM. This text file is initially 0 bytes in size until a snapshot is created. A VMSD file updates with information every time snapshots are created or deleted. Only one of these files exists regardless of the number of snapshots running, as they all update this single file. The snapshot information in a VMSD file consists of the name of the VMDK and VMSN file used by each snapshot, the display name and description, and the UID of the snapshot. Once your snapshots are all deleted, this file retains old snapshot information but increments the snapshot UID to be used with new snapshots. It also renames the first snapshot to "Consolidate Helper," presumably to be used with consolidated backups.
The .vmsn file. This file is used with snapshots to store the state of a virtual machine when a snapshot is taken. A separate .vmsn file is created for every snapshot that is created on a VM and is automatically deleted when the snapshot is deleted. The size of this file will vary based on whether or not you choose to include the VM's memory state with your snapshot. If you do choose to store the memory state, this file will be slightly larger than the amount of RAM that has been assigned to the VM, as the entire memory contents, including empty memory, is copied to this file. If you do not choose to store the memory state of the snapshot then this file will be fairly small (under 32 KB). This file is similar in nature to the .vmss that is used when VMs are suspended.
The .log file. LOG files are created to log information about the virtual machine and are often used for troubleshooting purposes. There will be a number of these files present in a VM's directory. The current log file is always named vmware.log and up to six older log files will also be retained with a number at the end of their names (i.e., vmware-2.log). A new log file is created either when a VM is powered off and back on or if the log file reaches the maximum defined size limit. The amount of log files that are retained and the maximum size limits are both defined as VM advanced configuration parameters (log.rotateSize and log.keepOld).
The .vmxf file. This file is a supplemental configuration file that is not used with ESX but is retained for compatibility purposes with VMware Workstation. It is in text format and is used by Workstation for VM teaming where multiple VMs can be assigned to a team so they can be powered on or off, or suspended and resumed as a single object.
[Editor's note: Article update in April 2013]
The .ctk file. VMware CTK files list any changes made to the VM between backups. This file describes the VMDK block and grows in proportion with the number of VMDK blocks. There is one CTK file per VMDK. Change tracking files originated with VMware's Changed Block Tracking (CBT) technology for incremental backups. The CTK file stores information about what VM information blocks changed, avoiding unnecessary block backups. VMware snapshots also use .ctk files. Like .log and .nvram files, .ctk files are small.
Other less-frequently seen file types include the .vmem virtual machine paging file and the .vmtm configuration file for team data. Like VMSN files, VMEM files back up a virtual machine's memory. They exist when the VM is running or in the event of a VM crash. VMTM files support VM teams, a feature in VMware Workstation that allows a group of VMs to work together via a private LAN segment.
That covers all the files that are associated with a VMware VM, and you should have a better understanding of VM anatomy. Check out the VMs on your own VMware hosts to see the various files that make up these virtual machines. You might find a few surprises from old data that has not been properly cleaned up on VMFS volumes. Just be careful before you start deleting any files and make sure that the files you delete are no longer needed and not being used.
ABOUT THE AUTHOR: Eric Siebert is a 25-year IT veteran with experience in programming, networking, telecom and systems administration. He is a guru-status moderator on the VMware community VMTN forums and maintains VMware-land.com, a VI3 information site.