Nested virtualization with VMware Fusion 4 and Workstation 8

The latest versions of VMware’s hosted virtualization platforms, VMware Fusion 4 for Mac OS and Workstation 8,  now allow administrators to run 64-bit nested virtual machines. This is useful in virtualization lab environments when learning and testing hypervisors like vSphere and Hyper-V.

In previous versions of Workstation and Fusion, you were limited to running 32-bit VMs. New in VMware Fusion 4 and Workstation 8 is the ability to run 64-bit nested VMs, which is especially important now that vCenter must run on a 64-bit Windows operating system and the new vCenter Server Appliance must run as a 64-bit VM.

Support for nested virtual machines (VMs) means that you can create a VM on Fusion or Workstation, run a hypervisor inside that VM, and then run another nested VM inside that hypervisor. For instance, you can run an ESXi virtual server in VMware Fusion 4, then run a Windows or Linux nested VM inside that server. But nested virtualization isn’t limited to ESXi; it also supports Microsoft Hyper-V or other hypervisors. Thus, you could run a Windows 2008 R2 VM in Fusion 4, enable Hyper-V on it, and then run a 64-bit Windows 7 desktop inside.

Using VMware Fusion 4 as the base, let’s take a look at two of the most common methods of nested virtualization: running 64-bit VMs in a virtual ESXi 5 server running under VMware Fusion 4; and running 64-bit VMs in a Hyper-V VM inside VMware Fusion 4.

Installing ESXi 5 as a VM inside VMware Fusion 4
Let’s say that you install ESXi 5 in VMware Fusion on your local Mac laptop to learn about vSphere virtualization. In Fusion 3, once ESXi was installed in a VM and you wanted to run a 64-bit nested VM on the virtual ESXi server, the VM would fail to power on.

In VMware Fusion 4, when you go to create your ESXi VMs, you can specify that you want to run ESXi version 5. But by default, you won’t be able to run a 64-bit nested VM inside the ESXi VM. In fact, when you install ESXi 5, you’ll get the error below (Figure 1).

Figure 1
You’ll get this error when you try to run ESXi 5. (Click image for an enlarged view.)

Before you can run 64-bit nested VMs with VMware Fusion 4, you have to fix this error. To do so, power down the VM and edit the ESXi VMX configuration file and add a line that says vhv.enable=TRUE. First, navigate in Finder to your virtual machines (usually under Documents).  Then, control-click on the ESXi VM package to get to Show Package Contents (Figure 2).

Figure 2
Show package contents to find the VMX file. (Click image for an enlarged view.)

Find the VMX file and control-click to go to Open With, then choose TextEdit. Now, edit the file and add vhv.enable=TRUE at the end of the file (Figure 3).

Figure 3
Add this text at the end of the file to avoid the installation error. (Click image for an enlarged view.)

Now, go through the regular ESXi installation procedure in the VM; the error message should no longer appear.

Running 64-bit VMs in ESXi 5 inside VMware Fusion 4
Once you install ESXi in VMware Fusion 4, you can connect the vSphere Client to the virtual ESXi server and create a new nested VM. As an example, let’s walk through creating a Windows 2008 R2 64-bit VM. With ESXi installed and the VM running, here’s what you might see from the vSphere Client, showing the 64-bit Windows 2008 Server VM console (Figure 4):

Figure 4
Here’s what the VM console looks like. (Click image for an enlarged view.)

And here’s what the “outer VM” -- the ESXi VM -- looks like from the perspective of VMware Fusion 4, on the Mac (Figure 5):

Figure 5
Here’s what the outer VM looks like on the Mac. (Click image for an enlarged view.)

Running 64-bit VMs in Hyper-V inside VMware Fusion 4
A new feature of VMware Fusion 4 and Workstation 8 is that you can run Microsoft Hyper-V and 64-bit nested VMs. In our example, let’s run Hyper-V inside Fusion 4 as a VM, and then run a Windows 2008 64-bit VM inside Hyper-V.

First, create a Windows 2008 R2 64-bit VM in VMware Fusion 4. Initially, when you try to add the Hyper-V role, you’ll get this message (Figure 6):

Figure 6
You’ll get this error when you try to run Hyper-V in VMware Fusion 4. (Click image for an enlarged view.)

To run Hyper-V in a VM inside VMware Fusion 4, you need to power down the Windows 2008 VM and edit the VMX configuration file, just like we did with the ESXi VM. However, this time, you need to add something different.

As we did before, navigate to the VMX file and add vhv.enable=TRUE. In addition, , you also need to add the line hypervisor.cpuid.v0=FALSE (Figure 7).

Figure 7
Add this text to the VMX file. (Click image for an enlarged view.)

Now you can power on the 64-bit nested VM under Hyper-V that’s running in VMware Fusion 4.

Keep in mind that there are many different ways to use nested VMs in VMware Fusion 4 or Workstation 8. You could run ESXi in ESXi, for example, or ESXi in Workstation, or Hyper-V in ESXi. The permutations are endless.

If you want more information about nested virtualization, including some of the specific hardware and CPU models you might need, check out the following resources:

Nesting Hyper-V in with VMware Workstation 8 and ESXi 5, by Ricky El-Qasem;
How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5, by William Lam;
VMware vSphere can virtualize itself + 64-bit nested guests, by Eric Gray; and
Running nested VMs, by the VMware Community.

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.