Why users say 'nay' to the big ESX-ESXi switch

It's no secret that VMware favors ESXi for the future. But from an administrator's standpoint, that switch would involve more pain than pleasure – at least in ESXi's current form.

Virtualization Viewpoints is a semi-monthly column that discusses current VMware- and virtualization-related trends, news and topics. Here we offer opinions and viewpoints on the competitive, quickly growing and ever-changing virtualization industry with a focus on VMware, the current virtualization market leader, which is in an ongoing battle to remain on top and distance itself from its competitors.

VMware ESXi has not seen large-scale adoption. Most shops still use ESX because it has been around for a long time and it's what administrators know. If you ask most veteran administrators, they will tell you they love their ESX Service Console. Why? They like the power and flexibility that it offers by having a full Linux operating system (OS) for managing ESX hosts.

VMware, however, seems intent on getting rid of ESX in favor of ESXi. The company has stated over and over that it's the future. But when the big switch will happen is still anyone's guess.

At their cores, ESX and ESXi are identical. They use the exact same hypervisor code, which is called the VMkernel and is approximately 32 MB in size. The difference between the two is in the management console, which runs as a privileged virtual machine (VM) on the host.

The management consoles

The management console in ESX is called the Service Console, and it's a full Red Hat Linux OS that has been modified and optimized for ESX. VMware added a number of drivers, services, processes and utilities so that it can manage the VMkernel -- which includes the well-known ESX-specific commands such as esxcfg-vnics and vmware-cmd.

Because of the Linux OS, the Service Console is about 8 GB when using the default partition sizes. You can log in to the Service Console using a secure shell (SSH) client and run commands, view logs, run scripts, and install drivers and applications (althought this is not recommended).

There is a lot you can do with the Service Console, and it's often handy when troubleshooting host problems. But if you're not careful, you can also cause problems. Because of Service Console's power, it's often seen as a security risk if it's not properly secured.

ESXi's management console is simply referred to as the Management Console. Instead of using a full OS, it uses a small Posix environment called Busybox, which provides a very limited remote interface. VMware was able to drastically reduce the footprint of ESXi to 60 MB and deliver it as a preconfigured image file, which means that ESXi does not require a full installation the way that ESX does. To reduce the size, VMware removed many of the commands from ESXi and relies on remote management tools instead. These tools connect to ESXi using advanced programming interfaces (APIs) through the management console. The tools used to manage ESXi include the vSphere CLI (vCLI, formerly known as the Remote Command Line Interface, also known as Remote CLI or RCLI), which also works with ESX. The vCLI installs on a PC and is a remote scripting environment (Perl) that interacts with ESXi hosts to enable host configuration through scripts or specific commands. It replicates nearly all the equivalent COS commands for configuring ESX.

ESXi's user interface

ESXi has a very limited user interface called the Direct Console User Interface (DCUI) that allows you to set up basic networking, restart the management agent, view logs and other basic configurations. There is a hidden command-line interface (CLI), however, that can be used for troubleshooting problems. This hidden CLI is referred to as Tech Support Mode, and VMware does not encourage people to use it unless they are instructed to by VMware support. By default, you cannot use SSH to gain access to an ESXi management console, and the vCLI is meant to be the main management tool for ESXi. But there is an unsupported method for enabling SSH access to an ESXi host by editing a configuration file in Tech Support Mode.

Issues with VMware ESXi

Instead of getting rid of ESX and its Service Console, why can't VMware keep both versions and let people choose the version that they prefer? Some people prefer ESX and its advantages, while others prefer ESXi for its smaller footprint and easier installation. Additionally, VMware continues to add new features to ESX, which doesn't give anyone an incentive to transition to ESXi. Furthermore, there are many technologies that aren't available in ESXi, such as:

  • official support for booting ESXi from a storage area network;
  • a Web interface for managing VMs;
  • support for network jumbo frames;
  • support for scriptable installations; and
  • support for Active Directory integration.

The vSphere CLI, which is used to remotely manage ESXi because of its very limited local console, doesn't support some of the commands that exist in the ESX Service Console. Further, some of the commands that the vSphere CLI does recognize are missing the functionality that their counterparts on the ESX Service Console have, as documented in this KnowledgeBase article: vSphere CLI and Service Console Parity issues. There is also the problem of managing the host if network connectivity has been lost, or you do not have a workstation to use to connect to it using the vSphere CLI.

The DCUI is very limited in what you can do directly at the ESXi console. Basically, you can view logs, configure the management console networking and restart the management agents. There is no supported method for simple things, such as powering-on or powering-off a VM, entering maintenance mode or managing VM snapshots, which you occasionally need to do directly from the console.

Is ESXi really more secure than ESX?

VMware has long maintained that ESXi is better than ESX because it is more secure, but that isn't entirely true. The security of both is dependent on how they are configured; both ESX and ESXi can easily be made more or less secure.

ESXi isn't necessarily more secure than ESX -- security is just different than with ESX, and it has vulnerabilities as well. You can argue that ESX requires more patches because of the Service Console and is therefore less secure. But I think ESX's track record speaks for itself, and it has always been thought of as a very secure platform.

How ESXi could be made more attractive

I feel VMware still has a ways to go before they convince people to abandon their beloved ESX and its Service Console. Of course, VMware could always force the issue by discontinuing ESX, but there would probably be a revolt from VMware's customers.

On the other hand, VMware can provide greater incentives to switch to ESXi if VMware did the following:

  • Improved the DCUI, adding more functionality, such as viewing the state of VMs and performing power operations on them. Simple text menus would suffice; we don't need flashy graphical user interfaces.
  • Make sure ESXi supports everything that ESX supports. I know support for some of the missing features is coming soon, but if one major product is placed on the back burner, its replacement should be comparable.
  • Improved the vSphere CLI. The commands in the ESX Service Console should work identically in the vSphere CLI.
  • A Web user interface (UI) is nice, especially for remote console access. It doesn't have to be flashy or rich, but at least let us do the basics with it. Even a small UI similar to vCenter Mobile Access would be OK.
  • Ditch the Tech Support Mode. It's there, so support it and let people use it. Have an entry in the DCUI to access the shell because people are using it despite VMware's attempt to hide it and discourage its use.
  • Make it possible for people to script ESXi installations. Not everybody is licensed to use host profiles, and deploying ESXi should be easier.
  • The ESXi installation is easy -- in fact, too easy. Add a few more screens to the installation process to allow some customization. It doesn't have to be too complicated, but at least force people to enter a password for the root user!

I realize that if VMware makes these improvements, ESXi will grow in size, but I don't think anyone is going to mind if ESXi expands by another 50 MB. Trying to maintain the super-slim diet that ESXi is on is just making it harder for people to use and administer. By making ESXi easier to use, it's likely that more people adopt it.

Many people still prefer ESX because of ESXi's many limitations. Ultimately, I'd still like to choose between the two: So please continue to give us a choice because a future without ESX, which has been around since 2001, just isn't right.

Eric Siebert is a 25-year IT veteran with experience in programming, networking, telecom and systems administration. He is a guru-status moderator on the VMware community VMTN forums and maintains VMware-land.com, a VI3 information site.

Dig Deeper on VMware basics