VMware ESX host security: Using TCP Wrappers, PAM and iptables for defense in depth

You can boost VMware ESX host security by layering security defenses. This tip explores how to use TCP Wappers, PAM and iptables to create a defense-in-depth security strategy.

Edward Haletky
Edward L. Haletky

As security threats to virtual machines increase, IT managers and admins can augment VMware ESX host security with a defense-in-depth security strategy. Defense in depth layers security so that a breach in one layer only leads the attacker to the next layer of defensive countermeasures rather than entry into the system.

Within the service console of VMware ESX you can enable this kind of defense-in-depth security with access control layering local to the virtualization host, which enables you to limit access by hostname, IP address, time of day, group membership and username. But to create these access limits, we need to employ three techniques: TCP Wrappers, pluggable authentication modules (PAM) and iptables.

In what follows, we look at how to implement TCP Wrappers to control which hosts can access which daemons on the VMware ESX host; how to use the pam_access PAM to govern who can log in to the system based on group membership, time of day and originating host; and we explore a global method to control which hosts can access the system by changing the built-in VMware ESX firewall.

TCP Wrappers
TCP Wrappers is implemented per daemon and not per host, and unfortunately depends on the daemon itself, including the proper libraries. Alternatively, you can modify the startup of any daemon to include the functionality if it does not already exist within the daemon. Most modern daemons like Tomcat, Apache and secure shell (SSH) already contain TCP Wrappers functionality. However, not every daemon within VMware ESX has this capability. To control access to the daemons by host there are two files to modify: /etc/hosts.allow and /etc/hosts.deny. Help is available on the system by using: man hosts.allow and man hosts.deny.

/etc/hosts.deny is by far the easiest to configure as all you need to do is add the following line to the file. This line will deny all access to every daemon that uses TCP Wrappers.



Now we configure /etc/hosts.allow. In this file you will allow access to daemons by daemon name and IP address. Explicitly setting which IPs or hostnames you wish to allow is better than explicitly denying IPs or hostnames as you could easily forget to deny a given IP. The check is to use the allow list and if not on the allow list, the deny list is used. For SSH you would use the following line:



Unfortunately, vmware-hostd, which is the main connection point for VMware ESX, does not use TCP Wrappers so other options are required to control this daemon. We could modify the startup of vmware-hostd to use the tcpd daemon. While this could be done, updates to your virtualization host would overwrite the changes so it is better to use a different approach to this. Help is available on using the tcpd approach by using 'man tcpd'.

TCP Wrappers can control access to the VMware Remote Console, however.

Pluggable authentication modules
The second tool for controlling access to a virtualization host is to use the PAM named pam_access. Help can be found using 'man pam_access' from any Linux host or on the Web. This manual page does not exist on VMware ESX. Pam_access is enabled by modifying the appropriate file within /etc/pam.d. Specifically the file named /etc/pam.d/system-auth. To enable add the following single line to the system-auth file.


account [default=bad success=ok user_unknown=ignore] /lib/security/$ISA/pam_access.so

Once this line is added we now switch to modify the file /etc/security/access.conf to allow and disallow access to the system by user, IP, group membership, and time of day. The following example is a good place to start.


# Access.conf
+ : root : crond console tty1 tty2 tty3 tty3 tty5 tty6
+ : root :
+ : @vmadmins EXCEPT badadmin : ALL
+ : badadmin :
- : root : ALL
- : ALL : ALL

Each line in the file is a test, so assume we are logging in as root from IP In the first line we are comparing from where the root user to logging in. Is the user logging in from the crond daemon, console and various tty ports? Since the user is not, we go to the next line which verifies whether the root user is coming from IP, which the user is not, so we move to the third line. This line is looking for the user to be part of the vmadmins group but not part of the badadmin group. Since the user is root and root is not part of these groups, we move to the next line which says to allow all users in group badadmin coming from host Unfortunately we are using the root user, which is not part of badadmin, so this test also fails. We move to the next one which says deny access to root from everywhere. The user meets that criteria, so in this example root is denied login.

Vmware-authd or vmware-hostd uses PAM modules to control access, so we know that such a file would also deny the ability to directly connect the VMware infrastructure client (VI Client) to the host if you attempt to log in as root through the VI Client. Ensure the vpxuser is a member of the vmadmins group so that log in is allowed from the vCenter server.

The last method to control access via IP address is to use iptables. This method is the first line of defense for the service console. The other methods happen after iptables does its checks. This is one way to deny access based on IP for all daemons and services. Iptables can be fairly complex, so unless you are very familiar with it you may wish to use the other methods or refer to the book LINUX iptables Pocket Reference. There is also further reading on iptables on the Web. The simplest way of denying access is to modify the file /etc/rc.d/rc.local by adding a line similar to the following to the end of the file.


/sbin/iptables -I INPUT 1 -s -j DROP

This line tells iptables to insert a new rule before rule 1 on the INPUT chain. This rule is to drop all network traffic originating at IP address Iptables rules, like those for the pam_access module, are used in order, so where you place the lines is very important.

There is a much more complex example of the use of iptables within chapter four of the book VMware ESX Server in the Enterprise: Planning and Securing Virtualization Hosts.

These three tools can increase the security of your virtualization host by providing a Defense in Depth security strategy.

ABOUT THE AUTHOR: Edward L. Haletky is the author of VMware ESX Server in the Enterprise: Planning and Securing Virtualization Servers. He recently left Hewlett-Packard Co., where he worked on the virtualization, Linux and high-performance computing teams. Haletky owns AstroArch Consulting Inc. and is a champion and moderator for the VMware Communities Forums.

This was last published in January 2009

Dig Deeper on Securing a VMware environment

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.