Troubleshooting VMware Consolidated Backup (VCB)

This tip helps you get debugging information outputted to VCB log files, understand that info and resolve common VCB issues quickly and successfully.

VMware Consolidated Backup (VCB), an optional component of VMware's Virtual Infrastructure 3 (VI3), can be hard

to configure and produces unhelpful error messages. This tip can help you get more debugging information outputted to the VCB log files, understand that info, and resolve VCB issues quickly and successfully.

VCB is VMware's attempt to provide an enterprise level backup solution for ESX. It is a very good effort, although it lacks in some fairly major areas. Some may compare it to the popular esxRanger software, but it may not be fair to make comparisons between a mature product and one that is still relatively new. Only time will tell what is made of VCB. For now, here are some tips that can help unravel some of this backup solution's Gordian knots."

So you've got VCB talking to your VirtualCenter Management Server, but you just cannot seem to get it back up your virtual machines (VM). The first step to figuring out what is going wrong is to increase the amount of debugging information that gets outputted to the VCB log files. Use notepad to open the following file, "C:\Program Files\VMware\VMware Consolidated Backup Framework\generic\glue.js". Search for the string "function PrepareForBackup". This will zoom to the function responsible for building the command that mounts the VMs for backup. Near the beginning of the function there is a line that reads:

cmd = StdCommandTemplate(vmMounterCmd);

Directly underneath this line insert the following text:

cmd += " -L 6 ";

Save the file and close notepad. Subsequent VCB executions will now generate verbose logs located in the temporary directory of the user who is running the backup process. Since most backup processes run as SYSTEM (unless one is executed manually outside the scope of the backup software by an administrator) the log files will be located in %WINDIR%\Temp. The log files adhere to the following naming convention:

pre_ESX-(File|FullVM)MDDYYYYHHmmSS.log and
post_ESX-(File|FullVM)MDDYYYYHHmmSS.log

The interesting information is going to be in the pre logs. Once the logging has been turned up, go ahead and attempt a backup of the VMs. After the backup job fails, it is time to examine the log file. Use the following guide as a way of determining where the problem is and how to fix it.

ERROR MESSAGE
Error: Snapshot creation failed: This operation is not supported.

PROBLEM
The VM in question appears to not have had its virtual hardware upgraded from an earlier version of ESX. Even the most vigilant ESX administrator can fall prey to this problem because although all VMs were upgraded when ESX was upgraded to 3.x, the template were not. If templates created in an earlier version of ESX are not upgraded as well, then VMs stamped out of those templates will still be using an previous virtual hardware version.

SOLUTION
Power off the VM, right-click on it, and select "Upgrade Hardware."

ERROR MESSAGE
Does not report an IP address.

PROBLEM
The VM's guest operating system (OS) in question either does not have the VMware Tools installed, the VMware Tools are not running, or they are corrupted.

SOLUTION
Install VMware Tools inside the VM's guest OS, ensure that they are running, or possibly reinstall them.

ERROR MESSAGE
Could not resolve IP address.

PROBLEM
VCB requires that all VMs have resolvable IP addresses. If the VM's name does not resolve to a valid IP address then VCB will fail to back it up.

SOLUTION
Ensure that the VM is resolvable via DNS. If for some reason you cannot create a DNS entry for this VM then edit the Windows hosts file at %WINDIR%\system32\drivers\etc\hosts on the VCB backup proxy and create a static entry for this VM.

ERROR MESSAGE
Failed to communicate with the remote host, either due to network errors or because the host is not responding.

PROBLEM
This message can be indicative of a misconfigured or corrupted VMware Tools installation. This message can also occur when there is a problem reading the VM's existing snapshots.

SOLUTION
Reinstall VMware Tools. If that does not fix the problem then it may be necessary to remove the VM's snapshots one at a time until the backup operation succeeds.

ERROR MESSAGE
Error: Failed to open the disk: One LUN could not be opened.

PROBLEM
This message means that the LUN IDs of the ESX LUNs as they are presented to the VCB backup proxy are not in the same order as they are on the ESX servers. VCB requires that the VCB backup proxy and the ESX servers each see the same LUNs in the same order. Adapter and Target IDs do not matter, only the LUN ID.

SOLUTION
Remove all of the ESX LUNs from the VCB backup proxy and re-add them in the same order that they were added to the ESX servers.

ERROR MESSAGE
Error: Could not find virtual machine specified by ipaddr

PROBLEM
This message means that the a successful vcbMounter command has never been executed against the VM in question. The first time vcbMounter is executed successfully against a VM it will cache the VM's IP address. If this IP address is not in the cache or if it cannot be obtained then the backup operation will fail. This typically occurs when VCB is trying to back up a VM that is powered off and has no cached IP address, or VCB is attempting to backup a VM that does not have VMware Tools installed.

SOLUTION
It may be necessary to power on the VM and execute the vcbMounter command so that the VM's IP address will be cached as well as making sure that the VM has the VMware Tools package installed.

ERROR MESSAGE
Error: Backup snapshot already exists.

PROBLEM
This is indicative of a previous backup attempt by VCB that did not clean up its snapshot after the backup operation.

SOLUTION
Manually remove the snapshot "_VCB-BACKUP_" from the VM.

I hope this article has helped its readers figure out some of the harder-to-solve issues that can crop of with VCB. I will be writing an upcoming series for SearchServerVirtualization.com called "Getting Started with VMware Consolidated Backup (VCB)," where I detail how to start using VCB from start to finish. Stay tuned!

About the author: Andrew Kutz is deeply embedded in the dark, dangerous world of virtualization. Andrew is an avid fan of .NET, Open Source, Terminal Services, coding, and comics. He is a Microsoft Certified Solutions Developer (MCSD), a SANS/GIAC Certified Windows Security Administrator (GCWN), and a VMware Certified Professional (VCP) in VI3. Andrew also recently submitted a security practical entitled "Sudo for Windows (sudowin)" to the SANS organization where it was accepted, thereby granting Andrew SANS GOLD status. Andrew graduated from the University of Texas at Austin with a BA in Ancient History and Classical Civilization and currently lives in Austin, TX with his wife Mandy and their two puppies, Lucy and CJ.

This was first published in February 2007

Dig deeper on VMware backup software

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

SearchServerVirtualization

SearchVirtualDesktop

SearchDataCenter

SearchCloudComputing

Close