Virtualisation Is Fun . . . .

. . . . Well, mostly.

I've been a big fan of virtualisation since about 2002. I started with VMWare GSX 2 on Redhat 7.2 and have been using VMWare server on Ubuntu for several years since. For the most part I've found that VMWare offers the stability and usability that I wanted for the typically small installations that I need.

Lately, however, I find I'm losing my enthusiasm for the product a bit.

Firstly, there are all the compatiblity problems that require workarounds with later versions of the kernel and other software packages.

1) Firefox 3.6+ does not work with the console plugin at all. Access to machine consoles requires a workaround. There is no console plugin available for Google Chrome.

2) You can't install vmware server on the later versions of the linux kernel without manually applying 3rd party patches.

3) There are no downloadable updates for the vmware guest tools packages. You need to download the latest version of VMWare Workstation and manually extract the "latest" versions from that.

4) There are other small bugs and glitches to do with web interface and there is no longer a standalone client app as there was with previous versions. The web interface also leaves much to be desired.

So, I have long been mulling moving to a different platform. I even made a concerted effort with Virtualbox with some success.

The point is that VMWare Server is not the only "free" (as in beer) product available, there is also their "Type 1" hypervisor called ESXi, so I thought I'd take a look at that.

I had tried ESXi in the past, but met a stumbling block where it was not compatible with the hardware I was using at the time. I have new hardware now, so I thought I might take a look at ESXi again.

Currently, the version of ESXi that is on the VMWare website is version 4.1. After reading up a bit, I discovered that ESXi does not include the dreaded Web Management Console any more. It requires a standalone application called vSphere Client, which is also a "free" download. Yay! Unfortunately however, the vSphere Client app is Windows only, there is no Linux version at all. Boo! Apparently it was written in .NET so it is locked into the Microsoft platform. There are indications that you may possibly be able to run it in Wine, but I'm not interested in any more dodgy workarounds so that won't work for me.

So, it looks like I will need to be looking at my options yet again.

Basically, there are two types of hypervisors, Type 1 and Type 2. Type 1 is known as a "bare metal" hypervisor, meaning the hypervisor runs directly on the server hardware. Type 2's are called "hosted" which means the hypervisor runs as an application on a host OS. VMware Server is type 2 where as ESXi is type 1. KVM and Xen describe themselves as Type 1 hypervisors but personally, technical distinctions aside, I think the case for both Xen and KVM being type 1 is fairly weak. If I can boot it up and get an otherwise complete Linux desktop then to me that makes it a type 2. Esoteric arguments about the hypervisor being built into the Linux kernel and therefore qualifying it as "bare metal" aren't entirely convincing to me. No matter though, as I don't really care that much as long as things work.

So, let's do a quick and dirty comparison of the "free" (as in beer) virtualisation products currently available (in no particular order).

1) VMWare Server 2 (Type 2)
Has been a reliable workhorse since the old GSX days but it hasn't seen much love for quite some time. I guess VMWare/EMC are focusing their energies elsewhere. VMware server also has paravirtualisation support so a more modern CPU with VT extensions is not absolutely required. Licence = Proprietary

2) VMware ESXi (Type 1)
This I guess is where the VMWare action is these days as all the ESX related management tools and other gubbins is where EMC makes their money. Unfortunately the complete lack of *nix based tools means it's a no go zone for any self respecting Linux geek. VT extensions are required and hardware compatibility issues are not uncommon. Licence = Proprietary

3) MS Hyper-V (Type 1)
It's a Microsoft product which isn't a good start. Nevertheless, I have read a bit about it and apparently it's a fairly basic hypervisor which doesn't have much in the way of decent management tools. Considering Microsoft's history and their penchant for slipping in hidden user lock-in "features" wherever possible and the general crappiness of their products (not to mention the company itself) I am quite hesitant to consider using this. Hyper V also requires VT extensions. Licence = Proprietary

4) Citrix XenServer (Type 1)
Xen is installed as part of a Linux OS so it is easy to mistake it for a Type 2 hypervisor. I was confused about this for some time but apparently since the hypervisor is part of the kernel then this qualifies it as a Type 1. Whatever. Citrix XenServer itself is payware, but there is a "time unlimited trial version" available from their website. There are also open source implementations of Xen including one in the Ubuntu repositories. XenCenter is their version of vSphere and it is, you guessed it, also Windows only. Sigh. I haven't yet tried Xen but one thing I'm concerned about is that all the "energy" in the OSS world right now seems to be in the KVM camp so the long term future of Xen is somewhat cloudy. I don't like the Windows only management centre either.Xen also requires CPU VT extensions. Licence = XOSL

5) Redhat KVM (Type 1)
There is some debate as to whether KVM is type 1 or 2 on the interwebs. My understanding is that it runs in a similar fashion to Xen so I guess that makes it officially a type 1? I have seen it described as a "Type 1.5" before though. Anyway, KVM is part of the RHEL stack, but being open source is available for most distributions. As mentioned before it is where all the action is at the moment so I intend to try KVM next. KVM used to require VT extensions, but last time I looked at it it offered the option to use QEMU for paravirtualisation. I'm not sure what the performance hit is however, the old QEMU I knew was a full x86 emulator and was pretty slow. I believe that this new QEMU also does paravirtualisation though so it may be OK. Stay tuned for more details! Licence = GPL/LGPL

6) Virtualbox (Type 2)
Originally from Innotek, taken over by Sun and now part of the Oracle juggernaut, Virtualbox comes in two flavours. Oracle VirtualBox is free (beer) whereas VirtualBox OSE is Free (as in freedom). However, the OSE version does not work "headless" so it is of no interest to those who want to virtualise servers like I do. I spent a couple of months battling VB and did manage to get things to work (mostly) but it is simply too messy to really use in a production environment. There are zero management tools. EVERYTHING is done via the command line. Now, I'm in no way afraid of the command line but there are some areas where being able to have a visual representation of the state of all the servers you are managing can make things infinitely easier to deal with. Having to type VBoxManage list runningvms at a console just to see what is currently up is cumbersome, especially with the totally unnecessary capitalisation. Automatically starting and stopping machines at shutdown is a pain to setup and if a guest gets its state confused then you are in for an absolute world of pain trying to convince it to forget being suspended and just start up from scratch already. In the end I had a machine that simply refused to start because it insisted that it had been suspended and I could not for the life of me get it to restart. Not being able to remove a guest from the database because "a drive is attached" is simply stupid. I am trying to remove the machine goddamit, I don't CARE that there is a drive attached. But no, you have to find the drive uid by typing in a bunch of cryptic commands. Determine which virtual IDE adaptor it is connected to by using even more cryptic commands, remove the drive from the specific adaptor it is attached to and only THEN can you actually unregister the machine. Why does it have to be so difficult? By all accounts VirtualBox makes a decent alternative to VMWare Workstation as far as running on the desktop. Running it in headless mode is a recipe for frustration. Do I sound bitter? Well, maybe a little. There is also the ongoing uncertainty as to what Oracle intends to do with VirtualBox, they haven't been particularly open source friendly in the past. License = PUEL

So there we go. Now I'm off to play around with KVM!