. . . . Well, mostly.
I've been a big fan of virtualisation since about 2002. I started with VMWare GSX 2 on Redhat 7.2 and have been using VMWare server on Ubuntu for several years since. For the most part I've found that VMWare offers the stability and usability that I wanted for the typically small installations that I need.
Lately, however, I find I'm losing my enthusiasm for the product a bit.
Firstly, there are all the compatiblity problems that require workarounds with later versions of the kernel and other software packages.
1) Firefox 3.6+ does not work with the console plugin at all. Access to machine consoles requires a workaround. There is no console plugin available for Google Chrome.
2) You can't install vmware server on the later versions of the linux kernel without manually applying 3rd party patches.
3) There are no downloadable updates for the vmware guest tools packages. You need to download the latest version of VMWare Workstation and manually extract the "latest" versions from that.
4) There are other small bugs and glitches to do with web interface and there is no longer a standalone client app as there was with previous versions. The web interface also leaves much to be desired.
So, I have long been mulling moving to a different platform. I even made a concerted effort with Virtualbox with some success.
The point is that VMWare Server is not the only "free" (as in beer) product available, there is also their "Type 1" hypervisor called ESXi, so I thought I'd take a look at that.
I had tried ESXi in the past, but met a stumbling block where it was not compatible with the hardware I was using at the time. I have new hardware now, so I thought I might take a look at ESXi again.
Currently, the version of ESXi that is on the VMWare website is version 4.1. After reading up a bit, I discovered that ESXi does not include the dreaded Web Management Console any more. It requires a standalone application called vSphere Client, which is also a "free" download. Yay! Unfortunately however, the vSphere Client app is Windows only, there is no Linux version at all. Boo! Apparently it was written in .NET so it is locked into the Microsoft platform. There are indications that you may possibly be able to run it in Wine, but I'm not interested in any more dodgy workarounds so that won't work for me.
So, it looks like I will need to be looking at my options yet again.
Basically, there are two types of hypervisors, Type 1 and Type 2. Type 1 is known as a "bare metal" hypervisor, meaning the hypervisor runs directly on the server hardware. Type 2's are called "hosted" which means the hypervisor runs as an application on a host OS. VMware Server is type 2 where as ESXi is type 1. KVM and Xen describe themselves as Type 1 hypervisors but personally, technical distinctions aside, I think the case for both Xen and KVM being type 1 is fairly weak. If I can boot it up and get an otherwise complete Linux desktop then to me that makes it a type 2. Esoteric arguments about the hypervisor being built into the Linux kernel and therefore qualifying it as "bare metal" aren't entirely convincing to me. No matter though, as I don't really care that much as long as things work.
So, let's do a quick and dirty comparison of the "free" (as in beer) virtualisation products currently available (in no particular order).
1) VMWare Server 2 (Type 2)
Has been a reliable workhorse since the old GSX days but it hasn't seen much love for quite some time. I guess VMWare/EMC are focusing their energies elsewhere. VMware server also has paravirtualisation support so a more modern CPU with VT extensions is not absolutely required. Licence = Proprietary
2) VMware ESXi (Type 1)
This I guess is where the VMWare action is these days as all the ESX related management tools and other gubbins is where EMC makes their money. Unfortunately the complete lack of *nix based tools means it's a no go zone for any self respecting Linux geek. VT extensions are required and hardware compatibility issues are not uncommon. Licence = Proprietary
3) MS Hyper-V (Type 1)
It's a Microsoft product which isn't a good start. Nevertheless, I have read a bit about it and apparently it's a fairly basic hypervisor which doesn't have much in the way of decent management tools. Considering Microsoft's history and their penchant for slipping in hidden user lock-in "features" wherever possible and the general crappiness of their products (not to mention the company itself) I am quite hesitant to consider using this. Hyper V also requires VT extensions. Licence = Proprietary
4) Citrix XenServer (Type 1)
Xen is installed as part of a Linux OS so it is easy to mistake it for a Type 2 hypervisor. I was confused about this for some time but apparently since the hypervisor is part of the kernel then this qualifies it as a Type 1. Whatever. Citrix XenServer itself is payware, but there is a "time unlimited trial version" available from their website. There are also open source implementations of Xen including one in the Ubuntu repositories. XenCenter is their version of vSphere and it is, you guessed it, also Windows only. Sigh. I haven't yet tried Xen but one thing I'm concerned about is that all the "energy" in the OSS world right now seems to be in the KVM camp so the long term future of Xen is somewhat cloudy. I don't like the Windows only management centre either.Xen also requires CPU VT extensions. Licence = XOSL
5) Redhat KVM (Type 1)
There is some debate as to whether KVM is type 1 or 2 on the interwebs. My understanding is that it runs in a similar fashion to Xen so I guess that makes it officially a type 1? I have seen it described as a "Type 1.5" before though. Anyway, KVM is part of the RHEL stack, but being open source is available for most distributions. As mentioned before it is where all the action is at the moment so I intend to try KVM next. KVM used to require VT extensions, but last time I looked at it it offered the option to use QEMU for paravirtualisation. I'm not sure what the performance hit is however, the old QEMU I knew was a full x86 emulator and was pretty slow. I believe that this new QEMU also does paravirtualisation though so it may be OK. Stay tuned for more details! Licence = GPL/LGPL
6) Virtualbox (Type 2)
Originally from Innotek, taken over by Sun and now part of the Oracle juggernaut, Virtualbox comes in two flavours. Oracle VirtualBox is free (beer) whereas VirtualBox OSE is Free (as in freedom). However, the OSE version does not work "headless" so it is of no interest to those who want to virtualise servers like I do. I spent a couple of months battling VB and did manage to get things to work (mostly) but it is simply too messy to really use in a production environment. There are zero management tools. EVERYTHING is done via the command line. Now, I'm in no way afraid of the command line but there are some areas where being able to have a visual representation of the state of all the servers you are managing can make things infinitely easier to deal with. Having to type VBoxManage list runningvms at a console just to see what is currently up is cumbersome, especially with the totally unnecessary capitalisation. Automatically starting and stopping machines at shutdown is a pain to setup and if a guest gets its state confused then you are in for an absolute world of pain trying to convince it to forget being suspended and just start up from scratch already. In the end I had a machine that simply refused to start because it insisted that it had been suspended and I could not for the life of me get it to restart. Not being able to remove a guest from the database because "a drive is attached" is simply stupid. I am trying to remove the machine goddamit, I don't CARE that there is a drive attached. But no, you have to find the drive uid by typing in a bunch of cryptic commands. Determine which virtual IDE adaptor it is connected to by using even more cryptic commands, remove the drive from the specific adaptor it is attached to and only THEN can you actually unregister the machine. Why does it have to be so difficult? By all accounts VirtualBox makes a decent alternative to VMWare Workstation as far as running on the desktop. Running it in headless mode is a recipe for frustration. Do I sound bitter? Well, maybe a little. There is also the ongoing uncertainty as to what Oracle intends to do with VirtualBox, they haven't been particularly open source friendly in the past. License = PUEL
So there we go. Now I'm off to play around with KVM!
Tuesday, 27 July 2010
Monday, 19 July 2010
On Tablets and the Death Of Windows
Tablets are coming and with it we will witness the slow and agonizing death of the Windows / Office hegemony.
There can be no doubt that a significant portion of people will purchase a tablet in the next decade. At first it will be as an adjunct to their "traditional" PC and not as a replacement.
However, as their traditional PC's get older and the tablets get better more than a few non tech people will actively consider making a decision to buy just one or the other. Increasingly they will choose to forgo on the traditional PC in favour of some form of small form factor touchscreen alternative, probably with a bluetooth, keyboard + mouse equipped dock for using it on a desk.
Microsoft on the other hand have lost any clue they may have once had. The only thing they have is their bullying relationship with their OEM "Partners", their various user lock in technologies and the existing Windows + Office monopoly.
Before we go any further we must remember Microsoft's Rule #1.
Rule#1 is to protect the Windows + Office monopoly. Everything else is secondary.
To do that Microsoft cannot afford to introduce products that will tempt their customers away from their overpriced and under performing flagship bloatware, therefore any "Tablet OS" will be either a poor second cousin to the desktop products or else they will continue to be what they are today, which is basically the same laptop/desktop OS crowbarred onto an overweight, over heating "touchscreen laptop" sporting the same old fashioned point and click UI that they have been pushing for decades.
This has failed for them for the last decade, there is no reason to believe that it will be any less of a fail in the future.
So, assuming that Ballmer has less success at bullying his OEM's into killing the tablet market like he did for netbooks that will leave most people choosing between various tablets sporting iOS or some form of Linux.
This is a huge problem for Microsoft. The main thing that keeps them in their position of dominance is the Windows monoculture. Most non techy people simply believe that if you want to type a letter you need Office and if you want to use the internet you click on the "Blue E". Various lockin "features" (docx files, .NET, Silverlight) help to reinforce this behaviour and serve to make developers lazy. Why develop open products that can be used cross platform when 95% of users will run IE6 with Active X on Windows" was the status quo for half a decade until Firefox came along. Eventually, the growing numbers of people who were NOT using IE6 on XP reached a tipping point where developers were forced to wake up and stop actively reinforcing Redmonds iron grip on the Industry. The internet is a better place for it now.
So it will be for tablets. Apple has already shown the way. Microsoft will totally fail to keep up as it continues to try and protect its existing monopoly while the Linux upstarts will take up the remainder of the market.
Eventually we will reach another tipping point as developers are forced by the market to wean themselves away from their Visual Studio plus .Net addiction and Joe Public comes to understand that the Microsoft way is not, in fact, the only way and they can write letters to their grandkids perfectly well without the need for a bloated, over featured and expensive PC that requires constant attention, vigilance and third party security products just to keep it functioning.
I fully expect that within 5 years Microsoft will be relegated to corporate desks, and even there their dominance will be waning.
There can be no doubt that a significant portion of people will purchase a tablet in the next decade. At first it will be as an adjunct to their "traditional" PC and not as a replacement.
However, as their traditional PC's get older and the tablets get better more than a few non tech people will actively consider making a decision to buy just one or the other. Increasingly they will choose to forgo on the traditional PC in favour of some form of small form factor touchscreen alternative, probably with a bluetooth, keyboard + mouse equipped dock for using it on a desk.
Microsoft on the other hand have lost any clue they may have once had. The only thing they have is their bullying relationship with their OEM "Partners", their various user lock in technologies and the existing Windows + Office monopoly.
Before we go any further we must remember Microsoft's Rule #1.
Rule#1 is to protect the Windows + Office monopoly. Everything else is secondary.
To do that Microsoft cannot afford to introduce products that will tempt their customers away from their overpriced and under performing flagship bloatware, therefore any "Tablet OS" will be either a poor second cousin to the desktop products or else they will continue to be what they are today, which is basically the same laptop/desktop OS crowbarred onto an overweight, over heating "touchscreen laptop" sporting the same old fashioned point and click UI that they have been pushing for decades.
This has failed for them for the last decade, there is no reason to believe that it will be any less of a fail in the future.
So, assuming that Ballmer has less success at bullying his OEM's into killing the tablet market like he did for netbooks that will leave most people choosing between various tablets sporting iOS or some form of Linux.
This is a huge problem for Microsoft. The main thing that keeps them in their position of dominance is the Windows monoculture. Most non techy people simply believe that if you want to type a letter you need Office and if you want to use the internet you click on the "Blue E". Various lockin "features" (docx files, .NET, Silverlight) help to reinforce this behaviour and serve to make developers lazy. Why develop open products that can be used cross platform when 95% of users will run IE6 with Active X on Windows" was the status quo for half a decade until Firefox came along. Eventually, the growing numbers of people who were NOT using IE6 on XP reached a tipping point where developers were forced to wake up and stop actively reinforcing Redmonds iron grip on the Industry. The internet is a better place for it now.
So it will be for tablets. Apple has already shown the way. Microsoft will totally fail to keep up as it continues to try and protect its existing monopoly while the Linux upstarts will take up the remainder of the market.
Eventually we will reach another tipping point as developers are forced by the market to wean themselves away from their Visual Studio plus .Net addiction and Joe Public comes to understand that the Microsoft way is not, in fact, the only way and they can write letters to their grandkids perfectly well without the need for a bloated, over featured and expensive PC that requires constant attention, vigilance and third party security products just to keep it functioning.
I fully expect that within 5 years Microsoft will be relegated to corporate desks, and even there their dominance will be waning.
Thursday, 15 July 2010
FIx: util-linux error doing dist-upgrade on Ubuntu
This bug was first "introduced" in 10.04 Lucid but appears to have carried over to Natty, Maverick and Oneiric.
If you get this error;
then you have struck a bug in the packaging of util-linux. The problem is that util-linux depends on the upstart-job but that is not correctly defined in the package.
Install upstart-job manually;
When that is done you should be able to do a apt-get dist-upgrade again without problems.
If you get this error;
Could not perform immediate configuration on 'util-linux'then you have struck a bug in the packaging of util-linux. The problem is that util-linux depends on the upstart-job but that is not correctly defined in the package.
Install upstart-job manually;
sudo apt-get install upstart-jobWhen that is done you should be able to do a apt-get dist-upgrade again without problems.
Moving your home directories to a separate partition.
I always have my user homes on a separate partition.
This has a couple of advantages.
1) Frees up space on your root (/) partition.
2) Separates user homes from the OS. If you ever have to re-install the OS you don't lose all your user data.
In general, the steps are as follows;
First, lets take a look around and see how the system is currently configured
We want to know what drives are currently mounted and where.
The main one you are interested in is the device where / is mounted.
It will probably be /dev/sdan but it may be something different.
Here is an example using sda1;
You will notice that I currently have only a single file system mounted, and that is root on sda1. (You can ignore all the "none" mounts they are used by the kernel)
Next is to identify a suitable place to mount /home.
Let's take a look at all the drives on my system.
This will show you all the current disks and partitions on your system. Here is mine;
In my case I already have a suitable unmounted partition /dev/sda2. The third partition is (obviously) my swap partition.
Once you have created a suitable partition, mount it to a temporary mount point. I will use the /dev/sda2 from now on as the target. Let's temporarily mount it to /tmp/sda2.
At this point you should be now looking at the existing contents of the partition. If this is an old partition you probably want to ensure that you do in fact want to delete all the files located here because after the next command there is no going back.
Once you have determined that you do in fact want to trash the contents of the partition type;
Next we want to copy our existing home directories over to the new partition. Use this command;
Depending on how much data is in your home directory this may take some time to complete.
When it is done do a quick visual check to see that everything looks OK.
Compare this with your existing homes;
Both should be the same.
If all is ok you can now do the final steps, which is adding the drive to fstab.
I like to use a disks uuid rather than the physical device. It makes things much easier down the track when you want to move or add drives to your system.
Let's find the uuid of our new partition;
This will return something like this;
You want to select and copy the uuid string without the quotes.
Now, edit /etc/fstab
Add a line like this;
making sure you use your own uuid.
Let's test to see if you can mount the new home partition.
If all goes well there should be no error returned.
Let's see if it is mounted;
As we can see above, /dev/sda2 is now mounted to /home
And that's it. All we need to do is reboot and make sure everything is working OK.
Good luck and take care . . .
This has a couple of advantages.
1) Frees up space on your root (/) partition.
2) Separates user homes from the OS. If you ever have to re-install the OS you don't lose all your user data.
In general, the steps are as follows;
!!!!! BACKUP YOUR SYSTEM OR YOU COULD LOSE EVERYTHING !!!!!
First, lets take a look around and see how the system is currently configured
We want to know what drives are currently mounted and where.
df -hThe main one you are interested in is the device where / is mounted.
It will probably be /dev/sdan but it may be something different.
Here is an example using sda1;
brettg@earth:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 11G 4.2G 6.4G 40% /
none 1.5G 296K 1.5G 1% /dev
none 1.5G 608K 1.5G 1% /dev/shm
none 1.5G 132K 1.5G 1% /var/run
none 1.5G 0 1.5G 0% /var/lock
none 1.5G 0 1.5G 0% /lib/init/rw
Note: If your root is not on /dev/sda1 then ensure you make a note of where it is mounted.
You will notice that I currently have only a single file system mounted, and that is root on sda1. (You can ignore all the "none" mounts they are used by the kernel)
Next is to identify a suitable place to mount /home.
Note: I will assume that your system has a single hard disk with some free space or an existing EXT partition. If you have a second hard disk then you will need to modify this procedure to suit your configuration!
Let's take a look at all the drives on my system.
sudo fdisk -lThis will show you all the current disks and partitions on your system. Here is mine;
Disk /dev/sda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00090790
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1459 11717632 83 Linux
/dev/sda2 1459 4623 25410560 83 Linux
/dev/sda3 4623 4866 1952768 82 Linux swap / Solaris
In my case I already have a suitable unmounted partition /dev/sda2. The third partition is (obviously) my swap partition.
Note: If the unused space on your target disk is shown as free space or is partitioned as something else (ie NTFS or FAT) then you will need to use gparted or fdisk to remove it and create a suitable EXT partition (ID=83).
! For Dogs sake ensure you know what is on the partition you are messing with, all data on the partition will be LOST FOR EVER !
Once you have created a suitable partition, mount it to a temporary mount point. I will use the /dev/sda2 from now on as the target. Let's temporarily mount it to /tmp/sda2.
sudo mkdir /tmp/sda2
sudo mount /dev/sda2 /tmp/sda2
cd /tmp/sda2
At this point you should be now looking at the existing contents of the partition. If this is an old partition you probably want to ensure that you do in fact want to delete all the files located here because after the next command there is no going back.
Once you have determined that you do in fact want to trash the contents of the partition type;
sudo rm -rf /tmp/sda2Next we want to copy our existing home directories over to the new partition. Use this command;
sudo cp -rfvp /home/* /tmp/sda2/Depending on how much data is in your home directory this may take some time to complete.
When it is done do a quick visual check to see that everything looks OK.
brettg@earth:~$ ls -al /tmp/sda1
total 16
drwxr-xr-x 3 root root 4096 2010-07-14 14:58 .
drwxr-xr-x 22 root root 4096 2010-07-14 16:33 ..
drwxr-xr-x 12 brettg users 4096 2010-02-12 08:55 brettg
drwxr-xr-x 26 andy users 4096 2010-02-12 09:39 andyCompare this with your existing homes;
brettg@earth:~$ ls -al /home
total 16
drwxr-xr-x 3 root root 4096 2010-07-14 14:58 .
drwxr-xr-x 22 root root 4096 2010-07-14 16:33 ..
drwxr-xr-x 12 brettg users 4096 2010-02-12 08:55 brettg
drwxr-xr-x 26 andy users 4096 2010-02-12 09:39 andyBoth should be the same.
If all is ok you can now do the final steps, which is adding the drive to fstab.
I like to use a disks uuid rather than the physical device. It makes things much easier down the track when you want to move or add drives to your system.
Let's find the uuid of our new partition;
sudo blkid /dev/sda2This will return something like this;
/dev/sda2: UUID="ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d" TYPE="ext4"You want to select and copy the uuid string without the quotes.
Now, edit /etc/fstab
sudo vi /etc/fstabAdd a line like this;
UUID=ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d /home ext4 errors=remount-ro 0 1making sure you use your own uuid.
Let's test to see if you can mount the new home partition.
sudo mount /homeIf all goes well there should be no error returned.
Let's see if it is mounted;
brettg@earth:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 11G 4.2G 6.4G 40% /
/dev/sda2 25G 2.2G 23.7G 9% /home
none 1.5G 300K 1.5G 1% /dev
none 1.5G 608K 1.5G 1% /dev/shm
none 1.5G 132K 1.5G 1% /var/run
none 1.5G 0 1.5G 0% /var/lock
none 1.5G 0 1.5G 0% /lib/init/rw
As we can see above, /dev/sda2 is now mounted to /home
And that's it. All we need to do is reboot and make sure everything is working OK.
Note: Be aware that the original data in /home on /dev/sda1 is still there and taking up the same space as it was before. It is hidden underneath the mounted drive. Once you have determined that everything is OK, you might want to login as root, manually umount /home to make the old data reappear and then rm -rf the old data. Of course the normal caveats apply and you should make triple sure you are really deleting what you think you are deleting before hitting Enter
Always remember when messing with partitions and rm -rf IRREVERSIBLE DATALOSS IS POSSIBLE !!!!!
Good luck and take care . . .
Monday, 12 July 2010
HOWTO: Backup with Amanda on 10.04 "Lucid"
I have updated my amanda backup article for the latest version of amanda on 10.04 Lucid.
Friday, 9 July 2010
CPU, RAM and Process Monitoring with HTOP
Thursday, 8 July 2010
Mapping /dev/sg to scsi disks
This is another thing that has been making life difficult, particularly when configuring scsi passthru devices under vmware.
Just say you have a device /dev/sdd that you want to pass through to a guest machine. When you configure it under vmware you can't use the normal nomenclature (ie /dev/sdd), you must pass through the "sg" device (ie /dev/sg3)
The trouble is, sometimes it is hard to figure out which sg device is which.
The answer is to use the "sg_map" command.
Easy peasy!
Just say you have a device /dev/sdd that you want to pass through to a guest machine. When you configure it under vmware you can't use the normal nomenclature (ie /dev/sdd), you must pass through the "sg" device (ie /dev/sg3)
The trouble is, sometimes it is hard to figure out which sg device is which.
The answer is to use the "sg_map" command.
sudo apt-get install sg3-utils
brettg@jupiter:~# sudo sg_map
/dev/sg0 /dev/sda
/dev/sg1 /dev/sdb
/dev/sg2 /dev/sdc
/dev/sg3 /dev/sdd
/dev/sg4 /dev/sde
/dev/sg5 /dev/sdf
/dev/sg6 /dev/sdg
/dev/sg7 /dev/sdhEasy peasy!
Tuesday, 6 July 2010
Add VMware Server certificate to Chrome
When you connect to your VMware Server host console using the Google Chrome browser you will be presented with a warning about the site certificate being invalid. This is because the certificate is self signed due to it being generated during the vmware install process.
In Firefox you simply click "allow exception" and this warning disappears for good.
Chrome does not have a similar exception mechanism unfortunately. Chrome uses the "NSS Shared DB" which is part of the OS. From their website "On Linux, Chromium uses the NSS Shared DB. Rather than reinvent the wheel and create another certificate configuration tool, we are going to wait for a system certificate configuration utility to be created and launch that. In the mean time, you can configure certificates with the NSS command line tools."
This is how we do it. My host server is named "jupiter", you should change this to suit yours obviously.
Install the nss toolset;
Obtain the certificate from your server;
This will produce a lot of output. Select and copy the section that looks like this;
Create a file to store the certificate;
and paste in the code you copied.
Add the certificate to your store;
You should be able to browse your host without seeing any error*
You can list the certificates that are installed like this;
To look at the details of a certificate;
And finally, to delete a certificate;
* You will still get a red "broken certificate" indicator in the browser address bar but you wont be asked to proceed every time you connect.
In Firefox you simply click "allow exception" and this warning disappears for good.
Chrome does not have a similar exception mechanism unfortunately. Chrome uses the "NSS Shared DB" which is part of the OS. From their website "On Linux, Chromium uses the NSS Shared DB. Rather than reinvent the wheel and create another certificate configuration tool, we are going to wait for a system certificate configuration utility to be created and launch that. In the mean time, you can configure certificates with the NSS command line tools."
This is how we do it. My host server is named "jupiter", you should change this to suit yours obviously.
Install the nss toolset;
sudo apt-get install libnss3-toolsObtain the certificate from your server;
echo QUIT | openssl s_client -connect site:8333 | sed -ne '/BEGIN CERT/,/CERT/p'END This will produce a lot of output. Select and copy the section that looks like this;
-----BEGIN CERTIFICATE-----
MIqDRzCCArACCQC6JbNYOIukPTANBgkqhkiG9w0BAQUFADCB5zELMAkGA1UEBhMC
VVMxEzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAcTCVBhbG8gQWx0bzEVMBMG
A1UEChMMVk13YXJlLCBJbmMuMSQwIgYDVQQLExtWTXdhcmUgTWFuYWdlbWVudCBJ
bnRlcmZhY2UxEDAOBgNVBAMTB2p1cGl0ZXIxNDAyBgkqhkiG9w0BCQITJSgxMjc4
MTEwNzE0KSwoNTY0ZDc3NjE3MjY1MjA0OTZlNjMyZSkxKjAoBgkqhkiG9w0BCQEW
G3NzbC1jZXJ0aWZpY2F0ZXNAdm13YXJlLmNvbTAeFw0xMDA3MDIyMjQ1MTRaFw0y
NDAzMTAyMjQ1MTRaMIHnMQswCQYDVQJGEwJVUzETMBEGA1UECB6KQ2FsaWZvcm5p
YTESMBAGA1UEBxMJUGFsbyBBbHRvMRUwEwYDVQQKEwxWTXdhcmUsIEluYy4xJDAi
BgNVBAsTG1ZNd2FyZSBNYW5hZ2VtZW50IEludGVyZmFjZTEQMA4GA1UEAxMHanVw
aXRlcjE02DIGCSqGSIb3DQEJAhMlKDEyNzgxMTA3MTQpLCg1NjRkNzc2MTcyNjUy
MDQ5NmU2MzJlKTEqMCgGCSqkSIb3DQEJARYbc3NsLWNlcnRpZmljYXRlc0B2bXdh
cmUuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDNYXoFdwyIkk4uoAiE
vdD/Wl19hYcPswPIROr+eyMRRs1uriMA6LAyF8hYQZL0BYkCxxP6XpCT23pmZwmu
/0ZLPGig4nl35MipeRSdVgkfWMyqmT2iwArvk+Vm3zwWx5IPeCXHOHilmfIB9Xe1
Kq3zUmIHWwkrkNt+Y4z7S1eiKwIDAQABMA0GCSqGSIb3DQEBBQUAA4GlACiRRhLx
kdH3QQdTZvDBXs3D+dcFEXzgWNNyTiyEx1Gyf34cDbmNdyoVtUspH4P1HjQIj+fC
XH++BhmtIjxDhwpzat378eo2X0h3d4sw/maaVu6luTBn4VRg8/6iAOijPcQR0PaJ
EmzxqPMUrxz2s072xj00y2v9vzGJhaVim5Ph
-----END CERTIFICATE-----
Note: You should include the BEGIN and END tag lines.
Create a file to store the certificate;
vi ~/jupiter.certand paste in the code you copied.
Add the certificate to your store;
certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n vmware-jupiter -i ~/jupiter.cert"vmware-jupiter" is a nickname, you can use whatever you like there as long as it makes some sense to you.
You should be able to browse your host without seeing any error*
You can list the certificates that are installed like this;
certutil -d sql:$HOME/.pki/nssdb -LTo look at the details of a certificate;
certutil -d sql:$HOME/.pki/nssdb -L -n certificate_nicknameAnd finally, to delete a certificate;
certutil -d sql:$HOME/.pki/nssdb -D -n certificate_nickname* You will still get a red "broken certificate" indicator in the browser address bar but you wont be asked to proceed every time you connect.
Saturday, 3 July 2010
VMware remote console + Firefox 3.6 (updated)
Firefox 3.6 is the default browser on Ubuntu since Karmic but unfortunately it no longer works when attempting to launch a VMWare server remote console session from within the web gui. This is one of the reasons I decided to move away from VMware to Virtualbox headless, with mixed results.
My virtualbox experiment is over and I have decided to go back to VMware. It is a shame but vbox just has too many ways to get itself tangled up and lacks the "management polish" that you get with vmware server.
That being said, there is still the problem that new versions of Firefox refuse to work with the remote console plugin and that is a show stopper. It forces me to keep Jaunty on my laptop just so I can get a console session when needed.
The good news is that I have just discovered a workaround for this problem.
Apparently, there is an undocumented* feature in vmware player that enables it to connect to a remote host!
I have long been wondering why such a feature was not available and now I find out it actually is! Yay!
Anyway, assuming you have installed vmware player, if you type the following at a shell prompt;
you will be presented with a dialog that looks like this;
Enter your server details (including the port which is 8333 by default)
Click OK and you will see a list of the guest machines on that host, like so;
Click on a host and open to create a remote console session and you are done!
I can't tell you how excited I am to discover this trick. Now I have no more reasons to avoid changing my main browser to Chrome.
[UPDATE]
I having been playing about and you can streamline the process by adding the host and login details to the command line.
vmplayer -h jupiter -u brettg -p mypassword
* It is not mentioned when you type
My virtualbox experiment is over and I have decided to go back to VMware. It is a shame but vbox just has too many ways to get itself tangled up and lacks the "management polish" that you get with vmware server.
That being said, there is still the problem that new versions of Firefox refuse to work with the remote console plugin and that is a show stopper. It forces me to keep Jaunty on my laptop just so I can get a console session when needed.
The good news is that I have just discovered a workaround for this problem.
Apparently, there is an undocumented* feature in vmware player that enables it to connect to a remote host!
I have long been wondering why such a feature was not available and now I find out it actually is! Yay!
Anyway, assuming you have installed vmware player, if you type the following at a shell prompt;
vmplayer -hyou will be presented with a dialog that looks like this;
Enter your server details (including the port which is 8333 by default)
Click OK and you will see a list of the guest machines on that host, like so;
Click on a host and open to create a remote console session and you are done!
I can't tell you how excited I am to discover this trick. Now I have no more reasons to avoid changing my main browser to Chrome.
[UPDATE]
I having been playing about and you can streamline the process by adding the host and login details to the command line.
vmplayer -h jupiter -u brettg -p mypassword
* It is not mentioned when you type
vmplayer --help at the console.
Friday, 2 July 2010
HOWTO: SAMBA + LDAP on 10.04 Lucid Part 1
Wow, it has taken me weeks to get this to the point that it seems to be working. The official documentation is horribly broken, but I have finally got what i think is a working solution. I will probably need to tweak this guide as time goes on (see the end of the post for revision history)
PLATFORM:
* Ubuntu Server 10.04 LTS (Lucid)
PREREQUISITES:
* A standard vanilla Ubuntu 10.04 server install.
* Set your admin user as a system user.
* Mount our user home directories to an NFS server
Network overview;
* domain name: tuxnetworks.com (change this to suit your own)
* ldap-server 10.1.1.5 (change this to suit your own)
Installing Samba
We will start by configuring samba.
Download this samba config file;
Note:
Create a samba directory;
Copy the new smb.conf file into place;
Install Samba;
Install the LDAP server
Next we want to install the OpenLDAP server daemon slapd and ldap-utils, a package containing LDAP management utilities;
Notes:
First, some additional schema files need to be loaded. In a terminal enter:
Download this LDIF file
Edit the file to change "dc=tuxnetworks,dc=com" and "mypassword" to suit your own domain details.
TIP:
Now add the LDIF to the LDAP directory:
Samba needs us to tell it the LDAP admin password which we can do with this command;
Note:
And finally, we restart samba again;
You can test that samba works by using the samba-client (when it asks for roots password just press Enter);
You should see something like this;
Note:
OK, now that that we have the basic part done you should proceed to part 2.
PLATFORM:
* Ubuntu Server 10.04 LTS (Lucid)
PREREQUISITES:
* A standard vanilla Ubuntu 10.04 server install.
* Set your admin user as a system user.
* Mount our user home directories to an NFS server
Network overview;
* domain name: tuxnetworks.com (change this to suit your own)
* ldap-server 10.1.1.5 (change this to suit your own)
Installing Samba
We will start by configuring samba.
Download this samba config file;
~$ wget http://www.tuxnetworks.com/configs/smb.confNote:
Edit this file to suit your own network. You need to change the "ldap suffix" & "ldap admin" values, but you will probably also want to change "workgroup" and "netbios name" as well.
Create a samba directory;
sudo mkdir /etc/samba/Copy the new smb.conf file into place;
~$ sudo cp smb.conf /etc/samba/Install Samba;
~$ sudo apt-get install samba samba-doc smbclientInstall the LDAP server
Next we want to install the OpenLDAP server daemon slapd and ldap-utils, a package containing LDAP management utilities;
~$ sudo apt-get install slapd ldap-utils libpam-smbpass smbldap-toolsNotes:
By default slapd is configured with minimal options needed to run the slapd daemon.
The configuration example in the following sections will match the domain name of the server. For example, if the machine's Fully Qualified Domain Name (FQDN) is ldap.tuxnetworks.com, the default suffix will be dc=tuxnetworks,dc=com.
OpenLDAP uses a separate directory which contains the cn=config Directory Information Tree (DIT). The cn=config DIT is used to dynamically configure the slapd daemon, allowing the modification of schema definitions, indexes, ACLs, etc without stopping the service.
The backend cn=config directory has only a minimal configuration and will need additional configuration options in order to populate the frontend directory. The frontend will be populated with a "classical" scheme that will be compatible with address book applications and with Unix Posix accounts. Posix accounts will allow authentication to various applications, such as web applications, email Mail Transfer Agent (MTA) applications, etc.
* For external applications to authenticate using LDAP they will each need to be specifically configured to do so. Refer to the individual application documentation for details.
Remember to change "dc=tuxnetworks,dc=com" in the following examples to match your LDAP configuration.
First, some additional schema files need to be loaded. In a terminal enter:
~$ sudo ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif
~$ sudo ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif
~$ sudo ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/inetorgperson.ldifDownload this LDIF file
~$ wget http://www.tuxnetworks.com/configs/backend.ldifEdit the file to change "dc=tuxnetworks,dc=com" and "mypassword" to suit your own domain details.
TIP:
A quick way to do this is to use sed;
sed -i s/dc=tuxnetworks,dc=com/dc=example,dc=net/g backend.ldif
Now add the LDIF to the LDAP directory:
~$ sudo ldapadd -Y EXTERNAL -H ldapi:/// -f ~/backend.ldif
SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "cn=module,cn=config"
adding new entry "olcDatabase=hdb,cn=config"Samba needs us to tell it the LDAP admin password which we can do with this command;
~$ sudo smbpasswd -W
Setting stored password for "cn=admin,dc=tuxnetworks,dc=net" in secrets.tdb
New SMB password:
Retype new SMB password:Note:
Use the password you entered in the backends.ldif file earlier.
And finally, we restart samba again;
~$ sudo service smbd restartYou can test that samba works by using the samba-client (when it asks for roots password just press Enter);
~$ sudo smbclient -L localhostYou should see something like this;
~$ sudo smbclient -L localhost
Enter root's password:
Anonymous login successful
Domain=[SAMBA] OS=[Unix] Server=[Samba 3.4.7]
Sharename Type Comment
--------- ---- -------
print$ Disk Printer Drivers Share
shared Disk
archive Disk
IPC$ IPC IPC Service (Samba 3.4.7)
Anonymous login successful
Domain=[SAMBA] OS=[Unix] Server=[Samba 3.4.7]
Server Comment
--------- -------
MYSAMBASERVER Samba 3.4.7
Workgroup Master
--------- -------
MYSAMBAWORKGROUP MYSAMBASERVERNote:
If you don't see the expected output, then you should stop right now and repeat the process. Having Samba incorrectly configured at this point will cause the rest of the procedure to fail.
OK, now that that we have the basic part done you should proceed to part 2.
HOWTO: SAMBA + LDAP on 10.04 Lucid Part 2
This is Part Two of my SAMBA + LDAP howto. You must successfully complete Part One before attempting this procedure.
OK, with Part one done we must now provide our samba users with profile and netlogon directories, let's create them now;
Next we must add the samba schemas to the LDAP server. These schemas are part of the samba-doc package that we installed in Part One.
Copy the schemas to the appropriate location;
These schemas must be converted to the "ldif" format before we can use them.
Create a file called schema_convert.conf
and paste in the following lines;
Next, use slapcat to convert the schemas;
slapcat will generate a file "~/cn\=samba.ldif". Edit this file;
and change the following attributes:
to
Also, remove all these lines from the bottom of the file.
Add the schema to the server;
You should see the following line with no errors reported.
Let's check how things are going with the following query (use an empty password);
You should see a metric shedload of output with this at the end;
If you see output like above then your LDAP server is working, but we still need to finish configuring samba.
Unpack the samba-ldap-tools (we downloaded this earlier)
Now we are going to execute a perl script which will set up samba for us. For almost every prompt you should just press Enter. There are a few of exceptions however. When asked for "logon home" and "logon path" enter a "." (fullstop) and nothing else. When asked for a password (ldap master/slave bind password) use the password for the "admin" account that you entered earlier. Remember, leave the default value for everything else!
Run the script;
Now that the script has created our configuration, we can use it to populate the server;
Note:
The final touches;
Make "root" the domain adminstrator;
If this returns;
with no errors then you are looking good!
Now, we need to allow clients to authenticate via LDAP. To do this we need to install a package.
During this process enter the following details;
We also need to tell PAM and the "Name Service Switch" (NSS) service to use LDAP for auth;
If all has gone well, you should now be able to add a user to the database;
Note:
You can check your new user by issuing this command;
If you get output like this then congratulations, you have successfully configured a combined Samba/LDAP server!
Next, you should go ahead and configure a client
Revisions:
13/7/2010- Karan Pratap Singh pointed me to another howto which does some things in a better way than mine did. I have merged the parts that I like into my document (see comments)
23/6/2011- Retested, confirmed the process still works. Some parts were cleaned up and extra output from some of the commands was added. Also I split the post into 2 parts as it was getting quite long.
OK, with Part one done we must now provide our samba users with profile and netlogon directories, let's create them now;
~$ sudo mkdir -v -m 777 /var/lib/samba/profiles
~$ sudo mkdir -v -p -m 777 /var/lib/samba/netlogonNext we must add the samba schemas to the LDAP server. These schemas are part of the samba-doc package that we installed in Part One.
Copy the schemas to the appropriate location;
~$ sudo cp /usr/share/doc/samba-doc/examples/LDAP/samba.schema.gz /etc/ldap/schema/
~$ sudo gzip -d /etc/ldap/schema/samba.schema.gzThese schemas must be converted to the "ldif" format before we can use them.
Create a file called schema_convert.conf
~$ vi ~/schema_convert.confand paste in the following lines;
include /etc/ldap/schema/core.schema
include /etc/ldap/schema/collective.schema
include /etc/ldap/schema/corba.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/duaconf.schema
include /etc/ldap/schema/dyngroup.schema
include /etc/ldap/schema/inetorgperson.schema
include /etc/ldap/schema/java.schema
include /etc/ldap/schema/misc.schema
include /etc/ldap/schema/nis.schema
include /etc/ldap/schema/openldap.schema
include /etc/ldap/schema/ppolicy.schema
include /etc/ldap/schema/samba.schemaNext, use slapcat to convert the schemas;
~$ slapcat -f ~/schema_convert.conf -F ~ -n0 -s "cn={12}samba,cn=schema,cn=config" > ~/cn=samba.ldifslapcat will generate a file "~/cn\=samba.ldif". Edit this file;
~$ vi ~/cn\=samba.ldifand change the following attributes:
dn: cn={12}samba,cn=schema,cn=config
...
cn: {12}sambato
dn: cn=samba,cn=schema,cn=config
...
cn: sambaAlso, remove all these lines from the bottom of the file.
structuralObjectClass: olcSchemaConfig
entryUUID: 99e797a8-07cb-102f-8c5c-739a8467e607
creatorsName: cn=config
createTimestamp: 20100609043122Z
entryCSN: 20100609043122.188753Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20100609043122ZAdd the schema to the server;
~$ sudo ldapadd -Y EXTERNAL -H ldapi:/// -f ~/cn\=samba.ldif
SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "cn=samba,cn=schema,cn=config"
You should see the following line with no errors reported.
adding new entry "cn=samba,cn=schema,cn=config"Let's check how things are going with the following query (use an empty password);
~$ sudo ldapsearch -Y EXTERNAL -H ldapi:/// -D cn=admin,cn=config -b cn=config -W olcDatabase={1}hdbYou should see a metric shedload of output with this at the end;
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1If you see output like above then your LDAP server is working, but we still need to finish configuring samba.
Unpack the samba-ldap-tools (we downloaded this earlier)
~$ sudo gzip -d /usr/share/doc/smbldap-tools/configure.pl.gzNow we are going to execute a perl script which will set up samba for us. For almost every prompt you should just press Enter. There are a few of exceptions however. When asked for "logon home" and "logon path" enter a "." (fullstop) and nothing else. When asked for a password (ldap master/slave bind password) use the password for the "admin" account that you entered earlier. Remember, leave the default value for everything else!
Run the script;
~$ sudo perl /usr/share/doc/smbldap-tools/configure.plNow that the script has created our configuration, we can use it to populate the server;
~$ sudo smbldap-populate
Populating LDAP directory for domain TUXNETWORKS (S-1-5-21-3403240416-131340500-4256605436)
(using builtin directory structure)
adding new entry: dc=tuxnetworks,dc=net
adding new entry: ou=Users,dc=tuxnetworks,dc=net
adding new entry: ou=Groups,dc=tuxnetworks,dc=net
adding new entry: ou=Computers,dc=tuxnetworks,dc=net
adding new entry: ou=Idmap,dc=tuxnetworks,dc=net
adding new entry: uid=root,ou=Users,dc=tuxnetworks,dc=net
adding new entry: uid=nobody,ou=Users,dc=tuxnetworks,dc=net
adding new entry: cn=Domain Admins,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Domain Users,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Domain Guests,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Domain Computers,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Administrators,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Account Operators,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Print Operators,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Backup Operators,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: cn=Replicators,ou=Groups,dc=tuxnetworks,dc=net
adding new entry: sambaDomainName=TUXNETWORKS,dc=tuxnetworks,dc=net
Please provide a password for the domain root:
Changing UNIX and samba passwords for root
New password:
Retype new password:
Note:
The "UNIX and samba passwords for root" can be anything, you don't need to use the LDAP admin password here.
The final touches;
~$ sudo /etc/init.d/slapd stop
~$ sudo slapindex
WARNING!
Runnig as root!
There's a fair chance slapd will fail to start.
Check file permissions!Ignore the warning!
~$ sudo chown openldap:openldap /var/lib/ldap/*
~$ sudo /etc/init.d/slapd startMake "root" the domain adminstrator;
~$ sudo smbldap-groupmod -m 'root' 'Administrators'
adding user root to group AdministratorsIf this returns;
adding user root to group Administrators with no errors then you are looking good!
Now, we need to allow clients to authenticate via LDAP. To do this we need to install a package.
~$ sudo apt-get --yes install ldap-auth-clientDuring this process enter the following details;
ldapi:///127.0.0.1:389/
dc=tuxnetworks,dc=com
3
Yes
No
cn=admin,dc=tuxnetworks,dc=com
mypasswordWe also need to tell PAM and the "Name Service Switch" (NSS) service to use LDAP for auth;
~$ sudo auth-client-config -t nss -p lac_ldap
~$ sudo pam-auth-update ldapIf all has gone well, you should now be able to add a user to the database;
~$ sudo smbldap-useradd -a -m -P brettg
Cannot confirm uidNumber 1000 is free: checking for the next one
Changing UNIX and samba passwords for brett
New password:
Retype new password:Note:
You will notice the above command returns "Cannot confirm uidNumber 1000 is free: checking for the next one" and your LDAP user ends up with UID of 1001. If you are an OCD type like me and want all your users on LDAP and starting at 1000, then you might want to consider changing the UID of the default Ubuntu system user to a number below 1000 before issuing this command. If you do that, then make sure that you can log in and get sudo privileges before you go any further of course.
You can check your new user by issuing this command;
~$ ldapsearch -xLLL -b "dc=tuxnetworks,dc=com" uid=brettg
dn: uid=brett,ou=Users,dc=tuxnetworks,dc=net
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
objectClass: sambaSamAccount
cn: brett
sn: brett
givenName: brett
uid: brett
uidNumber: 1001
gidNumber: 513
homeDirectory: /home/brett
loginShell: /bin/bash
gecos: System User
sambaLogonTime: 0
sambaLogoffTime: 2147483647
sambaKickoffTime: 2147483647
sambaPwdCanChange: 0
displayName: brett
sambaSID: S-1-5-21-3403240416-131340500-4256605436-3002
sambaPrimaryGroupSID: S-1-5-21-3403240416-131340500-4256605436-513
sambaLogonScript: allusers.bat
sambaLMPassword: 157FBB24ACBE1A68AAD3B435B51404EE
sambaAcctFlags: [U]
sambaNTPassword: 8462E3FFE0BA1C8CED841873EC989A29
sambaPwdLastSet: 1308795478
sambaPwdMustChange: 1312683478
shadowLastChange: 15148
shadowMax: 45If you get output like this then congratulations, you have successfully configured a combined Samba/LDAP server!
Next, you should go ahead and configure a client
Revisions:
13/7/2010- Karan Pratap Singh pointed me to another howto which does some things in a better way than mine did. I have merged the parts that I like into my document (see comments)
23/6/2011- Retested, confirmed the process still works. Some parts were cleaned up and extra output from some of the commands was added. Also I split the post into 2 parts as it was getting quite long.
Subscribe to:
Posts (Atom)