Physical Disk Access On VirtualBox Guest

See my previous post regarding using Virtualbox on a headless system here.

In vmware-server, mapping a drive to a physical disk is easy.

To do this in virtualbox it is ugly and hackish. The simple solution as described in the manual and repeated all over the interwebs is to use this command;

VBoxManage internalcommands createrawvmdk -filename physicaldisk.vmdk -rawdisk /dev/sdd -register

Unfortunately, this produces "VERR_ACCESS_DENIED" errors amongst other things.

Well, that's OK you might think, in vmware you need to give your user write access to the physical disk by adding it to the "disks" group. Logout, login, try again, same result.

Damn.

Here is what you have to do;

Run the command as root

sudo VBoxManage internalcommands createrawvmdk -filename physicaldisk.vmdk -rawdisk /dev/sdd -register

Ignore the nasty error

RAW host disk access VMDK file 2TbExt.vmdk created successfully.
ERROR: Could not find file for the medium '/root/.VirtualBox/HardDisks/2TbExt.vmdk' (VERR_FILE_NOT_FOUND)

Change the ownership back to your user

sudo chown brettg:brettg physicaldisk.vmdk

Apply full permissions to all on the physical drive

sudo chmod 777 /dev/sdd*

Attach it to your guest

VBoxManage storageattach ganymede --storagectl IDE0 --port 1 --device 0 --type hdd --medium ~/.VirtualBox/Machines/ganymede/2TbExt.vmdk

It's not pretty or nice, pretty nasty in fact but it works. It's hasn't done much to convince me to stick with VirtualBox over the long haul though.

Thoughts on moving from VMWare-Server to VirtualBox

I am currently looking into whether moving away from VMWare server will be a good thing. It has been quite some time since VMWare updated their server product and it is becoming more and more difficult to manage it without implementing a lot of hacks and workarounds. Currently, I am simply unable to initiate a vmware-remote-console session using Firefox 3.6 and there is no solution in sight.

I initially looked at KVM, which is the "official" ubuntu virtualisation platform but that plan fell by the wayside when I realised that KVM requires hardware VT extensions in the CPU and if they are not found will fall back to full-on cpu emulation using qemu. The Atom processor on my server is already underpowered without trying to run full x86 emulation in software. Therefore, I'm back to trying out VirtualBox.

The problem with VirtualBox is that it does not have a high level management system for the machines on the host. It is all text console based which I normally don't mind, but I do find that it is very convenient to fire up a web browser to take a quick look at which machines are currently running and the details for each one.

You can't do this with VirtualBox.

Then there is the uncertainty surrounding Oracle swallowing up Sun. There is no guarantee that Oracle will keep developing and supporting VirtualBox or indeed whether it will remain free.

Already the Open Source Edition lacks certain features such as headless operation and decent usb support which means we need to use the "full" edition which must be downloaded from the Oracle website. Currently, VirtualBox is free, as in beer but unfortunately Oracle have a long history of charging exorbitantly for everything they possibly can get away with. Most recently this was demonstrated by their insane decison to start charging $90USD for an MS Office ODF plugin that was previously available for free while at Sun.

There is nothing to say that they won't do the same thing to VirtualBox.

However, despite these concerns I am still willing to push on. The only alternative is to replace my hardware with VT capable gear which means spending money and vastly increased power usage. Avoiding these things is the whole reason I went with the Atom board in the first place.

At least until I can find a motherboard that ships with one of the VTx enabled Atoms (Silverthorne Z520, Z520PT, Z530, Z530P, Z540 and Z550) on it anyway.

Converting VMWare .vmdk images to VirtualBox .vdi

First, we need to install the qemu emulator so that we can convert the VMWare image to a generic binary image

    sudo aptitude install qemu

Use the qemu-img tool to convert to a generic .bin file;

    qemu-img convert /path/to/original.vmdk converted.bin

This binary image can now be converted to Virtualbox's native .vdi format.

    VBoxManage convertdd converted.bin converted.vdi

Note: If your .vmdk files are split into multiple 2Gb chunks you will need to create a single file using the following command.

vmware-vdiskmanager -r source_multiples.vmdk -t 2 single_file.vmdk

To convert a vdi back to vmware .vmdk use this command;

VBoxManage internalcommands converthd -srcformat VDI -dstformat VMDK sda.vdi sda.vmdk

HOWTO: VirtualBox "headless" on Lucid Lynx

As of karmic koala, Vbox 3.x is provided via the standard Ubuntu repositories. Unfortunately, this is the OSE version and it does not appear to work headless.

So, we have to download the "free as in beer" version from the Oracle website, which is currently here

At the time of writing the current build for Lucid is virtualbox-3.2_3.2.4-62467~Ubuntu~lucid_i386.deb

Before we can install the deb, we will also need to install some dependencies.

sudo apt-get install libcurl3 dkms libqt4-network libqtgui4 libxslt1.1 libasound2 \
libdirectfb-1.2-0 libgl1-mesa-dri libgl1-mesa-glx libqt4-opengl libsdl1.2debian \
libsdl1.2debian-alsa libsysfs2 libts-0.0-0 libxcursor1 linux-headers-`uname -r` \
libxdamage1 libxfixes3 libxmu6 libxxf86vm1 tsconf libqtcore4 libaudio2 libmng1

Now we can install the virtualbox deb that we downloaded earlier.

sudo dpkg -i virtualbox-3.2_3.2.4-62467~Ubuntu~lucid_i386.deb

Next, add your user account to the vboxusers group

sudo adduser brettg vboxusers

Virtualbox machines that you create will by default go in your home directory

/home/brettg/.VirtualBox/

Ensure vboxusers have appropriate permissions to the kernel, create the file;

sudo vi /etc/udev/rules.d/40-permissions.rules

/etc/udev/rules.d/40-permissions.rules

KERNEL=="vboxdrv", GROUP="vboxusers", MODE="0660"

That's it, Virtualbox should be installed and ready to go.
Now we can move on to creating a virtual machine
Create a machine named "io"

VBoxManage createvm -name io --ostype Ubuntu -register

Configure it with a nic bridged to eth0, 256Mb RAM, enable acpi and set to boot from DVD

VBoxManage modifyvm io --memory 256 --pae on --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0

Create a virtual IDE controller

VBoxManage storagectl io --name IDE0 --add ide

Create a virtual HDD

VBoxManage createvdi -filename ~/.VirtualBox/Machines/io/sda.vdi -size 48000 -register

Attach the virtual HDD

VBoxManage storageattach io --storagectl IDE0 --port 0 --device 0 --type hdd --medium ~/.VirtualBox/Machines/io/sda.vdi

Create and attach a virtual DVD drive to the controller and insert the DVD image

VBoxManage storageattach io --storagectl IDE0 --port 1 --device 0 --type dvddrive --medium /store/archive/ISO/ubuntu-10.04-server-i386.iso

The default vrdp port for machines is 3389, however, if you intend to run more than one guest then each one will need to listen on a different port. I use the 3xxx range with the xxx being the last octet of the machines IP address. For example, 192.168.0.1 would be 3001.

VBoxManage modifyvm io --vrdpport 3001

And thats it, your machine has been created. Time to start it up and give it a test drive!

Using the virtual machine

Start the machine

nohup VBoxHeadless -startvm io &

On a GUI workstation, establish a remote desktop connection to the machine. In my case, the host server is called "jupiter" so I type;

rdesktop -a 8 jupiter:3001

After you have installed the OS, you need to tell the machine to boot from the hdd.

VBoxManage modifyvm io --boot1 disk

You can also deregister the dvd image if you don't intend to use it again.

VBoxManage unregisterimage dvd /store/archive/ISO/ubuntu-10.04-server-i386.iso

Here are some other useful commands;

VBoxManage showvminfo io
VBoxManage list hdds
VBoxManage list runningvms
VBoxManage controlvm io poweroff
VBoxManage unregistervm io --delete
VBoxManage controlvm io savestate
VBoxManage closemedium disk UUID
VBoxManage modifyhd UUID --type immutable

Arrow keys in vmware

This applies to every ubuntu since Hardy Heron.

Symptom: Only the arrow keys on the numpad work in vmware-remote-console sessions. Other strange behaviours with alt, ctrl and most of the "special" keys.

Solution: Create a file ~/.vmware/config

vi ~/.vmware/config

Add the following contents;

xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78 = 0x46 # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu

You will need to close and reopen any sessions you have open.

HOWTO: LDAP Client on 10.04 Lucid Lynx

This is essentially the same as my previous Hardy Heron LDAP howto but with some steps removed. Rather than edit the old article I thought I'd just reproduce it with the appropriate parts omitted. This howto is also relevant to Jaunty and Karmic. The LDAP Server howto can be found here.

PLATFORMS TESTED: Ubuntu 10.04 LTS (Lucid), Ubuntu 11.04 LTS (Natty)

PREREQUISITES:
* A vanilla Ubuntu 10.04 (or later) desktop or server install.
* You have shared your user home directories from an NFS server
* You have successfully installed a working LDAP server

Network overview;
* domain name: tuxnetworks.com
* Servername/IP: ldap.tuxnetworks.com 10.1.1.5
* The user "brettg" is a valid LDAP user on your server.

We are going to set up a Lucid client connected to an LDAP server. We should aready have our home directories mounted via NFS.

~$ sudo apt-get install libpam-ldap libnss-ldap nss-updatedb libnss-db nscd ldap-utils

You will again be asked a bunch of questions;

LDAP server Uniform Resource Identifier: ldap://ldap.tuxnetworks.com
Distinguished name of the search base: dc=tuxnetworks,dc=com
ldap://ldap.tuxnetworks.com 3
Make local root Database admin: Yes
Does the LDAP database require login? No
LDAP account for root: cn=admin,dc=tuxnetworks,dc=com
LDAP root password: (The server LDAP root password)


Now we need to edit the following files;

~$ sudo vi /etc/ldap.conf

and edit these lines to look like this;

bind_policy soft

pam_password crypt

Find the line that begins with uri ldapi:// . . .

Comment the line out and replace it with a line like so;

uri ldap://ldap.tuxnetworks.com/

Edit this file;

~$ sudo vi /etc/ldap/ldap.conf

Edit it to look like this;

BASE    dc=tuxnetworks,dc=com
URI     ldap://ldap.tuxnetworks.com 

SIZELIMIT       0
TIMELIMIT       0
DEREF           never

Edit nsswitch.conf

~$ sudo vi /etc/nsswitch.conf

Enter the following lines;

passwd:         files ldap
group:          files ldap
shadow:         files ldap

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

Now update nss to use ldap.

~$ sudo nss_updatedb ldap
passwd... done.
group... done.


Note:

If you get an error . . .
Failed to enumerate nameservice: No such file or directory

. . . then check that your uri line in /etc/ldap.conf is correct and the address is pingable.

You should now be able to check the server with;

~$ ldapsearch -x

That command should output a tonne of stuff from the server LDAP directory.

getent passwd

You should now be able to login to the client via ssh using the user "brettg"s credentials

brettg@jupiter:~$ ssh brettg@galileo
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

Last login: Fri Jun 24 14:13:05 2011 from 10.1.1.80
brettg@galileo:~$ 

Take a look at your passwd file to make double sure you are not logging in using local auth;

grep brettg /etc/passwd

If that returns a line then you are probably logged in using a local user. Remove that line from /etc/passwd and try again.

For Gnome Desktop users.

Assigning users to the correct groups at login, create a new file called group.conf and place the following line in it;

vi /etc/security/group.conf

gdm;*;*;Al0000-9000;floppy,audio,cdrom,video,plugdev,scanner

We also need to tell pam to use the group.conf settings;

vi /etc/pam.d/gdm 

Add this line;

auth optional pam_group.so

Reboot your PC and you should be able to login to gnome using ldap!

Network monitoring with ntop

Install ntop

sudo apt-get install ntop

Create the directories that for some reason are not created by the installer

sudo mkdir /var/lib/ntop/rrd
sudo mkdir /var/lib/ntop/rrd/graphics
sudo mkdir /var/lib/ntop/rrd/flows
sudo mkdir /var/lib/ntop/rrd/interfaces
sudo mkdir /var/lib/ntop/rrd/interfaces/eth0
sudo mkdir /var/lib/ntop/rrd/interfaces/ppp0
sudo chmod -R 775 /var/lib/ntop

Start ntop on selected interface ppp0 (you will be asked to create an admin password, don't forget it!)

sudo ntop -i ppp0

browse to the ntop web interface at

http://router.example.com:3000

To manually start as a daemon, use

sudo ntop -i ppp0 --daemon

You can use the usual init scripts to start ntop;

sudo /etc/init.d/ntop start

In my case I need to monitor ppp0 so I need to edit /var/lib/ntop/init.cfg

vi /var/lib/ntop/init.cfg

and change the interface to ppp0.