HOWTO: Configure a NAS/Fileserver with LVM

LVM is a logical volume manager for the Linux kernel; it manages disk drives and similar mass-storage devices, in particular large ones. The term "volume" refers to a disk drive or partition thereof. (Source: wikipedia)

The way to visualise LVM is that you have the bottom layer, which are your physical drives. On top of that you place an abstraction layer which can span multiple, disparate drive devices. On top of this layer you then create your logical volumes. These become like the hard disk devices that you would normally mount but can be carved up into whatever sizes you like.

For example, you may have 3 hard disks in these sizes, 1Tb, 750GB and 500GB.

These drives can all be used to create a 2250GB "Volume Group".

Then, say we want 1.5TB to store media files and 400GB for user home directories. We go ahead and create the appropriate sized "Logical Volumes" leaving 850GB unused. At any time in the future it is extremely easy to expand one of your logical volumes, add a new one, add new physical disks to the volume group or replace one of the smaller disks with a bigger drive.

I am going to step through the process of installing and configuring LVM such as you might use for storing large numbers of files on a file server or NAS. In my case I will be using four almost identical 1TB drives but as mentioned earlier, using an eclectic mix of different sized drives works just as well. In fact the ability to join a number of different drives together to for one (or more) logical volumes is one of the main benefits of using LVM.

So let's get started. Actually. before we do a warning.

We will be messing around with filesystems and partition tables in this guide and that is ALWAYS a recipe for disaster. Please, do not attempt this if you have valuable data stored on any of the drives on the system you will be working on. If you do for the love of dog make a backup before you proceed. You have been warned!

OK, now that's done we can begin.

What physical disks are we going to be using for our logical volumes?

Let's take a look at the disks presently in our system;

$sudo fdisk -l

Disk /dev/sda: 8069 MB, 8069677056 bytes
255 heads, 63 sectors/track, 981 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000771b4

Device Boot Start End Blocks Id System
/dev/sda1 * 1 933 7490560 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 933 981 387072 82 Linux swap / Solaris
Partition 2 does not end on cylinder boundary.


Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda92f63f

Disk /dev/sdc doesn't contain a valid partition table

[trimmed remaining output]

What we are looking for here are all the drives that say "doesn't contain a valid partition table".

In my system I have 4, they are all 1TB drives (I trimmed the output for the last three) They are /dev/sdc /dev/sdd /dev/sde /dev/sdf.

We need to partition the drives as LVM members. Let's do the first one, /dev/sdc;

Open the disk with fdisk;

sudo fdisk /dev/sdc

You should use the 'p' command to check that the drive is indeed empty, once you are sure then we create a primary partition (using all the default entries);

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-10443, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-10443, default 10443): 10443

The drive now has a partition but we also need to define the type of partition we want. In our case of course it will be an LVM partion (Hex code 8e).

While still in fdisk we set our partition type, save and exit;

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Now, if we redo our fdisk command from earlier we should see that /dev/sdc now has an LVM partition;

sudo fdisk -l

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda92f63f

Device Boot Start End Blocks Id System
/dev/sdc1 1 121601 976760001 8e Linux LVM

In particular, we are interested in the line that says;

/dev/sdc1 1 121601 976760001 8e Linux LVM

If the output for your drive looks like this then great, you should go ahead and repeat the process for every drive that you want to include in our LVM group.

Checkpoint: At this stage you should have 1 or more drives that are partitioned as LVM and you should know which ones they are (/dev/sdb1, /dev/sdc1 etc). Make a list!

Next, we need to install some packages;

sudo apt-get install lvm2 dmsetup reiserfsprogs xfsprogs

Using our list of member drives we create a "physical volume";

sudo pvcreate /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
Physical volume "/dev/sdc1" successfully created
Physical volume "/dev/sdd1" successfully created
Physical volume "/dev/sde1" successfully created
Physical volume "/dev/sdf1" successfully created

We can take a look at our handiwork like so;

sudo pvdisplay
"/dev/sdc1" is a new physical volume of "931.51 GiB"
--- NEW Physical volume ---
PV Name /dev/sdc1
VG Name
PV Size 931.51 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID 3Qjtvy-I6DX-YpT9-1Abk-IWu3-oJT0-kqBLHv

[trimmed output for 3 remaining drives]

Assuming that went well, now we can go ahead and create a "volume group"

sudo vgcreate store /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
Volume group "store" successfully created

Let's take a look at the new volume group;

sudo vgdisplay
--- Volume group ---
VG Name store
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 4
Act PV 4
VG Size 3.64 TiB
PE Size 4.00 MiB
Total PE 953865
Alloc PE / Size 0 / 0
Free PE / Size 953865 / 3.64 TiB
VG UUID AhHEFG-Q0ql-0WV4-2q4d-Wv0B-uuz3-I7dvf3

OK, that's looking good. Now we can create our logical volumes. I am only going to create one but there is no need to use the full capacity of the volume group because another one of the great things about LVM is that it is trivially easy to enlarge (or reduce) volumes as necessary. In fact, let's do that now;

We will create a LV that does not take up the entire capacity we have available. In my case I have 3.6TB available but I will create a 2TB LV.

sudo lvcreate --name archive --size 2000G store
Logical volume "archive" created

We can take a look and see what we have created;

sudo lvdisplay
--- Logical volume ---
LV Name /dev/store/archive
VG Name store
LV UUID Pc0EVc-DF1b-n2Mt-jIfr-YHHX-CrtU-SPtyW0
LV Write Access read/write
LV Status available
# open 0
LV Size 1.95 TiB
Current LE 512000
Segments 3
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 251:0

So, just say we actually need more room in our "archive" volume. Well, that is no problem, we just need to extend it;

sudo lvextend -L3600G /dev/store/archive
Extending logical volume archive to 3.52 TiB
Logical volume archive successfully resized

We can confirm that this has worked by checking with the lvdisplay command again;

sudo lvdisplay
--- Logical volume ---
LV Name /dev/store/archive
VG Name store
LV UUID Pc0EVc-DF1b-n2Mt-jIfr-YHHX-CrtU-SPtyW0
LV Write Access read/write
LV Status available
# open 0
LV Size 3.52 TiB
Current LE 921600
Segments 4
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 251:0

Checkpoint: You should have all your drives added to the 'volume group" and have created one (or more) "logical volumes" at this point. Make sure the output from the "lvdisplay" command above corresponds with what you are seeing here.

So, we have set up our LVM volumes but just like any other hard drive we can't use them without a filesystem, let's format ours with ext3

sudo mkfs.ext3 /dev/store/archive
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
235929600 inodes, 943718400 blocks
47185920 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
28800 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

When that finally completes all we need to do now is create a mountpoint and add the volume to fstab.

I prefer to use uuid's rather than the physical device paths so that is the way we are going to mount our volume here. If you prefer of course you can mount the volume just like any device in fstab.

First, create a mountpoint for the volume;

sudo mkdir -p /store/archive

Now we need to find the UUID of our new volume;

sudo blkid /dev/store/archive

This will return something like this;

/dev/store/archive: UUID="ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d" SEC_TYPE="ext2" TYPE="ext3"

You want to select and copy the UUID string (without the quotes).

Now, edit /etc/fstab

sudo vi /etc/fstab

Add a line like this;

UUID=ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d /store/archive ext3 errors=remount-ro 0 1

Let's test to see if we can mount the new volume.

sudo mount /store/archive

If all goes well there should be no error returned.

Let's see if it is mounted;

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 7.1G 5.1G 1.7G 76% /
none 998M 220K 997M 1% /dev
none 1002M 0 1002M 0% /dev/shm
tmpfs 250G 40K 250G 1% /tmp
none 1002M 284K 1002M 1% /var/run
none 1002M 0 1002M 0% /var/lock
none 1002M 0 1002M 0% /lib/init/rw
/dev/sdb1 688G 389G 265G 60% /mnt/sdb1
/dev/mapper/store-archive
3.5T 197M 3.3T 1% /store/archive

Great! As we can see above, /dev/mapper/store-archive is now mounted to /store/archive

The final tweak is optional and involves setting the reserved block count down from 5% to something more reasonable. On very large filesystems 5% is too much and by reducing these reserved blocks we can free up gigabytes of space.

Use tune2fs to set reserved blocks at 1%;

sudo tune2fs -m 1 /dev/store/archive

And that's it. You should now be able start placing data in your new LVM volume!

Here are some other commands that you will find useful, they are pretty self explanatory so I'm not going to detail their use here.

vgrename

lvreduce

lvremove

pvrename

pvremove

lvrename

vgscan

lvscan


Change Your Shell To BASH in FreeBSD

First, need to build from the ports tree.

cd /usr/ports/shells/bash
make install

Edit the shells file

vi /etc/shells

Add this line if does not already exist;

/usr/local/bin/bash

Execute the chsh command and change the line that says;

Shell: /bin/csh

to

Shell: /usr/local/bin/bash

Logout and login again and you will have a bash shell!

HOWTO: Team Fortress 2 Dedicated Server

Create a folder in your home directory and cd into it;

mkdir ~/hlds
cd ~/hlds

You need to obtain the half Life Dedicated Server Update Tool from the steam website;

wget http://www.steampowered.com/download/hldsupdatetool.bin

When the download is done make the bin file executable and execute it;

chmod +x hldsupdatetool.bin
./hldsupdatetool.bin

Do an initial install of the dreaded steam;

./steam

This will do some downloading and updating for a while.

When it finishes it will probably say "Please retry the command." So, we do as it says and it will update again.

When that is done we then need to download the actual Team Fortress server files;

./steam -command update -game "tf" -dir .

Now, for some reason when you do an install, it doesn't actually download all the necessary files. We need to do a "verify-all" to force it to download all the missing files;

./steam -command update -game tf -verify_all

It also doesn't provide a server config file. You can create one or use the one here.

vi orangebox/tf/cfg/server.cfg

// Team Fortress 2 Server Configuration File, To be used with TF2 only!
hostname "Tuxnetworks TF2 Server" // this is your server name as shown in the server list
sv_password "" // your server password. a pair of double quotes means it is not set and anyone can join

//rcon settings
rcon_password "change_this_password" // your rcon password to log into the dev rcon console or HLSW rcon console

// Rcon Cvars
sv_rcon_banpenalty 15 //Number of minutes to ban users who fail rcon authentication
sv_rcon_log 1 //Enable/disable rcon logging.
sv_rcon_maxfailures 3 //Max number of times a user can fail rcon authentication before being banned
sv_rcon_minfailures 5 //Number of times a user can fail rcon authentication in sv_rcon_minfailuretime before being banned
sv_rcon_minfailuretime 10 //Number of seconds to track failed rcon authentications

sv_cheats 0
mp_autoteambalance 0
mp_teams_unbalance_limit 0


// Server Cvars
mp_allowspectators 1 //Toggles whether the server allows spectator mode or not
mp_autocrosshair 0
mp_bonusroundtime 5 //Time in seconds after round win until round restarts
mp_chattime 5 //amount of time in seconds players can chat after the game is over
mp_limitteams 0
mp_decals 1
mp_defaultteam 1
mp_disable_autokick 1 //Prevents a userid from being auto-kicked
mp_enableroundwaittime 1 //Enable timers to wait between rounds.
mp_fadetoblack 0 //fade a player's screen to black when he dies
mp_falldamage 5 //Amount of damage players sustains from a fall
mp_flashlight 0 //Toggles flashlight on or off
mp_footsteps 1 //Toggles footsteps on or off
mp_forcecamera 0 //Restricts spectator modes for dead players
mp_forcerespawn 0
mp_forcerespawnplayers 1 //Force all players to respawn.
mp_forcewin 1 //Forces team to win
mp_fraglimit 0
mp_idledealmethod 0 //Deals with Idle Players. 1 = Sends them into Spectator mode then kicks them if they're still idle, 2 = Kicks them out of the game
mp_idlemaxtime 1 //Maximum time a player is allowed to be idle (in minutes)
mp_maxrounds 10 //max number of rounds to play before server changes maps
mp_teams_unbalance_limit 2 //Teams are unbalanced when one team has this many more players than the other team. (0 disables check)
mp_teststalemate 0 //Test the stalemate mode. Parameter: <0/1>. If 1, the map will reset at the end.
mp_time_between_capscoring 5 //Delay between scoring of owned capture points.
mp_timelimit 120 //game time per map in minutes
mp_winlimit 10 //Max number of rounds one team can win before server changes maps
sv_allow_color_correction 1 //Allow or disallow clients to use color correction on this server.
sv_allow_wait_command 0 //Allow or disallow the wait command on clients connected to this server.
sv_allowdownload 1 //Allow clients to download files
sv_allowupload 1 //Allow clients to upload customizations files
sv_alltalk 0 //Players can hear all other players, no team restrictions
sv_alternateticks 0 //If set, server only simulates entities on even numbered ticks.
sv_autosave 0 //Set to 1 to autosave game on level transition. Does not affect autosave triggers.
sv_bonus_challenge 0 //Set to values other than 0 to select a bonus map challenge type.
sv_cacheencodedents 1 //If set to 1, does an optimization to prevent extra SendTable_Encode calls.
sv_cheats 0 //Allow cheats on server
sv_clearhinthistory 0 //Clear memory of server side hints displayed to the player.
sv_consistency 1 //Whether the server enforces file consistency for critical files
sv_contact "" //Contact email for server sysop
sv_downloadurl "" //Location from which clients can download missing files
sv_enableoldqueries 1 //Enable support for old style (HL1) server queries
sv_pausable 0 //Is the server pausable.

// Lan or internet play, Server region cvars
sv_lan 0 //Server is a lan server ( no heartbeat, no authentication, no non-class C addresses )
sv_region 255 // Region Codes: 0 - US East coast, 1 - US West coast, 2 - South America, 2 - South America, 3 - Europe, 4 - Asia, 5 - Australia, 6 - Middle East, 7 - Africa, 255 - world

//server Logging
sv_log_onefile 0 //Log server information to only one file.
sv_logbans 1 //Log server bans in the server logs.
sv_logblocks 0 //If true when log when a query is blocked (can cause very large log files)
sv_logecho 0 //Echo log information to the console.
sv_logfile 1 //Log server information in the log file.
sv_logflush 0 //Flush the log file to disk on each write (slow).
sv_logsdir "logs" //Folder in the game directory where server logs will be stored.

//Server Rates
sv_maxcmdrate 0 //(If sv_mincmdrate is > 0), this sets the maximum value for cl_cmdrate.
sv_maxrate 20000 //Max bandwidth rate allowed on server, 0 == unlimited
sv_maxreplay 2 //Maximum replay time in seconds
sv_maxupdaterate 100 //Maximum updates per second that the server will allow
sv_mincmdrate 0 //This sets the minimum value for cl_cmdrate. 0 == unlimited.
sv_minrate 0 //Min bandwidth rate allowed on server, 0 == unlimited
sv_minupdaterate 30 //Minimum updates per second that the server will allow


Finally, we want to start our server. It's easiest to do this using a short shell script;

vi tf2.sh

#!/bin/sh
echo "Starting Team Fortress Server"
sleep 1
cd ./orangebox
./srcds_run -console -game tf +map cp_dustbowl +maxplayers 8 -autoupdate +ip 1.2.3.4


make your script executable;

chmod +x tf2.sh

When you execute the script there are all sorts of warnings and errors. I've looked up some of them and apparently this is "normal". The server seems to run OK anyway.

Stop cron jobs from sending email

If you have a cronjob that is constantly spamming you with emails, then add the following to the offending line in your crontab;

>> /dev/null 2>&1

Example;

* * * * * root /root/checkvpn.cron >> /dev/null 2>&1

Voila! No more annoying emails!

Reverting your default editor to vi

In their ongoing quest to destroy ubuntu, Team Shuttleworth have decided to change the default editor for visudo to the train wreck that is nano. To change it back we need to change our default editor back to vi.

#sudo update-alternatives --config editor
There are 3 choices for the alternative editor (providing /usr/bin/editor).

Selection Path Priority Status
------------------------------------------------------------
* 0 /bin/nano 40 auto mode
1 /bin/nano 40 manual mode
2 /usr/bin/vim.basic 30 manual mode
3 /usr/bin/vim.tiny 10 manual mode

Press enter to keep the current choice[*], or type selection number: 3
update-alternatives: using /usr/bin/vim.tiny to provide /usr/bin/editor (editor) in manual mode.


As shown in the code box above, you need to choose option 3