LVM is a logical volume manager for the Linux kernel; it manages disk drives and similar mass-storage devices, in particular large ones. The term "volume" refers to a disk drive or partition thereof. (Source:
wikipedia)
The way to visualise LVM is that you have the bottom layer, which are your physical drives. On top of that you place an abstraction layer which can span multiple, disparate drive devices. On top of this layer you then create your logical volumes. These become like the hard disk devices that you would normally mount but can be carved up into whatever sizes you like.
For example, you may have 3 hard disks in these sizes, 1Tb, 750GB and 500GB.
These drives can all be used to create a 2250GB "Volume Group".
Then, say we want 1.5TB to store media files and 400GB for user home directories. We go ahead and create the appropriate sized "Logical Volumes" leaving 850GB unused. At any time in the future it is extremely easy to expand one of your logical volumes, add a new one, add new physical disks to the volume group or replace one of the smaller disks with a bigger drive.
I am going to step through the process of installing and configuring LVM such as you might use for storing large numbers of files on a file server or NAS. In my case I will be using four almost identical 1TB drives but as mentioned earlier, using an eclectic mix of different sized drives works just as well. In fact the ability to join a number of different drives together to for one (or more) logical volumes is one of the main benefits of using LVM.
So let's get started. Actually. before we do a warning.
We will be messing around with filesystems and partition tables in this guide and that is ALWAYS a recipe for disaster. Please, do not attempt this if you have valuable data stored on any of the drives on the system you will be working on. If you do for the love of dog make a backup before you proceed. You have been warned!
OK, now that's done we can begin.
What physical disks are we going to be using for our logical volumes?
Let's take a look at the disks presently in our system;
$sudo fdisk -l
Disk /dev/sda: 8069 MB, 8069677056 bytes
255 heads, 63 sectors/track, 981 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000771b4
Device Boot Start End Blocks Id System
/dev/sda1 * 1 933 7490560 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 933 981 387072 82 Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda92f63f
Disk /dev/sdc doesn't contain a valid partition table
[trimmed remaining output]What we are looking for here are all the drives that say "doesn't contain a valid partition table".
In my system I have 4, they are all 1TB drives (I trimmed the output for the last three) They are /dev/sdc /dev/sdd /dev/sde /dev/sdf.
We need to partition the drives as LVM members. Let's do the first one, /dev/sdc;
Open the disk with fdisk;
sudo fdisk /dev/sdcYou should use the '
p' command to check that the drive is indeed empty, once you are sure then we create a primary partition (using all the default entries);
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-10443, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-10443, default 10443): 10443The drive now has a partition but we also need to define the type of partition we want. In our case of course it will be an LVM partion (Hex code 8e).
While still in fdisk we set our partition type, save and exit;
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.Now, if we redo our fdisk command from earlier we should see that /dev/sdc now has an LVM partition;
sudo fdisk -l
Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda92f63f
Device Boot Start End Blocks Id System
/dev/sdc1 1 121601 976760001 8e Linux LVMIn particular, we are interested in the line that says;
/dev/sdc1 1 121601 976760001 8e Linux LVMIf the output for your drive looks like this then great, you should go ahead and repeat the process for every drive that you want to include in our LVM group.
Checkpoint: At this stage you should have 1 or more drives that are partitioned as LVM and you should know which ones they are (/dev/sdb1, /dev/sdc1 etc). Make a list!
Next, we need to install some packages;
sudo apt-get install lvm2 dmsetup reiserfsprogs xfsprogsUsing our list of member drives we create a "physical volume";
sudo pvcreate /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
Physical volume "/dev/sdc1" successfully created
Physical volume "/dev/sdd1" successfully created
Physical volume "/dev/sde1" successfully created
Physical volume "/dev/sdf1" successfully createdWe can take a look at our handiwork like so;
sudo pvdisplay
"/dev/sdc1" is a new physical volume of "931.51 GiB"
--- NEW Physical volume ---
PV Name /dev/sdc1
VG Name
PV Size 931.51 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID 3Qjtvy-I6DX-YpT9-1Abk-IWu3-oJT0-kqBLHv
[trimmed output for 3 remaining drives]Assuming that went well, now we can go ahead and create a "volume group"
sudo vgcreate store /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
Volume group "store" successfully createdLet's take a look at the new volume group;
sudo vgdisplay
--- Volume group ---
VG Name store
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 4
Act PV 4
VG Size 3.64 TiB
PE Size 4.00 MiB
Total PE 953865
Alloc PE / Size 0 / 0
Free PE / Size 953865 / 3.64 TiB
VG UUID AhHEFG-Q0ql-0WV4-2q4d-Wv0B-uuz3-I7dvf3OK, that's looking good. Now we can create our logical volumes. I am only going to create one but there is no need to use the full capacity of the volume group because another one of the great things about LVM is that it is trivially easy to enlarge (or reduce) volumes as necessary. In fact, let's do that now;
We will create a LV that does not take up the entire capacity we have available. In my case I have 3.6TB available but I will create a 2TB LV.
sudo lvcreate --name archive --size 2000G store
Logical volume "archive" createdWe can take a look and see what we have created;
sudo lvdisplay
--- Logical volume ---
LV Name /dev/store/archive
VG Name store
LV UUID Pc0EVc-DF1b-n2Mt-jIfr-YHHX-CrtU-SPtyW0
LV Write Access read/write
LV Status available
# open 0
LV Size 1.95 TiB
Current LE 512000
Segments 3
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 251:0So, just say we actually need more room in our "archive" volume. Well, that is no problem, we just need to extend it;
sudo lvextend -L3600G /dev/store/archive
Extending logical volume archive to 3.52 TiB
Logical volume archive successfully resizedWe can confirm that this has worked by checking with the lvdisplay command again;
sudo lvdisplay
--- Logical volume ---
LV Name /dev/store/archive
VG Name store
LV UUID Pc0EVc-DF1b-n2Mt-jIfr-YHHX-CrtU-SPtyW0
LV Write Access read/write
LV Status available
# open 0
LV Size 3.52 TiB
Current LE 921600
Segments 4
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 251:0Checkpoint: You should have all your drives added to the 'volume group" and have created one (or more) "logical volumes" at this point. Make sure the output from the "lvdisplay" command above corresponds with what you are seeing here.
So, we have set up our LVM volumes but just like any other hard drive we can't use them without a filesystem, let's format ours with ext3
sudo mkfs.ext3 /dev/store/archive
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
235929600 inodes, 943718400 blocks
47185920 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
28800 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.When that finally completes all we need to do now is create a mountpoint and add the volume to fstab.
I prefer to use uuid's rather than the physical device paths so that is the way we are going to mount our volume here. If you prefer of course you can mount the volume just like any device in fstab.
First, create a mountpoint for the volume;
sudo mkdir -p /store/archiveNow we need to find the UUID of our new volume;
sudo blkid /dev/store/archiveThis will return something like this;
/dev/store/archive: UUID="ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d" SEC_TYPE="ext2" TYPE="ext3"You want to select and copy the UUID string (without the quotes).
Now, edit /etc/fstab
sudo vi /etc/fstabAdd a line like this;
UUID=ee88bfd6-1a7e-486f-85ff-2f2e4c81bd6d /store/archive ext3 errors=remount-ro 0 1Let's test to see if we can mount the new volume.
sudo mount /store/archiveIf all goes well there should be no error returned.
Let's see if it is mounted;
df -hFilesystem Size Used Avail Use% Mounted on
/dev/sda1 7.1G 5.1G 1.7G 76% /
none 998M 220K 997M 1% /dev
none 1002M 0 1002M 0% /dev/shm
tmpfs 250G 40K 250G 1% /tmp
none 1002M 284K 1002M 1% /var/run
none 1002M 0 1002M 0% /var/lock
none 1002M 0 1002M 0% /lib/init/rw
/dev/sdb1 688G 389G 265G 60% /mnt/sdb1
/dev/mapper/store-archive
3.5T 197M 3.3T 1% /store/archive
Great! As we can see above, /dev/mapper/store-archive is now mounted to /store/archive
The final tweak is optional and involves setting the reserved block count down from 5% to something more reasonable. On very large filesystems 5% is too much and by reducing these reserved blocks we can free up gigabytes of space.
Use tune2fs to set reserved blocks at 1%;
sudo tune2fs -m 1 /dev/store/archive And that's it. You should now be able start placing data in your new LVM volume!
Here are some other commands that you will find useful, they are pretty self explanatory so I'm not going to detail their use here.
vgrename
lvreduce
lvremove
pvrename
pvremove
lvrename
vgscan
lvscan
