Using ZFS on FreeBSD 9

I've decided to retire my Ubuntu based NAS and reload it with FreeBSD so that I can use ZFS.

I wanted to use ZFS deduplication which means that ZFS version 23 or later is required.

Since the upcoming FreeBSD 9 has ZFS v28 I decided to go with that, even though it is still only an RC.

I'm not going to boot off ZFS so there is no need to muck about trying to get that to work, although I believe it can be done.

Maybe another day.

So, I just did a vanilla FreeBSD install to my OCZ SSD and ignored the remaining drives in my server for now.

Once FreeBSD is installed, log in as root and do the following to create some ZFS "pools".

First, you need to identify the hard disks devices that are installed in your system;

# dmesg | grep ad | grep device
ada0: <OCZ 02.10104> ATA-8 SATA 2.x device
ada1: <SAMSUNG 1AA01113> ATA-7 SATA 2.x device
ada2: <ST32000542AS> ATA-8 SATA 2.x device
ada3: <ST32000542AS> ATA-8 SATA 2.x device

ada0 is my system drive which I will ignore.

The Samsung drive is a 1GB drive that I use for non critical stuff while the two ST32000 Seagates are 2TB drives that I will use to create my main pool for a total 4TB capacity.

Creating a ZFS pool is super easy. Lets' create a zpool called "store" out of the 2 x Seagates;

# zpool create store ada2 ada3

We can take a look at our pool;

# zpool list
NAME    SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
store  3.62T  0.00T   3.62T    0%  1.00x  ONLINE  -

To get a more detailed report, use the "status" command;

# zpool status
  pool: store
 state: ONLINE
 scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    store       ONLINE       0     0     0
      ada2      ONLINE       0     0     0
      ada3      ONLINE       0     0     0

errors: No known data errors

If I had wanted to make a mirror from my two Seagates, I simply add the raidz parameter;

zpool create raidz store ada2 ada3

So, presently I have a ZFS pool, which already has a default filesystem. There is no need to do a mkfs. You can see that it is mounted using the df command;

# df -h
Filesystem       Size    Used   Avail Capacity  Mounted on
/dev/ada0p2       55G     21G     29G    42%    /
devfs            1.0k    1.0k      0B   100%    /dev
store            3.4T    1.0k    3.4T     0%    /store

Normally, you would not just start dumping files straight onto the pool (which you can do if you really want to), but instead you create another filesystem to store your files in. You do this with the "zfs" command.

# zfs create store/archive

Check your mounted filesystems again;

# df -h
Filesystem       Size    Used   Avail Capacity  Mounted on
/dev/ada0p2       55G     21G     29G    42%    /
devfs            1.0k    1.0k      0B   100%    /dev
store            3.4T    1.0k    3.4T     0%    /store
store/archive    3.4T    1.0k    3.4T     0%    /store/archive

Now, one of the reasons for using ZFS is to use ZFS's deduplication and compression features. Let's turn those on;

# zfs set dedup=on store/archive
# zfs set compression=on store/archive

You could apply those commands directly to the pool if you like. When dedup is applied to the pool then the deduplication process applies to all filesystems within the pool.

Another neat thing about ZFS is how easy it is to share a filesystem using nfs. Of course NFS must be enabled on your system in /etc/rc.conf for this to work.

With NFS enabled, let's share store/archive;

zfs sharenfs="-maproot=0:0" store/archive

Unlike with "normal" NFS there is no need to restart any services after issuing this command, although you should note that is not recommended that you mix "normal" NFS (ie: /etc/exports) with ZFS controlled NFS.

In other words, keep your /etc/exports file empty.

My archive filesystem is now shared, but it is open to everybody. Usually I don't care about that at home but in other scenarios you may wish to restrict access to certain networks;

# zfs sharenfs="-maproot=0:0 -network 10.1.1.0 -mask 255.255.255.0" store/archive

You can see your existing exports by viewing the /etc/zfs/exports file;

# cat /etc/zfs/exports
# !!! DO NOT EDIT THIS FILE MANUALLY !!!

/store/archive    -maproot=0:0

You can get a whole bunch of stuff with this command;

# zfs get all store/archive
NAME           PROPERTY              VALUE                  SOURCE
store/archive  type                  filesystem             -
store/archive  creation              Mon Oct 31 10:39 2011  -
store/archive  used                  0.00K                  -
store/archive  available             3.4T                   -
store/archive  referenced            0.00K                  -
store/archive  compressratio         1.00x                  -
store/archive  mounted               yes                    -
store/archive  quota                 none                   default
store/archive  reservation           none                   default
store/archive  recordsize            128K                   default
store/archive  mountpoint            /store/archive         default
store/archive  sharenfs              -maproot=0:0           local
store/archive  checksum              on                     default
store/archive  compression           on                     local
store/archive  atime                 on                     default
store/archive  devices               on                     default
store/archive  exec                  on                     default
store/archive  setuid                on                     default
store/archive  readonly              off                    default
store/archive  jailed                off                    default
store/archive  snapdir               hidden                 default
store/archive  aclmode               discard                default
store/archive  aclinherit            restricted             default
store/archive  canmount              on                     default
store/archive  xattr                 off                    temporary
store/archive  copies                1                      default
store/archive  version               5                      -
store/archive  utf8only              off                    -
store/archive  normalization         none                   -
store/archive  casesensitivity       sensitive              -
store/archive  vscan                 off                    default
store/archive  nbmand                off                    default
store/archive  sharesmb              off                    default
store/archive  refquota              none                   default
store/archive  refreservation        none                   default
store/archive  primarycache          all                    default
store/archive  secondarycache        all                    default
store/archive  usedbysnapshots       0                      -
store/archive  usedbydataset         0.00K                  -
store/archive  usedbychildren        0                      -
store/archive  usedbyrefreservation  0                      -
store/archive  logbias               latency                default
store/archive  dedup                 on                     local
store/archive  mlslabel                                     -
store/archive  sync                  standard               default
store/archive  refcompressratio      1.00x

Finally, the list command will display all your ZFS filesystems;

# zfs list
NAME            USED  AVAIL  REFER  MOUNTPOINT
store          5.46T   841G  2.60T  /store
store/archive  2.70T   841G  2.70T  /store/archive

You may have noticed the numbers in the above grab and wonder "what's that?" My store pool has 5.46T used but it only has a capacity of 3.6T! What gives?

Well, this command was issued after loading a whole bunch of files to the NAS and it just so happens that there are a lot of duplicates on there. The zfs list command shows you the total amount of space used as it appears to the operating system as opposed to the actual amount used on the disk.

If I issue the zpool list command I can see how much of my disk is deduped;

# zpool list
NAME    SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
store  3.62T  2.70T   942G    74%  2.02x  ONLINE  -

From this we can see that my dedup ratio is 2.02. This is abnormally high however, you should expect a much lower value than that in typical usage scenarios.

So, that's the basics of ZFS, enjoy!