Encrypted Btrfs Array with LUKS

Written on 2022-01-16. Last updated on 2023-11-18.

In this article we'll go over how to set up a drives for encrypted access using Btrfs, dm-crypt, and LUKS. This method is similar to the one used by default in the Ubuntu installer.

Topics covered:

Adding full-drive encryption to a drive.
Make the encrypted drive come up on boot.
Setting up Btrfs on the encrypted drive.
Adding new drive to a Btrfs array.
Removing drives from a Btrfs array.

If you only want to know how to encrypt a drive or manage Btrfs, these sections are clearly separated.

Assumptions

I will be making the following assumptions about your setup:

Your Btrfs array will be mounted at /mnt/btrfs.
You will be naming all your volumes like crypt-btrfs-xxxx where xxxx is the first four characters of the UUID of the LUKS volume.
You will be storing your key files at /root/btrfs-xxxx.keyfile.
Device paths will be listed as /dev/sdX and /dev/sdX1 where X is the drive letter and 1 is the partition number of the drive you're currently working with.

If you want to use a different naming scheme or different paths, just replace where appropriate.

Note on root drive encryption

Following this article, the key files for your drive array will be stored on your root drive in plaintext. If you want to protect these key files as well, your root drive should be encrypted. Encrypting your root drive is beyond the scope of this article and should be done during OS installation.

There are many motivations for encrypting a drive array like we do in this article. For instance, you may want to encrypt your large harddrives so you can send them for RMA without worrying about exposing private data. If that's your only reason for encryption, then your root drive may not need it. Alternatively, if you want to protect your entire system against physical theft, then your root drive should be encrypted.

If you encrypt your root drive, a password will be needed during boot. If it's unencrypted, the system will start without user interaction.

It's also worth noting any security system is only as strong as its weakest link. If your root drive is encrypted with a weak password and a low iteration time, any extra security added to the drive array may be moot. Do what you think is best for you and your situation.

WARNING

Many of the commands in this article deal with managing keys for encrypted volumes. Many of these commands will execute with no warning. ALWAYS double-check the command is correct before executing it. Use at your own risk!

Surface scan (optional)

After installing a new harddrive, but before setting it up, I like to run a complete surface scan. This gives the physical drive a chance to go over all of its sectors and swap out anything that's not working right. If you get errors during this check and it's a brand new drive, you probably want to return it.

dd if=/dev/sdX of=/dev/null bs=256K conv=noerror

Encrypting a new drive

Once the drive is physically plugged into the machine, verify the device name on the system. For a normal SATA drive it will be something like /dev/sdX. It's a good idea to run sudo lshw and confirm the make and serial number of the drive to be absolutely sure. For extra safety also run lsblk --fs to double-check the device.

Init the partition table

If the drive doesn't have a partition table yet, you need to make one. I suggest using a GUI like KDE Partition Manager or GParted, as they guide you through the process and make sure everything is OK.

However, here's how to do it from the command-line. First, create the partition table:

parted /dev/sdX mklabel gpt

Then create a single partition on the entire disk:

parted /dev/sdX mkpart primary 0% 100%

A user on Reddit pointed out that you may want to leave a bit of empty space at the end of your partition. Btrfs supports a replace command which takes the data on an old drive and moves it to a new drive before removing the old drive from the array. For this operation to succeed, the next drive must be the same size or larger as the original. Since a drive from one manufacturer may not be the exact size as the one from another, leaving a gigabyte or so free means you can easily run a replace. This is not something I do personally, but it's a valid tip if you think you will be replacing drives a lot. It's still possible to replace an old drive without the specific replace command by adding the new drive and then removing the old one, both of which are covered at the end of the article. However, the replace command is faster and more efficient. replace is not currently covered in this article.

Init new key file

The encryption keys for each drive will be stored in a key file. The best way to create one is to just use local random data:

dd if=/dev/urandom of=btrfs-new.keyfile bs=128 count=1
chmod 400 btrfs-new.keyfile

You can also download a binary file from random.org. You can use a key file as large as you want as there's no cost to increased size. However, you should use at least 512 bits (64 bytes) as this is the key size of the underlying encryption used in LUKS. I chose a length double that of the underlying key size, 128 bytes or 1024 bits, in order to guard against any potential shortcomings the RNG.

Notice that the file is called btrfs-new.keyfile and not btrfs-xxx.keyfile. This is because we do not yet know the UUID generated by LUKS. We will move this file later.

Format the drive for LUKS

LUKS needs to be initialised on the new partition you created above.

cryptsetup luksFormat -v --iter-time 10000 --key-file /root/btrfs-new.keyfile /dev/sdX1

The --iter-time 10000 argument specifies how long it should take in milliseconds to decrypt the key on the current machine. Higher values are more secure, as it will take longer for an attacker to brute-force. Higher values also increase the time it takes for your machine to boot. I think 10 seconds is acceptable. Adjust to your liking. Just be aware that Ubuntu does have a maximum amount of time it will wait for drives to become ready, so don't make it too long.

Now lsblk --fs should look something like this:

sdX
└─sdX1                     crypto_LUKS       xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Note down the first four characters of the long UUID on the right and move the key file into place:

mv /root/btrfs-new.keyfile /root/btrfs-xxxx.keyfile

Open the volume:

cryptsetup open /dev/sdX1 crypt-btrfs-xxxx --key-file=/root/btrfs-xxxx.keyfile

Now lsblk --fs should print something like this:

sdX
└─sdX1                     crypto_LUKS       xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  └─crypt-btrfs-xxxx

Open crypttab with vim /etc/crypttab and add an extra like this:

crypt-btrfs-xxxx UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /root/btrfs-xxxx.keyfile luks

Now your machine will automatically open the encrypted volume on boot.

Adding backup key

Unless you back up your raw key files somewhere, I recommend adding a backup key. This way, you can recover your data even if your root drive fails and you lose your key files. I recommend generating a password with:

pwgen -s 172 1

and storing that in a safe place, like your password manager. Add the password to the drive like this:

cryptsetup luksAddKey -v --iter-time 10000 --verify-passphrase /dev/sdX1 -d /root/btrfs-xxxx.keyfile

The length is chosen because with an alphanumeric password, 172 bytes is needed to get ≈ 1024 bits equivalent security when using pwgen -s. It should be just as secure as the keyfile we generated earlier. You can make this as long as you want at no additional cost. This will be your only key to recovery.

At this point, I recommend rebooting to ensure everything is set up properly and the drive is automatically opened.

Verifying a key

If you want to verify a key is stored in LUKS correctly, use this command:

cryptsetup luksOpen --test-passphrase -v /dev/sdX1

Or, to test a key file:

cryptsetup luksOpen --test-passphrase -v /dev/sdX1 --key-file /root/btrfs-xxxx.keyfile

Replacing a key

To change a key file, use luksChangeKy and remember to test it afterwards:

cryptsetup luksChangeKey --iter-time 10000 --key-file /root/btrfs-xxxx.keyfile /dev/sdX1 /root/btrfs-new.keyfile
mv /root/btrfs-xxxx.keyfile /root/btrfs-xxxx.keyfile~
mv /root/btrfs-new.keyfile /root/btrfs-xxxx.keyfile
cryptsetup luksOpen --test-passphrase -v /dev/sdX1 --key-file /root/btrfs-xxxx.keyfile

To replace a backup key:

cryptsetup luksChangeKey --iter-time 10000 --verify-passphrase /dev/sdX1
cryptsetup luksOpen --test-passphrase -v /dev/sdX1

This will prompt for the previous key first to be entered and then the new key.

Create a brand-new Btrfs array

To create a new Btrfs array, use mkfs.btrfs:

mkfs.btrfs -m raid1 -d raid1 /dev/mapper/crypt-btrfs-xxxx /dev/mapper/crypt-btrfs-xxxx ...

Here, two drives are specified, which is the minimum you should be using. If you have more to start with, you can specify them all on the initial command and they will be added automatically.

In this command raid1 means two copies will be stored of all your metadata (-m) and data (-d). It's worth noting that Btrfs doesn't do traditional RAID. In traditional RAID1, the drives would be exact copies of each other and have to be the same size. In Btrfs, raid1 just means exactly two copies will be stored and those copies will be on different devices.

Add a volume to the array

Assuming you have an already existing Btrfs array and want to add a drive to it, use this command:

btrfs device add /dev/mapper/crypt-btrfs-xxxx /mnt/btrfs

Run btrfs filesystem show /mnt/btrfs to verify it was added. At this point, I recommend rebooting to ensure everything comes up correctly.

Rebalance the array

Once you've added a drive to a Btrfs array, you want to rebalance the data. Balancing the array spreads out the data evenly between all the drives and ensures they're used to their fullest.

btrfs balance start --full-balance --bg /mnt/btrfs

This command will exit immediately and the balance will run in the background.

To check the status of a running balance: btrfs balance status /mnt/btrfs
To pause a running balance: btrfs balance pause /mnt/btrfs
To cancel a running balance: btrfs balance cancel /mnt/btrfs
To resume a cancelled balance from where it left off: btrfs balance resume --full-balance /mnt/btrfs

Removing a drive from the array

Before remove lsblk --fs:

sdd
└─sdd1                     crypto_LUKS       xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  └─crypt-btrfs-xxxx       btrfs             aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

Run remove:

btrfs device remove /dev/mapper/crypt-btrfs-xxxx /mnt/btrfs

This command can take hours to run, as it needs to move all the data off the drive to other drives.

After remove:

sdX
└─sdX1                     crypto_LUKS       xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  └─crypt-btrfs-xxxx

To remove a drive that's no longer accessible, use missing instead of the drive path:

btrfs device remove missing /mnt/btrfs

Remove the encrypted volume

Remove the entry from crypttab:

vim /etc/crypttab

Close the current volume:

cryptsetup luksClose crypt-btrfs-xxxx

Restart your machine to ensure everything works correctly.

Wipe the encrypted volume

To permanently erase the encryption header from disk and make it unrecoverable:

cryptsetup luksErase /dev/sdX1

Now lsblk --fs should look something like this:

sdX
└─sdX1                     crypto_LUKS       xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Remove the drive key:

rm /root/btrfs-xxxx.keyfile

Wipe the head of the drive to ensure it's unrecoverable:

for i in {1..100}; do dd if=/dev/urandom of=/dev/sdX1 bs=1M count=5 oflag=sync; echo $i; done

Now lsblk --fs should look something like this:

sdX
└─sdX1

You can now remove the drive from the machine.

Encrypted Btrfs Array with LUKS

Assumptions

Note on root drive encryption

WARNING

Surface scan (optional)

Encrypting a new drive

Init the partition table

Init new key file

Format the drive for LUKS

Adding backup key

Verifying a key

Replacing a key

Create a brand-new Btrfs array

Add a volume to the array

Rebalance the array

Removing a drive from the array

Remove the encrypted volume

Wipe the encrypted volume

Resources