Btrfs
btrfs is a modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration.
Installation
Partition the disk
# printf "label: gpt\n,550M,U\n,,L\n" | sfdisk /dev/sdX
Format partitions and create subvolumes
# nix-shell -p btrfs-progs
# mkfs.fat -F 32 /dev/sdX1
# mkfs.btrfs /dev/sdX2
# mkdir -p /mnt
# mount /dev/sdX2 /mnt
# btrfs subvolume create /mnt/root
# btrfs subvolume create /mnt/home
# btrfs subvolume create /mnt/nix
# umount /mnt
Mount the partitions and subvolumes
# mount -o compress=zstd,subvol=root /dev/sdX2 /mnt
# mkdir /mnt/{home,nix}
# mount -o compress=zstd,subvol=home /dev/sdX2 /mnt/home
# mount -o compress=zstd,noatime,subvol=nix /dev/sdX2 /mnt/nix
# mkdir /mnt/boot
# mount /dev/sdX1 /mnt/boot
Install NixOS
# nixos-generate-config --root /mnt
# nano /mnt/etc/nixos/configuration.nix # manually add mount options (see Compression below for an example)
# nixos-install
Configuration
Compression
nixos-generate-config --show-hardware-config
doesn't detect mount options automatically, so to enable compression, you must specify it and other mount options in a persistent configuration:
fileSystems = {
"/".options = [ "compress=zstd" ];
"/home".options = [ "compress=zstd" ];
"/nix".options = [ "compress=zstd" "noatime" ];
"/swap".options = [ "noatime" ];
};
Swap file
Optionally, create a separate subvolume for the swap file. Be sure to regenerate your hardware-configuration.nix
if you choose to do this.
# mkdir -p /mnt
# mount /dev/sdXY /mnt
# btrfs subvolume create /mnt/swap
# umount /mnt
# mkdir /swap
# mount -o noatime,subvol=swap /dev/sdXY /swap
Then, create the swap file and adjust its size as desired:
# btrfs filesystem mkswapfile --size 8g --uuid clear /swap/swapfile
Finally, add the swap file to your configuration and nixos-rebuild switch
:
swapDevices = [ { device = "/swap/swapfile"; } ];
Scrubbing
Btrfs filesystems by default keep checksums for all files, to monitor if the file has changed due to hardware malfunctions or other external effects.
Scrubbing is the process of checking file consistency, which may use checksums and/or duplicated copies of data, from raid for example. Scrubbing may be done "online", meaning you don't need to unmount a subvolume to scrub it.
You can enable automatic scrubbing withː
services.btrfs.autoScrub.enable = true;
Automatic scrubbing by default is performed once a month, but you can change that withː
services.btrfs.autoScrub.interval = "weekly";
interval
syntax is defined by systemd.timer's Calendar Events
By default, autoscrub will scrub all detected btrfs mount points. However, in case of mounted nested subvolumes (e.g. the example above where /nix
and /home
are nested subvolumes under /
), you only need to scrub the topmost one. So an example configuration may look like this:
services.btrfs.autoScrub = {
enable = true;
interval = "monthly";
fileSystems = [ "/" ];
};
The result of the periodic auto scrub will be saved to the system journal, and you can check the status of the last scrubː
btrfs scrub status /
You can also start a scrub in the background manuallyː
btrfs scrub start /
You can check the status of the ongoing scrubbing process with the same status
command as above.
Deduplication
Files with (partially) equal contents can be deduplicated using bees or duperemove.
bees can be configured in configuration.nix
:
services.beesd.filesystems = {
root = {
spec = "LABEL=root";
hashTableSizeMB = 2048;
verbosity = "crit";
extraOptions = [ "--loadavg-target" "5.0" ];
};
};
This will run the daemon in the background. To disable auto-start, use systemd.services."beesd@root".wantedBy = lib.mkForce [ ];
for each filesystem.
Usage
Subvolumes
Create a subvolume
btrfs subvolume create /mnt/nixos
Remove a subvolume
btrfs subvolume delete /mnt/nixos
Snapshots
A snapshot in btrfs is simply a subvolume that shares its data (and metadata) with some other subvolume, using btrfs's CoW capabilities. Because of that, there is no special location for snapshots and you can decide where you want to store them. It can be a simple directory inside the root subvolume, or a directory inside a dedicated "snapshots" subvolume.
For this example we are going to store snapshots in a directory /snapshots
, that has to be created beforehand with sudo mkdir /snapshots
To take a read-only (-r
) snapshot called home_snapshot_202302
of the subvolume mounted at /home
btrfs subvolume snapshot -r /home /snapshots/home_snapshot_202302
You can also snapshot the root subvolume. But keep in mind that nested subvolumes are not part of a snapshot. So if you have subvolumes /nix /home
, taking a snapshot of /
will not include them.
btrfs subvolume snapshot -r / /snapshots/nixos_snapshot_202302
Make snapshot read-write againː
btrfs property set -ts /snapshots/home_snapshot_202302 ro false
However, changing read-only property of a snapshot in-place may causes issues with any future incremental send/receive. Instead, a read-only snapshot itself (being a simple subvolume) can be snapshotted again as a read-write snapshot like this:
btrfs subvolume snapshot /snapshots/home_snapshot_202302 /snapshots/home_snapshot_202302_rw
Or it can be restored directly to /home
straight away like this:
btrfs subvolume delete /home
btrfs subvolume snapshot /snapshots/home_snapshot_202302 /home
After this you can mount /home
again.
Transfer snapshot
Sending the snapshot /snapshots/nixos_snapshot_202302
compressed to a remote host via ssh at root@192.168.178.110
and saving it to a subvolume mounted on a directory at /mnt/nixos
sudo btrfs send /snapshots/nixos_snapshot_202302 | zstd | ssh root@192.168.178.110 'zstd -d | btrfs receive /mnt/nixos'
Tips and tricks
Installation with encryption
Using Luks2:
cryptsetup --verify-passphrase -v luksFormat "$DISK"p2
cryptsetup open "$DISK"p2 enc
You can use any device partition for your bootloader. Note that this bootloader is unencrypted by default:
mkfs.vfat -n BOOT "$DISK"p1
Creating subvolumes
mkfs.btrfs /dev/mapper/enc # Creating btrfs partition
mount -t btrfs /dev/mapper/enc /mnt
# Create the subvolumes
btrfs subvolume create /mnt/root # The subvolume for /
btrfs subvolume create /mnt/home # The subvolume for /home, which should be backed up
btrfs subvolume create /mnt/nix # The subvolume for /nix, which needs to be persistent but is not worth backing up, as it’s trivial to reconstruct
btrfs subvolume create /mnt/log # The subvolume for /var/log.
Unmount to mount on the subvolumes for the next steps:
umount /mnt
Once the subvolumes have been created, mount them with the desired options.
Example with Zstandard compression and noatime:
mount -o subvol=root,compress=zstd,noatime /dev/mapper/enc /mnt
mkdir /mnt/home
mount -o subvol=home,compress=zstd,noatime /dev/mapper/enc /mnt/home
mkdir /mnt/nix
mount -o subvol=nix,compress=zstd,noatime /dev/mapper/enc /mnt/nix
mkdir -p /mnt/var/log
mount -o subvol=log,compress=zstd,noatime /dev/mapper/enc /mnt/var/log
# do not forget to create and mount the bootloader
mkdir /mnt/boot
mount "$DISK"p1 /mnt/boot
Configure hardware-configuration.nix
# enable btrfs support
boot.supportedFilesystems = [ "btrfs" ];
fileSystems."/var/log" =
{ device = "/dev/disk/by-uuid/X";
fsType = "btrfs";
# enable noatime and zstd to the other subvolumes aswell
options = [ "subvol=log" "compress=zstd" "noatime" ];
# to have a correct log order
neededForBoot = true;
};
Generate Nixconfig:
nixos-generate-config --root /mnt
Convert Ext3/Ext4 system partition to Btrfs
To convert the existing filesystem (Ext3/4) to Btrfs, boot into a NixOS live system and run the following commandː
fsck -f /dev/sdXY
btrfs-convert /dev/sdXY
Replace the device path with the target partition. Converting larger filesystems can take a long time. Next, mount the converted filesystem and chroot into itː
mount /dev/sdXY /mnt
nixos-enter --root /mnt
Replace the partition UUID with the new one, which can be obtained using the command blkid
, in /etc/nixos/hardware-configuration.nix
and also change the filesystem to btrfs
.
fileSystems."/" =
{ device = "/dev/disk/by-uuid/44444444-4444-4444-8888-888888888888";
fsType = "btrfs";
};
Apply the changes
nixos-rebuild boot
If the Grub bootloader is used and it doesn't get reinstalled correctly, you can run the following command inside chrootː
grub-install /dev/sdX
If the conversion was successful and no rollback is required, the backup image which was stored by btrfs-convert can be removed withː
btrfs subvolume delete /btrfs/ext2_saved