Jump to content

Bcachefs: Difference between revisions

From Official NixOS Wiki
imported>0x4A6F
mNo edit summary
Onny (talk | contribs)
NixOS installation on bcachefs: Cleanup instruction
 
(46 intermediate revisions by 10 users not shown)
Line 1: Line 1:
[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase, more stability, greater speed and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.  
[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase<ref name=":0">citation needed</ref>, more stability<ref name=":0" />, greater speed<ref name=":0" /> and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.  


== Installation ==
== Installation ==
Line 11: Line 11:
== Usage ==
== Usage ==


Format and mount a signle device
Format and mount a single device


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
Line 17: Line 17:
# mount -t bcachefs /dev/sda /mnt
# mount -t bcachefs /dev/sda /mnt
</syntaxhighlight>
</syntaxhighlight>
Or, format and mount multiple devices
<syntaxhighlight lang="console">
# bcachefs format /dev/sda:/dev/sdb:/dev/sdc
# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc
</syntaxhighlight>
The same works with partitions, which is probably better for future proofing depending on your specific needs
<syntaxhighlight lang="console">
# bcachefs format /dev/sda1:/dev/sdb2:/dev/sdc3
# mount -t bcachefs /dev/sda1:/dev/sdb2:/dev/sdc3
</syntaxhighlight>


Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:
Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:
Line 24: Line 39:
# bcachefs unlock /dev/sda
# bcachefs unlock /dev/sda
# mount -t bcachefs /dev/sda /mnt
# mount -t bcachefs /dev/sda /mnt
</syntaxhighlight>
Format a drive with compression on by default, foreground and background
(Available Compression options are <code>gzip</code>, <code>lz4</code>, and <code>zstd</code>)
<syntaxhighlight lang="console">
# bcachefs format --compression=lz4 --background_compression=zstd /dev/sda
# mount -t bcachefs /dev/sda
</syntaxhighlight>
Format a multiple devices with storage tiers, so that reads and writes happen on the fastest disks, with data being stored on slower, bigger drives based on usage patterns
<syntaxhighlight lang="console">
# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs /dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg
</syntaxhighlight>
For a better mounting experience in the previous example, use the external UUID that was printed.
<syntaxhighlight lang="console">
# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs UUID=<UUID>
</syntaxhighlight>
</syntaxhighlight>


Line 31: Line 83:
# bcachefs subvolume snapshot /mnt /mnt/snap1
# bcachefs subvolume snapshot /mnt /mnt/snap1
</syntaxhighlight>
</syntaxhighlight>
Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on <code>/dev/sda</code>:
<syntaxhighlight lang="console">
# bcachefs fsck /dev/sda
</syntaxhighlight>Change partition encryption password for <code>/dev/sda1</code><syntaxhighlight lang="console">
# bcachefs set-passphrase /dev/sda1
</syntaxhighlight>
== Configuration ==
Every option for the filesystem can be set by editing <code>/sys/fs/bcachefs/<uuid>/options</code>, for example the file <code>background_compression</code> will change the background compression scheme for background compression. These are persisted with the filesystems, so a bcachefs storage device being mounted on a different computer won't need to know what mount options to use to maintain the same compression levels.
Change encryption password for Bcachefs formatted device <code>/dev/sda1</code>
<syntaxhighlight lang="console">
# bcachefs set-passphrase /dev/sda1
</syntaxhighlight>
Enable zstd compression for device <code>/dev/sda1</code> at mount time
{{file|/etc/nixos/hardware-configuration.nix|nix|<nowiki>
fileSystems."/" =
{ device = "/dev/sda1";
  fsType = "bcachefs";
  options = [ "compression=zstd" ];
};
</nowiki>}}


== Tips and tricks ==
== Tips and tricks ==


=== Generate bcachefs enabled installation media ===
=== Generate bcachefs enabled installation media ===
Use following Nix [[Flakes|Flake-expression]] to generate a ISO installation image with a bcachefs enabled kernel
{{file|flake.nix|nix|<nowiki>
{
  description = "Bcachefs enabled installation media";
  inputs.nixos.url = "nixpkgs/nixos-25.11";
  outputs = { self, nixos }: {
    nixosConfigurations = {
      exampleIso = nixos.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
          ({ lib, pkgs, ... }: {
            # Required as a workaround for bug
            # https://github.com/NixOS/nixpkgs/issues/32279
            environment.systemPackages = [ pkgs.keyutils ];
            boot.supportedFilesystems = [ "bcachefs" ];
          })
        ];
      };
    };
  };
}
</nowiki>}}
The following commands will generate the iso-image which will be available in the directory <code>./result/iso</code>
<syntaxhighlight lang="console"># nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage
</syntaxhighlight>
=== NixOS installation on bcachefs ===
Using the installation media generated above, continue the installation as usual following the [https://nixos.org/manual/nixos/stable/index.html#ch-installation instructions of the NixOS manual].
For a UEFI installation, the partitioning needs to be adjusted as following
{{note|Be sure on which disk you'll perform these filesystem operations. All existing data on it will be erased.}}
<syntaxhighlight lang="console">
# parted /dev/sda -- mklabel gpt
# parted /dev/sda -- mkpart ESP fat32 1MB 512MB
# parted /dev/sda -- set 1 esp on
# parted /dev/sda -- mkpart primary 512MB 100%
</syntaxhighlight>
Formatting the boot partition <code>/dev/sda1</code> and the root filesystem <code>/dev/sda2</code>
<syntaxhighlight lang="console">
# mkfs.fat -F 32 -n boot /dev/sda1
# mkfs.bcachefs -L nixos /dev/sda2
</syntaxhighlight>


Use following Nix expression to generate a ISO installation image with a bcachefs enabled kernel
Formatting and unlocking the encrypted partition would look like this


<syntaxhighlight lang="nix">
<syntaxhighlight lang="console">
# iso.nix
# keyctl link @u @s
# This module defines a small NixOS installation CD.  It does not
# bcachefs format --label=nixos --encrypted /dev/sda2
# contain any graphical stuff.
# bcachefs unlock /dev/sda2
{ config, pkgs, ... }:
{
  imports = [
    <nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix>
  ];
  boot.supportedFilesystems = [ "bcachefs" ];
  isoImage.squashfsCompression = "gzip -Xcompression-level 1";
}
</syntaxhighlight>
</syntaxhighlight>
Mount filesystems. Use <code>lsblk -o +uuid,fsType | grep bcachefs</code> to get bcachefs partition uuid.


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# nix-build '<nixpkgs/nixos>' -A config.system.build.isoImage -I nixos-config=iso.nix
# mount /dev/disk/by-uuid/<...> /mnt
# mkdir /mnt/boot
# mount /dev/disk/by-label/boot /mnt/boot
</syntaxhighlight>
</syntaxhighlight>
Continue installation as recommended by the [https://nixos.org/manual/nixos/stable/index.html#ch-installation NixOS manual].
=== Remote encrypted disk unlocking ===
See article on [[Remote disk unlocking#Bcachefs unlocking|remote disk unlocking]] for a guide on how to enable SSH decryption of Bcachefs enabled systems.
=== Automatically mount encrypted device on boot ===
Since the Bcachefs mount options do [https://github.com/koverstreet/bcachefs-tools/pull/266 not support supplying a key file yet], we could use the <code>bcachefs</code> command and run it on boot using a [[Systemd]] unit:<syntaxhighlight lang="nix">
systemd.services."bcachefs-mount" = {
  after = [ "local-fs.target" ];
  wantedBy = [ "multi-user.target" ];
  environment = {
    DEVICE_PATH = "/dev/sda1";
    MOUNT_POINT = "/mnt";
  };
  script = ''
    #!${pkgs.runtimeShell} -e
    ${pkgs.keyutils}/bin/keyctl link @u @s
    # Check if the device path exists
    if [ ! -b "$DEVICE_PATH" ]; then
      echo "Error: Device path $DEVICE_PATH does not exist."
      exit 1
    fi
    # Check if the drive is already mounted
    if ${pkgs.util-linux}/bin/mountpoint -q "$MOUNT_POINT"; then
      echo "Drive already mounted at $MOUNT_POINT. Skipping..."
      exit 0
    fi
    # Wait for the device to become available
    while [ ! -b "$DEVICE_PATH" ]; do
      echo "Waiting for $DEVICE_PATH to become available..."
      sleep 5
    done
    # Mount the device
    ${pkgs.bcachefs-tools}/bin/bcachefs mount -f /etc/keyfile_test "$DEVICE_PATH" "$MOUNT_POINT"
  '';
  serviceConfig = {
    Type = "oneshot";
    User = "root";
  };
};
</syntaxhighlight>This example unit mounts the Bcachefs encrypted partition <code>/dev/sda1</code> to the target <code>/mnt</code> by using the key file <code>/etc/keyfile_test</code>.
[[Category:Filesystem]]

Latest revision as of 13:27, 9 December 2025

Bcachefs is a next-generation CoW filesystem that aims to provide features from Btrfs and ZFS with a cleaner codebase[1], more stability[1], greater speed[1] and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.

Installation

To enable filesystem support and availability of user-space utils, add following line to the system configuration

❄︎ /etc/nixos/configuration.nix
boot.supportedFilesystems = [ "bcachefs" ];

Usage

Format and mount a single device

# bcachefs format /dev/sda
# mount -t bcachefs /dev/sda /mnt

Or, format and mount multiple devices

# bcachefs format /dev/sda:/dev/sdb:/dev/sdc
# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc

The same works with partitions, which is probably better for future proofing depending on your specific needs

# bcachefs format /dev/sda1:/dev/sdb2:/dev/sdc3
# mount -t bcachefs /dev/sda1:/dev/sdb2:/dev/sdc3


Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:

# bcachefs format --encrypt /dev/sda
# bcachefs unlock /dev/sda
# mount -t bcachefs /dev/sda /mnt

Format a drive with compression on by default, foreground and background (Available Compression options are gzip, lz4, and zstd)

# bcachefs format --compression=lz4 --background_compression=zstd /dev/sda 
# mount -t bcachefs /dev/sda

Format a multiple devices with storage tiers, so that reads and writes happen on the fastest disks, with data being stored on slower, bigger drives based on usage patterns

# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs /dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg

For a better mounting experience in the previous example, use the external UUID that was printed.

# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs UUID=<UUID>

Create a subvolume of a mounted bcachefs filesystem. The snapshot of the filesystem state is accessible in the directory /mnt/snap1.

# bcachefs subvolume snapshot /mnt /mnt/snap1

Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on /dev/sda:

# bcachefs fsck /dev/sda

Change partition encryption password for /dev/sda1

# bcachefs set-passphrase /dev/sda1

Configuration

Every option for the filesystem can be set by editing /sys/fs/bcachefs/<uuid>/options, for example the file background_compression will change the background compression scheme for background compression. These are persisted with the filesystems, so a bcachefs storage device being mounted on a different computer won't need to know what mount options to use to maintain the same compression levels.

Change encryption password for Bcachefs formatted device /dev/sda1

# bcachefs set-passphrase /dev/sda1

Enable zstd compression for device /dev/sda1 at mount time

❄︎ /etc/nixos/hardware-configuration.nix
fileSystems."/" =
{ device = "/dev/sda1";
  fsType = "bcachefs";
  options = [ "compression=zstd" ];
};

Tips and tricks

Generate bcachefs enabled installation media

Use following Nix Flake-expression to generate a ISO installation image with a bcachefs enabled kernel

❄︎ flake.nix
{
  description = "Bcachefs enabled installation media";
  inputs.nixos.url = "nixpkgs/nixos-25.11";
	
  outputs = { self, nixos }: {
    nixosConfigurations = {
      exampleIso = nixos.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
          ({ lib, pkgs, ... }: {
            # Required as a workaround for bug
            # https://github.com/NixOS/nixpkgs/issues/32279
            environment.systemPackages = [ pkgs.keyutils ];
            boot.supportedFilesystems = [ "bcachefs" ];
          })
        ];
      };
    };
  };
}

The following commands will generate the iso-image which will be available in the directory ./result/iso

# nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage

NixOS installation on bcachefs

Using the installation media generated above, continue the installation as usual following the instructions of the NixOS manual.

For a UEFI installation, the partitioning needs to be adjusted as following

Note: Be sure on which disk you'll perform these filesystem operations. All existing data on it will be erased.
# parted /dev/sda -- mklabel gpt
# parted /dev/sda -- mkpart ESP fat32 1MB 512MB
# parted /dev/sda -- set 1 esp on
# parted /dev/sda -- mkpart primary 512MB 100%

Formatting the boot partition /dev/sda1 and the root filesystem /dev/sda2

# mkfs.fat -F 32 -n boot /dev/sda1
# mkfs.bcachefs -L nixos /dev/sda2

Formatting and unlocking the encrypted partition would look like this

# keyctl link @u @s
# bcachefs format --label=nixos --encrypted /dev/sda2
# bcachefs unlock /dev/sda2

Mount filesystems. Use lsblk -o +uuid,fsType | grep bcachefs to get bcachefs partition uuid.

# mount /dev/disk/by-uuid/<...> /mnt
# mkdir /mnt/boot
# mount /dev/disk/by-label/boot /mnt/boot

Continue installation as recommended by the NixOS manual.

Remote encrypted disk unlocking

See article on remote disk unlocking for a guide on how to enable SSH decryption of Bcachefs enabled systems.

Automatically mount encrypted device on boot

Since the Bcachefs mount options do not support supplying a key file yet, we could use the bcachefs command and run it on boot using a Systemd unit:

systemd.services."bcachefs-mount" = {
  after = [ "local-fs.target" ];
  wantedBy = [ "multi-user.target" ];
  environment = {
    DEVICE_PATH = "/dev/sda1";
    MOUNT_POINT = "/mnt";
  };
  script = ''
    #!${pkgs.runtimeShell} -e

    ${pkgs.keyutils}/bin/keyctl link @u @s

    # Check if the device path exists
    if [ ! -b "$DEVICE_PATH" ]; then
      echo "Error: Device path $DEVICE_PATH does not exist."
      exit 1
    fi

    # Check if the drive is already mounted
    if ${pkgs.util-linux}/bin/mountpoint -q "$MOUNT_POINT"; then
      echo "Drive already mounted at $MOUNT_POINT. Skipping..."
      exit 0
    fi

    # Wait for the device to become available
    while [ ! -b "$DEVICE_PATH" ]; do
      echo "Waiting for $DEVICE_PATH to become available..."
      sleep 5
    done

    # Mount the device
    ${pkgs.bcachefs-tools}/bin/bcachefs mount -f /etc/keyfile_test "$DEVICE_PATH" "$MOUNT_POINT"
  '';
  serviceConfig = {
    Type = "oneshot";
    User = "root";
  };
};

This example unit mounts the Bcachefs encrypted partition /dev/sda1 to the target /mnt by using the key file /etc/keyfile_test.

  1. 1.0 1.1 1.2 citation needed