Jump to content

Bcachefs: Difference between revisions

From Official NixOS Wiki
Onny (talk | contribs)
→Tips and tricks: Add note on Bcachefs remote disk unlocking
Onny (talk | contribs)
 
(22 intermediate revisions by 5 users not shown)
Line 1: Line 1:
[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase, more stability, greater speed and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.  
[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase<ref name=":0">citation needed</ref>, more stability<ref name=":0" />, greater speed<ref name=":0" /> and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.  


== Installation ==
== Installation ==
Line 36: Line 36:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# bcachefs format --encrypt /dev/sda
# bcachefs format --encrypted /dev/sda
# bcachefs unlock /dev/sda
# bcachefs unlock /dev/sda
# mount -t bcachefs /dev/sda /mnt
# mount -t bcachefs /dev/sda /mnt
Line 57: Line 57:
     --label=hdd.hdd3 /dev/sde \
     --label=hdd.hdd3 /dev/sde \
     --label=ssd.ssd1 /dev/sdf \
     --label=ssd.ssd1 /dev/sdf \
     --label=ssd.ssd2 /dev/sdg
     --label=ssd.ssd2 /dev/sdg \
     --foreground_target=ssd \
     --foreground_target=ssd \
     --promote_target=ssd \
     --promote_target=ssd \
Line 71: Line 71:
     --label=hdd.hdd3 /dev/sde \
     --label=hdd.hdd3 /dev/sde \
     --label=ssd.ssd1 /dev/sdf \
     --label=ssd.ssd1 /dev/sdf \
     --label=ssd.ssd2 /dev/sdg
     --label=ssd.ssd2 /dev/sdg \
     --foreground_target=ssd \
     --foreground_target=ssd \
     --promote_target=ssd \
     --promote_target=ssd \
Line 82: Line 82:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# bcachefs subvolume snapshot /mnt /mnt/snap1
# bcachefs subvolume snapshot /mnt /mnt/snap1
</syntaxhighlight>
Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on <code>/dev/sda</code>:
<syntaxhighlight lang="console">
# bcachefs fsck /dev/sda
</syntaxhighlight>Change partition encryption password for <code>/dev/sda1</code><syntaxhighlight lang="console">
# bcachefs set-passphrase /dev/sda1
</syntaxhighlight>
</syntaxhighlight>


Line 106: Line 114:
== Tips and tricks ==
== Tips and tricks ==


=== Generate bcachefs enabled installation media ===
=== Generate bcachefs enabled installation media ===
 
Use following Nix [[Flakes|Flake-expression]] to generate a ISO installation image with a bcachefs enabled kernel
Use following Nix [[Flakes|Flake-expression]] to generate a ISO installation image with a bcachefs enabled kernel
 
{{file|flake.nix|nix|<nowiki>
{{file|flake.nix|nix|<nowiki>
{
{
   description = "Bcachefs enabled installation media";
   description = "Bcachefs enabled installation media";
   inputs.nixos.url = "nixpkgs/nixos-23.11";
   inputs.nixos.url = "nixpkgs/nixos-25.11";
   outputs = { self, nixos }: {
   outputs = { self, nixos }: {
     nixosConfigurations = {
     nixosConfigurations = {
Line 121: Line 131:
           "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
           "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
           ({ lib, pkgs, ... }: {
           ({ lib, pkgs, ... }: {
            # Required as a workaround for bug
            # https://github.com/NixOS/nixpkgs/issues/32279
            environment.systemPackages = [ pkgs.keyutils ];
             boot.supportedFilesystems = [ "bcachefs" ];
             boot.supportedFilesystems = [ "bcachefs" ];
            boot.kernelPackages = lib.mkOverride 0 pkgs.linuxPackages_latest;
           })
           })
         ];
         ];
Line 132: Line 144:


The following commands will generate the iso-image which will be available in the directory <code>./result/iso</code>
The following commands will generate the iso-image which will be available in the directory <code>./result/iso</code>
 
<syntaxhighlight lang="console">
<syntaxhighlight lang="console"># nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage
# git init
# git add flake.nix
# nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage
</syntaxhighlight>
</syntaxhighlight>


Line 161: Line 170:
</syntaxhighlight>
</syntaxhighlight>


In case you want to enable filesystem encryption, there's a workaround for [https://github.com/NixOS/nixpkgs/issues/32279 a bug] affecting NixOS 23.11. Formatting and unlocking the encrypted partition would look like this
Formatting and unlocking the encrypted partition would look like this


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# nix-env -iA nixos.keyutils
# keyctl link @u @s
# keyctl link @u @s
# bcachefs format --encrypted /dev/sda2
# bcachefs format --label=nixos --encrypted /dev/sda2
# bcachefs unlock /dev/sda2
# bcachefs unlock /dev/sda2
</syntaxhighlight>
</syntaxhighlight>


Mount filesystems
Mount filesystems. Use <code>lsblk -o +uuid,fsType | grep bcachefs</code> to get bcachefs partition uuid.
 
There are two ways to mount a bcachefs filesystem. First, by device:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# mount /dev/sda2 /mnt
# mount /dev/disk/by-uuid/<...> /mnt
# mkdir /mnt/boot
# mkdir /mnt/boot
# mount /dev/disk/by-label/boot /mnt/boot
# mount /dev/disk/by-label/boot /mnt/boot
</syntaxhighlight>
</syntaxhighlight>
For better reliability as with any other filesystem, one should mount the filesystem by UUID, by replacing any drive specifiers with it.
When you formatted, there was an external UUID shown, but if you lost it, you can retrieve it using
<syntaxhighlight lang="console">
# lsblk -o +uuid,fsType | grep bcachefs
</syntaxhighlight>
And using it like <code>UUID=<UUID></code> in place of <code>/dev/sda1:/dev/sdb1</code> or even just <code>/dev/sda</code>.


Continue installation as recommended by the [https://nixos.org/manual/nixos/stable/index.html#ch-installation NixOS manual].
Continue installation as recommended by the [https://nixos.org/manual/nixos/stable/index.html#ch-installation NixOS manual].


Before <code>nixos-install</code> you'll have to adapt the generated NixOS configuration to also use the latest Linux kernel supporting Bcachefs. Add the following line to the <code>configuration.nix</code> in your installation root (/mnt):
=== Remote encrypted disk unlocking ===
See article on [[Remote disk unlocking#Bcachefs unlocking|remote disk unlocking]] for a guide on how to enable SSH decryption of Bcachefs enabled systems.


{{file|/mnt/etc/nixos/configuration.nix|nix|<nowiki>
=== Automatically mount encrypted device on boot ===
boot.supportedFilesystems = [ "bcachefs" ];
Since the Bcachefs mount options do [https://github.com/koverstreet/bcachefs-tools/pull/266 not support supplying a key file yet], we could use the <code>bcachefs unlock</code> command and run it on boot using a [[Systemd]] unit:<syntaxhighlight lang="nix">fileSystems."/mnt" = {
boot.kernelPackages = pkgs.linuxPackages_latest;
  device = "/dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97";
</nowiki>}}
  fsType = "bcachefs";
  options = [ "nofail" ];
};


=== Remote encrypted disk unlocking ===
# Ensure to match the correct systemd unit name which gets created by NixOS
See article on [[Remote disk unlocking#Bcachefs unlocking|remote disk unlocking]] for a guide on how to enable SSH decryption of Bcachefs enabled systems.
# in the first place. We override the script part.
systemd.services."unlock-bcachefs-mnt" = {
  serviceConfig.LoadCredential = [ "bcachefs-mnt:/etc/secret.key" ];
  script = lib.mkForce ''
    ${lib.getExe' pkgs.keyutils "keyctl"} link @u @s
    ${config.boot.initrd.systemd.package}/bin/systemd-ask-password --credential=bcachefs-mnt --timeout=0 "enter passphrase for /mnt" | \
      exec ${lib.getExe pkgs.bcachefs-tools} unlock \
      "/dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97"
  '';
};</syntaxhighlight>This example unit unlocks the Bcachefs encrypted partition <code>/dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97</code> whereas the fstab entry mounts it to the target <code>/mnt</code> by using the key file <code>/etc/secret.key</code>. Ensure that you replace all disk uuid and target file path occurences.
[[Category:Filesystem]]
[[Category:Filesystem]]

Latest revision as of 15:20, 12 August 2026

Bcachefs is a next-generation CoW filesystem that aims to provide features from Btrfs and ZFS with a cleaner codebase[1], more stability[1], greater speed[1] and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.

Installation

To enable filesystem support and availability of user-space utils, add following line to the system configuration

❄︎ /etc/nixos/configuration.nix
boot.supportedFilesystems = [ "bcachefs" ];

Usage

Format and mount a single device

# bcachefs format /dev/sda
# mount -t bcachefs /dev/sda /mnt

Or, format and mount multiple devices

# bcachefs format /dev/sda:/dev/sdb:/dev/sdc
# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc

The same works with partitions, which is probably better for future proofing depending on your specific needs

# bcachefs format /dev/sda1:/dev/sdb2:/dev/sdc3
# mount -t bcachefs /dev/sda1:/dev/sdb2:/dev/sdc3


Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:

# bcachefs format --encrypted /dev/sda
# bcachefs unlock /dev/sda
# mount -t bcachefs /dev/sda /mnt

Format a drive with compression on by default, foreground and background (Available Compression options are gzip, lz4, and zstd)

# bcachefs format --compression=lz4 --background_compression=zstd /dev/sda 
# mount -t bcachefs /dev/sda

Format a multiple devices with storage tiers, so that reads and writes happen on the fastest disks, with data being stored on slower, bigger drives based on usage patterns

# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs /dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg

For a better mounting experience in the previous example, use the external UUID that was printed.

# bcachefs format \
    --label=hdd.hdd1 /dev/sdc \
    --label=hdd.hdd2 /dev/sdd \
    --label=hdd.hdd3 /dev/sde \
    --label=ssd.ssd1 /dev/sdf \
    --label=ssd.ssd2 /dev/sdg \
    --foreground_target=ssd \
    --promote_target=ssd \
    --background_target=hdd \
# mount -t bcachefs UUID=<UUID>

Create a subvolume of a mounted bcachefs filesystem. The snapshot of the filesystem state is accessible in the directory /mnt/snap1.

# bcachefs subvolume snapshot /mnt /mnt/snap1

Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on /dev/sda:

# bcachefs fsck /dev/sda

Change partition encryption password for /dev/sda1

# bcachefs set-passphrase /dev/sda1

Configuration

Every option for the filesystem can be set by editing /sys/fs/bcachefs/<uuid>/options, for example the file background_compression will change the background compression scheme for background compression. These are persisted with the filesystems, so a bcachefs storage device being mounted on a different computer won't need to know what mount options to use to maintain the same compression levels.

Change encryption password for Bcachefs formatted device /dev/sda1

# bcachefs set-passphrase /dev/sda1

Enable zstd compression for device /dev/sda1 at mount time

❄︎ /etc/nixos/hardware-configuration.nix
fileSystems."/" =
{ device = "/dev/sda1";
  fsType = "bcachefs";
  options = [ "compression=zstd" ];
};

Tips and tricks

Generate bcachefs enabled installation media

Use following Nix Flake-expression to generate a ISO installation image with a bcachefs enabled kernel

❄︎ flake.nix
{
  description = "Bcachefs enabled installation media";
  inputs.nixos.url = "nixpkgs/nixos-25.11";
	
  outputs = { self, nixos }: {
    nixosConfigurations = {
      exampleIso = nixos.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
          ({ lib, pkgs, ... }: {
            # Required as a workaround for bug
            # https://github.com/NixOS/nixpkgs/issues/32279
            environment.systemPackages = [ pkgs.keyutils ];
            boot.supportedFilesystems = [ "bcachefs" ];
          })
        ];
      };
    };
  };
}

The following commands will generate the iso-image which will be available in the directory ./result/iso

# nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage

NixOS installation on bcachefs

Using the installation media generated above, continue the installation as usual following the instructions of the NixOS manual.

For a UEFI installation, the partitioning needs to be adjusted as following

Note: Be sure on which disk you'll perform these filesystem operations. All existing data on it will be erased.
# parted /dev/sda -- mklabel gpt
# parted /dev/sda -- mkpart ESP fat32 1MB 512MB
# parted /dev/sda -- set 1 esp on
# parted /dev/sda -- mkpart primary 512MB 100%

Formatting the boot partition /dev/sda1 and the root filesystem /dev/sda2

# mkfs.fat -F 32 -n boot /dev/sda1
# mkfs.bcachefs -L nixos /dev/sda2

Formatting and unlocking the encrypted partition would look like this

# keyctl link @u @s
# bcachefs format --label=nixos --encrypted /dev/sda2
# bcachefs unlock /dev/sda2

Mount filesystems. Use lsblk -o +uuid,fsType | grep bcachefs to get bcachefs partition uuid.

# mount /dev/disk/by-uuid/<...> /mnt
# mkdir /mnt/boot
# mount /dev/disk/by-label/boot /mnt/boot

Continue installation as recommended by the NixOS manual.

Remote encrypted disk unlocking

See article on remote disk unlocking for a guide on how to enable SSH decryption of Bcachefs enabled systems.

Automatically mount encrypted device on boot

Since the Bcachefs mount options do not support supplying a key file yet, we could use the bcachefs unlock command and run it on boot using a Systemd unit:

fileSystems."/mnt" = {
  device = "/dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97";
  fsType = "bcachefs";
  options = [ "nofail" ];
};

# Ensure to match the correct systemd unit name which gets created by NixOS
# in the first place. We override the script part.
systemd.services."unlock-bcachefs-mnt" = {
  serviceConfig.LoadCredential = [ "bcachefs-mnt:/etc/secret.key" ];
  script = lib.mkForce ''
    ${lib.getExe' pkgs.keyutils "keyctl"} link @u @s
    ${config.boot.initrd.systemd.package}/bin/systemd-ask-password --credential=bcachefs-mnt --timeout=0 "enter passphrase for /mnt" | \
      exec ${lib.getExe pkgs.bcachefs-tools} unlock \
      "/dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97"
  '';
};

This example unit unlocks the Bcachefs encrypted partition /dev/disk/by-uuid/3c0d7d93-3293-49a3-842e-d9ef77576d97 whereas the fstab entry mounts it to the target /mnt by using the key file /etc/secret.key. Ensure that you replace all disk uuid and target file path occurences.

  1. ↑ 1.0 1.1 1.2 citation needed