Bcachefs: Difference between revisions

(20 intermediate revisions by 8 users not shown)

Line 1:

[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase, more stability, greater speed and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.

[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase<ref name=":0">citation needed</ref>, more stability<ref name=":0" />, greater speed<ref name=":0" /> and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.

== Installation ==

Line 17:

# mount -t bcachefs /dev/sda /mnt

</syntaxhighlight>

Or, format and mount multiple devices

# bcachefs format /dev/sda:/dev/sdb:/dev/sdc

# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc

</syntaxhighlight>

The same works with partitions, which is probably better for future proofing depending on your specific needs

# bcachefs format /dev/sda1:/dev/sdb2:/dev/sdc3

# mount -t bcachefs /dev/sda1:/dev/sdb2:/dev/sdc3

</syntaxhighlight>

Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:

Line 24:

Line 39:

# bcachefs unlock /dev/sda

# mount -t bcachefs /dev/sda /mnt

</syntaxhighlight>

Format a drive with compression on by default, foreground and background

(Available Compression options are <code>gzip</code>, <code>lz4</code>, and <code>zstd</code>)

# bcachefs format --compression=lz4 --background_compression=zstd /dev/sda

# mount -t bcachefs /dev/sda

</syntaxhighlight>

Format a multiple devices with storage tiers, so that reads and writes happen on the fastest disks, with data being stored on slower, bigger drives based on usage patterns

# bcachefs format \

--label=hdd.hdd1 /dev/sdc \

--label=hdd.hdd2 /dev/sdd \

--label=hdd.hdd3 /dev/sde \

--label=ssd.ssd1 /dev/sdf \

--label=ssd.ssd2 /dev/sdg

--foreground_target=ssd \

--promote_target=ssd \

--background_target=hdd \

# mount -t bcachefs /dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg

</syntaxhighlight>

For a better mounting experience in the previous example, use the external UUID that was printed.

# bcachefs format \

--label=hdd.hdd1 /dev/sdc \

--label=hdd.hdd2 /dev/sdd \

--label=hdd.hdd3 /dev/sde \

--label=ssd.ssd1 /dev/sdf \

--label=ssd.ssd2 /dev/sdg

--foreground_target=ssd \

--promote_target=ssd \

--background_target=hdd \

# mount -t bcachefs UUID=<UUID>

</syntaxhighlight>

Line 30:

Line 82:

# bcachefs subvolume snapshot /mnt /mnt/snap1

</syntaxhighlight>

Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on <code>/dev/sda</code>:

# bcachefs fsck /dev/sda

</syntaxhighlight>Change partition encryption password for <code>/dev/sda1</code><syntaxhighlight lang="console">

# bcachefs set-passphrase /dev/sda1

</syntaxhighlight>

== Configuration ==

Every option for the filesystem can be set by editing <code>/sys/fs/bcachefs/<uuid>/options</code>, for example the file <code>background_compression</code> will change the background compression scheme for background compression. These are persisted with the filesystems, so a bcachefs storage device being mounted on a different computer won't need to know what mount options to use to maintain the same compression levels.

Change encryption password for Bcachefs formatted device <code>/dev/sda1</code>

Line 40:

Line 102:

</syntaxhighlight>

Enable zstd compression for device <code>/dev/sda1</code>

Enable zstd compression for device <code>/dev/sda1</code> at mount time

{{file|/etc/nixos/hardware-configuration.nix|nix|<nowiki>

Line 59:

Line 121:

{

description = "Bcachefs enabled installation media";

inputs.nixos.url = "nixpkgs/nixos-23.11";

inputs.nixos.url = "nixpkgs/nixos-24.11";

outputs = { self, nixos }: {

nixosConfigurations = {

Line 67:

Line 129:

"${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"

({ lib, pkgs, ... }: {

# Might be required as a workaround for bug

# https://github.com/NixOS/nixpkgs/issues/32279

environment.systemPackages = [ pkgs.keyutils ];

boot.supportedFilesystems = [ "bcachefs" ];

~~boot.kernelPackages = lib.mkOverride 0 pkgs.linuxPackages_testing;~~

})

];

Line 80:

Line 144:

~~# git init~~

~~# git add flake.nix~~

# nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage

</syntaxhighlight>

Line 107:

Line 169:

</syntaxhighlight>

~~In case you want to enable filesystem encryption, there's a workaround for [https://github.com/NixOS/nixpkgs/issues/32279 a bug] affecting NixOS 23.05.~~ Formatting and unlocking the encrypted partition would look like this

Formatting and unlocking the encrypted partition would look like this

# nix-~~env~~ -~~iA nixos.~~keyutils

# nix-shell -p keyutils --run 'keyctl link @u @s'

# keyctl link @u @s

# bcachefs format --encrypted /dev/sda2

# bcachefs format --~~encrypt~~ /dev/sda2

# bcachefs unlock /dev/sda2

</syntaxhighlight>

Mount filesystems

There are two ways to mount a bcachefs filesystem. First, by device:

Line 123:

Line 186:

# mount /dev/disk/by-label/boot /mnt/boot

</syntaxhighlight>

For better reliability as with any other filesystem, one should mount the filesystem by UUID, by replacing any drive specifiers with it.

When you formatted, there was an external UUID shown, but if you lost it, you can retrieve it using

# lsblk -o +uuid,fsType | grep bcachefs

</syntaxhighlight>

And using it like <code>UUID=<UUID></code> in place of <code>/dev/sda1:/dev/sdb1</code> or even just <code>/dev/sda</code>.

Note: this is currently broken as per this [https://github.com/NixOS/nixpkgs/issues/317901 github issue]. As a workaround, use `/dev/disk/by-uuid/<UUID>` or `/dev/disk/by-id` instead. This seems to work for single device systems atleast, although atleast at the time of this writing I could not get it to work on my 4+ device raid system.

Continue installation as recommended by the [https://nixos.org/manual/nixos/stable/index.html#ch-installation NixOS manual].

Before <code>nixos-install</code> you'll have to adapt the generated NixOS configuration to also use the latest Linux kernel supporting Bcachefs. Add the following line to the <code>configuration.nix</code> in your installation root (/mnt):

{{file|/mnt/etc/nixos/configuration.nix|nix|<nowiki>

boot.supportedFilesystems = [ "bcachefs" ];

boot.kernelPackages = pkgs.linuxPackages_latest;

</nowiki>}}

=== Remote encrypted disk unlocking ===

See article on [[Remote disk unlocking#Bcachefs unlocking|remote disk unlocking]] for a guide on how to enable SSH decryption of Bcachefs enabled systems.

=== Automatically mount encrypted device on boot ===

Since the Bcachefs mount options do [https://github.com/koverstreet/bcachefs-tools/pull/266 not support supplying a key file yet], we could use the <code>bcachefs</code> command and run it on boot using a [[Systemd]] unit:<syntaxhighlight lang="nix">

systemd.services."bcachefs-mount" = {

after = [ "local-fs.target" ];

wantedBy = [ "multi-user.target" ];

environment = {

DEVICE_PATH = "/dev/sda1";

MOUNT_POINT = "/mnt";

};

script = ''

#!${pkgs.runtimeShell} -e

${pkgs.keyutils}/bin/keyctl link @u @s

# Check if the device path exists

if [ ! -b "$DEVICE_PATH" ]; then

echo "Error: Device path $DEVICE_PATH does not exist."

exit 1

fi

# Check if the drive is already mounted

if ${pkgs.util-linux}/bin/mountpoint -q "$MOUNT_POINT"; then

echo "Drive already mounted at $MOUNT_POINT. Skipping..."

exit 0

fi

# Wait for the device to become available

while [ ! -b "$DEVICE_PATH" ]; do

echo "Waiting for $DEVICE_PATH to become available..."

sleep 5

done

# Mount the device

${pkgs.bcachefs-tools}/bin/bcachefs mount -f /etc/keyfile_test "$DEVICE_PATH" "$MOUNT_POINT"

'';

serviceConfig = {

Type = "oneshot";

User = "root";

};

</syntaxhighlight>This example unit mounts the Bcachefs encrypted partition <code>/dev/sda1</code> to the target <code>/mnt</code> by using the key file <code>/etc/keyfile_test</code>.

[[Category:Filesystem]]

@@ Line 1: / Line 1: @@
-[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase, more stability, greater speed and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.
+[https://bcachefs.org Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase<ref name=":0">citation needed</ref>, more stability<ref name=":0" />, greater speed<ref name=":0" /> and a GPL-compatible license. It is built upon Bcache and is mainly developed by Kent Overstreet.
 == Installation ==
@@ Line 17: / Line 17: @@
 # mount -t bcachefs /dev/sda /mnt
 </syntaxhighlight>
+Or, format and mount multiple devices
+<syntaxhighlight lang="console">
+# bcachefs format /dev/sda:/dev/sdb:/dev/sdc
+# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc
+</syntaxhighlight>
+The same works with partitions, which is probably better for future proofing depending on your specific needs
+<syntaxhighlight lang="console">
+# bcachefs format /dev/sda1:/dev/sdb2:/dev/sdc3
+# mount -t bcachefs /dev/sda1:/dev/sdb2:/dev/sdc3
+</syntaxhighlight>
 Format drive with encryption enabled, unlock and mount it afterwards. Following bcachefs commands will ask for a password:
@@ Line 24: / Line 39: @@
 # bcachefs unlock /dev/sda
 # mount -t bcachefs /dev/sda /mnt
+</syntaxhighlight>
+Format a drive with compression on by default, foreground and background
+(Available Compression options are <code>gzip</code>, <code>lz4</code>, and <code>zstd</code>)
+<syntaxhighlight lang="console">
+# bcachefs format --compression=lz4 --background_compression=zstd /dev/sda
+# mount -t bcachefs /dev/sda
+</syntaxhighlight>
+Format a multiple devices with storage tiers, so that reads and writes happen on the fastest disks, with data being stored on slower, bigger drives based on usage patterns
+<syntaxhighlight lang="console">
+# bcachefs format \
+    --label=hdd.hdd1 /dev/sdc \
+    --label=hdd.hdd2 /dev/sdd \
+    --label=hdd.hdd3 /dev/sde \
+    --label=ssd.ssd1 /dev/sdf \
+    --label=ssd.ssd2 /dev/sdg
+    --foreground_target=ssd \
+    --promote_target=ssd \
+    --background_target=hdd \
+# mount -t bcachefs /dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg
+</syntaxhighlight>
+For a better mounting experience in the previous example, use the external UUID that was printed.
+<syntaxhighlight lang="console">
+# bcachefs format \
+    --label=hdd.hdd1 /dev/sdc \
+    --label=hdd.hdd2 /dev/sdd \
+    --label=hdd.hdd3 /dev/sde \
+    --label=ssd.ssd1 /dev/sdf \
+    --label=ssd.ssd2 /dev/sdg
+    --foreground_target=ssd \
+    --promote_target=ssd \
+    --background_target=hdd \
+# mount -t bcachefs UUID=<UUID>
 </syntaxhighlight>
@@ Line 30: / Line 82: @@
 <syntaxhighlight lang="console">
 # bcachefs subvolume snapshot /mnt /mnt/snap1
+</syntaxhighlight>
+Filesystem check, fix errors and corruptions where a Bcachefs filesystem is on <code>/dev/sda</code>:
+<syntaxhighlight lang="console">
+# bcachefs fsck /dev/sda
+</syntaxhighlight>Change partition encryption password for <code>/dev/sda1</code><syntaxhighlight lang="console">
+# bcachefs set-passphrase /dev/sda1
 </syntaxhighlight>
 == Configuration ==
+Every option for the filesystem can be set by editing <code>/sys/fs/bcachefs/<uuid>/options</code>, for example the file <code>background_compression</code> will change the background compression scheme for background compression. These are persisted with the filesystems, so a bcachefs storage device being mounted on a different computer won't need to know what mount options to use to maintain the same compression levels.
 Change encryption password for Bcachefs formatted device <code>/dev/sda1</code>
@@ Line 40: / Line 102: @@
 </syntaxhighlight>
-Enable zstd compression for device <code>/dev/sda1</code>
+Enable zstd compression for device <code>/dev/sda1</code> at mount time
 {{file|/etc/nixos/hardware-configuration.nix|nix|<nowiki>
@@ Line 59: / Line 121: @@
 {
    description = "Bcachefs enabled installation media";
-   inputs.nixos.url = "nixpkgs/nixos-23.11";
+   inputs.nixos.url = "nixpkgs/nixos-24.11";
    outputs = { self, nixos }: {
      nixosConfigurations = {
@@ Line 67: / Line 129: @@
            "${nixos}/nixos/modules/installer/cd-dvd/installation-cd-minimal-new-kernel-no-zfs.nix"
            ({ lib, pkgs, ... }: {
+            # Might be required as a workaround for bug
+            # https://github.com/NixOS/nixpkgs/issues/32279
+            environment.systemPackages = [ pkgs.keyutils ];
              boot.supportedFilesystems = [ "bcachefs" ];
-            boot.kernelPackages = lib.mkOverride 0 pkgs.linuxPackages_testing;
            })
          ];
@@ Line 80: / Line 144: @@
 <syntaxhighlight lang="console">
-# git init
-# git add flake.nix
 # nix build .#nixosConfigurations.exampleIso.config.system.build.isoImage
 </syntaxhighlight>
@@ Line 107: / Line 169: @@
 </syntaxhighlight>
-In case you want to enable filesystem encryption, there's a workaround for [https://github.com/NixOS/nixpkgs/issues/32279 a bug] affecting NixOS 23.05. Formatting and unlocking the encrypted partition would look like this
+Formatting and unlocking the encrypted partition would look like this
 <syntaxhighlight lang="console">
-# nix-env -iA nixos.keyutils
+# nix-shell -p keyutils --run 'keyctl link @u @s'
-# keyctl link @u @s
+# bcachefs format --encrypted /dev/sda2
-# bcachefs format --encrypt /dev/sda2
 # bcachefs unlock /dev/sda2
 </syntaxhighlight>
 Mount filesystems
+There are two ways to mount a bcachefs filesystem. First, by device:
 <syntaxhighlight lang="console">
@@ Line 123: / Line 186: @@
 # mount /dev/disk/by-label/boot /mnt/boot
 </syntaxhighlight>
+For better reliability as with any other filesystem, one should mount the filesystem by UUID, by replacing any drive specifiers with it.
+When you formatted, there was an external UUID shown, but if you lost it, you can retrieve it using
+<syntaxhighlight lang="console">
+# lsblk -o +uuid,fsType | grep bcachefs
+</syntaxhighlight>
+And using it like <code>UUID=<UUID></code> in place of <code>/dev/sda1:/dev/sdb1</code> or even just <code>/dev/sda</code>.
+Note: this is currently broken as per this [https://github.com/NixOS/nixpkgs/issues/317901 github issue]. As a workaround, use `/dev/disk/by-uuid/<UUID>` or `/dev/disk/by-id` instead. This seems to work for single device systems atleast, although atleast at the time of this writing I could not get it to work on my 4+ device raid system.
 Continue installation as recommended by the [https://nixos.org/manual/nixos/stable/index.html#ch-installation NixOS manual].
+Before <code>nixos-install</code> you'll have to adapt the generated NixOS configuration to also use the latest Linux kernel supporting Bcachefs. Add the following line to the <code>configuration.nix</code> in your installation root (/mnt):
+{{file|/mnt/etc/nixos/configuration.nix|nix|<nowiki>
+boot.supportedFilesystems = [ "bcachefs" ];
+boot.kernelPackages = pkgs.linuxPackages_latest;
+</nowiki>}}
+=== Remote encrypted disk unlocking ===
+See article on [[Remote disk unlocking#Bcachefs unlocking|remote disk unlocking]] for a guide on how to enable SSH decryption of Bcachefs enabled systems.
+=== Automatically mount encrypted device on boot ===
+Since the Bcachefs mount options do [https://github.com/koverstreet/bcachefs-tools/pull/266 not support supplying a key file yet], we could use the <code>bcachefs</code> command and run it on boot using a [[Systemd]] unit:<syntaxhighlight lang="nix">
+systemd.services."bcachefs-mount" = {
+  after = [ "local-fs.target" ];
+  wantedBy = [ "multi-user.target" ];
+  environment = {
+    DEVICE_PATH = "/dev/sda1";
+    MOUNT_POINT = "/mnt";
+  };
+  script = ''
+    #!${pkgs.runtimeShell} -e
+    ${pkgs.keyutils}/bin/keyctl link @u @s
+    # Check if the device path exists
+    if [ ! -b "$DEVICE_PATH" ]; then
+      echo "Error: Device path $DEVICE_PATH does not exist."
+      exit 1
+    fi
+    # Check if the drive is already mounted
+    if ${pkgs.util-linux}/bin/mountpoint -q "$MOUNT_POINT"; then
+      echo "Drive already mounted at $MOUNT_POINT. Skipping..."
+      exit 0
+    fi
+    # Wait for the device to become available
+    while [ ! -b "$DEVICE_PATH" ]; do
+      echo "Waiting for $DEVICE_PATH to become available..."
+      sleep 5
+    done
+    # Mount the device
+    ${pkgs.bcachefs-tools}/bin/bcachefs mount -f /etc/keyfile_test "$DEVICE_PATH" "$MOUNT_POINT"
+  '';
+  serviceConfig = {
+    Type = "oneshot";
+    User = "root";
+  };
+};
+</syntaxhighlight>This example unit mounts the Bcachefs encrypted partition <code>/dev/sda1</code> to the target <code>/mnt</code> by using the key file <code>/etc/keyfile_test</code>.
 [[Category:Filesystem]]