Btrbk: Difference between revisions

From NixOS Wiki
imported>Onny
mNo edit summary
Onny (talk | contribs)
No edit summary
 
(15 intermediate revisions by 4 users not shown)
Line 3: Line 3:
== Setup ==
== Setup ==


When transfering backups of root filesystem snapshots using Btrbk, it is recommended to mount the root Btrfs drive with subvolume id 5 (in this example <code>/dev/sda1</code>) to a specific mountpoint where Btrbk can operate with. So in this case all subvolumes will be available as a subdirectory in <code>/btr_pool</code>.
When transferring backups of root filesystem snapshots using Btrbk, it is recommended to mount the root Btrfs drive with subvolume id 5 (in this example <code>/dev/sda1</code>) to a specific mountpoint where Btrbk can operate with. So in this case all subvolumes will be available as a subdirectory in <code>/btr_pool</code>.


{{file|/etc/nixos/hardware-configuration.nix|nix|<nowiki>
{{file|/etc/nixos/hardware-configuration.nix|nix|<nowiki>
Line 17: Line 17:
== Configuration ==
== Configuration ==


Following example configuration will create a weekly incremental backup of a local Btrfs subvolume called <code>nixos</code> and sends it compressed to a remote host <code>myhost</code> via ssh using provided authentication credentials. Note that this references the mount point <code>/btr_pool</code> from above.
=== Basic example ===
Following example configuration will create a weekly incremental backup of a local Btrfs subvolume called <code>nixos</code> and sends it compressed to the remote host <code>myhost</code>. The mount point <code>/btr_pool</code>, as referenced above, contains the subvolume.
 
The user <code>btrbk</code> together with the private key <code>/etc/btrbk_key</code> is used for authentication.


{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
services.btrbk = {
services.btrbk = {
  extraPackages = [ pkgs.lz4 ];
   instances."remote_myhost" = {
   instances.remote = {
     onCalendar = "weekly";
     onCalendar = "weekly";
     settings = {
     settings = {
       ssh_identity = "/etc/btrbk_key";
       ssh_identity = "/etc/btrbk_key"; # NOTE: must be readable by user/group btrbk
       ssh_user = "btrbk";
       ssh_user = "btrbk";
       stream_compress = "lz4";
       stream_compress = "lz4";
Line 31: Line 33:
         target = "ssh://myhost/mnt/mybackups";
         target = "ssh://myhost/mnt/mybackups";
         subvolume = "nixos";
         subvolume = "nixos";
        # "nixos" could instead be an attribute set with other volumes to
        # back up and to give subvolume specific configuration.
        # See man btrbk.conf for possible options.
        /*
        subvolume = {
          home = { snapshot_create = "always"; };
          nixos = {};
        };
        */
       };
       };
     };
     };
Line 37: Line 48:
</nowiki>}}
</nowiki>}}


For transport stream compression using <code>lz4</code> to work, the package must also be installed on the target host.
The user has to be created on the remote host and needs root permissions on the commands <code>btrfs</code>, <code>readlink</code> and <code>test</code>, for example via [[sudo]]. For transport stream compression using <code>lz4</code> to work, the package must also be installed on the target host. The target host configuration for Btrbk could look like this:
 
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
security.sudo = {
  enable = true;
  extraRules = [{
    commands = [
      {
        command = "${pkgs.coreutils-full}/bin/test";
        options = [ "NOPASSWD" ];
      }
      {
        command = "${pkgs.coreutils-full}/bin/readlink";
        options = [ "NOPASSWD" ];
      }
      {
        command = "${pkgs.btrfs-progs}/bin/btrfs";
        options = [ "NOPASSWD" ];
      }
    ];
    users = [ "btrbk" ];
  }];
  extraConfig = with pkgs; ''
    Defaults:picloud secure_path="${lib.makeBinPath [
      btrfs-progs coreutils-full
    ]}:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin"
  '';
};
 
environment.systemPackages = [ pkgs.lz4 ];
</nowiki>}}
 
=== Local <code>/home</code> Snapshots ===
{{Warning|This is not a backup solution alone. If the entire disk fails, local snapshots will be lost along with it.}}
 
If <code>/home</code> is its own subvolume and important files are backed up separately or combined with the above section, this configuration takes snapshots hourly, retains them for at least a week, and keeps weekly snapshots for two weeks under <code>/snapshots</code>.
 
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
services.btrbk = {
  instances."home" = {
    onCalendar = "hourly";
    settings = {
      snapshot_preserve_min = "1w";
      snapshot_preserve = "2w";
      volume = {
          "/" = {
            snapshot_dir = "/snapshots";
            subvolume = "home";
          };
        };
      };
  };
};
# Btrbk does not create snapshot directories automatically, so create one here.
systemd.tmpfiles.rules = [
  "d /snapshots 0755 root root"
];
</nowiki>}}
 
=== Retention policy ===
The following example takes daily snapshot but won't store them forever with the given retention policy:
 
* '''7d''': For the most recent week, you will have a '''daily snapshot''' stored from each day.
* '''4w''': After a week, you'll only keep one snapshot per week for the next 4 weeks (so older daily snapshots get removed).
* '''12m''': After a month, the policy will keep only '''monthly snapshots''' for the next 12 months.
 
The option <code>snapshot_preserve_min</code>ensures that all daily snapshots from the last 7 days are preserved, regardless of the other retention rules. It's a safety net to guarantee that no daily snapshot from the past week is deleted prematurely.<syntaxhighlight lang="nix">
services.btrbk.instances."remote_myhost" = {
  onCalendar = "daily";
  settings = {
      snapshot_preserve = "7d 4w 12m";
      snapshot_preserve_min = "7d";
      target_preserve = "7d 4w 12m";
  };
};
</syntaxhighlight>This retention policy will ensure you have a balance between recent, frequent backups (daily) and older, more spaced-out backups (weekly/monthly) while preserving space.


== Manual usage ==
== Manual usage ==
Line 44: Line 130:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
btrbk -c /etc/btrbk/remote.conf --dry-run --progress --verbose run
btrbk -c /etc/btrbk/remote_myhost.conf --dry-run --progress --verbose run
</syntaxhighlight>
</syntaxhighlight>


The filename <code>remote.conf</code> references the instance name choosen in the example configuration above.
The filename <code>remote_myhost.conf</code> references the instance name choosen in the example configuration above.


[[Category:Applications]]
[[Category:Applications]]
[[Category:Backup]]
[[Category:Backup]]

Latest revision as of 15:52, 20 October 2024

Btrbk, a tool for creating snapshots and remote backups of btrfs subvolumes.

Setup

When transferring backups of root filesystem snapshots using Btrbk, it is recommended to mount the root Btrfs drive with subvolume id 5 (in this example /dev/sda1) to a specific mountpoint where Btrbk can operate with. So in this case all subvolumes will be available as a subdirectory in /btr_pool.

/etc/nixos/hardware-configuration.nix
fileSystems = {
  "/btr_pool" = {
    device = "/dev/sda1";
    fsType = "btrfs";
    options = [ "subvolid=5" ];
  };
};

Configuration

Basic example

Following example configuration will create a weekly incremental backup of a local Btrfs subvolume called nixos and sends it compressed to the remote host myhost. The mount point /btr_pool, as referenced above, contains the subvolume.

The user btrbk together with the private key /etc/btrbk_key is used for authentication.

/etc/nixos/configuration.nix
services.btrbk = {
  instances."remote_myhost" = {
    onCalendar = "weekly";
    settings = {
      ssh_identity = "/etc/btrbk_key"; # NOTE: must be readable by user/group btrbk
      ssh_user = "btrbk";
      stream_compress = "lz4";
      volume."/btr_pool" = {
        target = "ssh://myhost/mnt/mybackups";
        subvolume = "nixos";
        # "nixos" could instead be an attribute set with other volumes to
        # back up and to give subvolume specific configuration.
        # See man btrbk.conf for possible options.
        /*
        subvolume = {
          home = { snapshot_create = "always"; };
          nixos = {};
        };
        */
      };
    };
  };
};

The user has to be created on the remote host and needs root permissions on the commands btrfs, readlink and test, for example via sudo. For transport stream compression using lz4 to work, the package must also be installed on the target host. The target host configuration for Btrbk could look like this:

/etc/nixos/configuration.nix
security.sudo = {
  enable = true;
  extraRules = [{
    commands = [
      {
        command = "${pkgs.coreutils-full}/bin/test";
        options = [ "NOPASSWD" ];
      }
      {
        command = "${pkgs.coreutils-full}/bin/readlink";
        options = [ "NOPASSWD" ];
      }
      {
        command = "${pkgs.btrfs-progs}/bin/btrfs";
        options = [ "NOPASSWD" ];
      }
    ];
    users = [ "btrbk" ];
  }];
  extraConfig = with pkgs; ''
    Defaults:picloud secure_path="${lib.makeBinPath [
      btrfs-progs coreutils-full
    ]}:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin"
  '';
};

environment.systemPackages = [ pkgs.lz4 ];

Local /home Snapshots

Warning: This is not a backup solution alone. If the entire disk fails, local snapshots will be lost along with it.

If /home is its own subvolume and important files are backed up separately or combined with the above section, this configuration takes snapshots hourly, retains them for at least a week, and keeps weekly snapshots for two weeks under /snapshots.

/etc/nixos/configuration.nix
services.btrbk = {
  instances."home" = {
    onCalendar = "hourly";
    settings = {
      snapshot_preserve_min = "1w";
      snapshot_preserve = "2w";
      volume = {
          "/" = {
            snapshot_dir = "/snapshots";
            subvolume = "home";
          };
        };
      };
  };
};
# Btrbk does not create snapshot directories automatically, so create one here.
systemd.tmpfiles.rules = [
  "d /snapshots 0755 root root"
];

Retention policy

The following example takes daily snapshot but won't store them forever with the given retention policy:

  • 7d: For the most recent week, you will have a daily snapshot stored from each day.
  • 4w: After a week, you'll only keep one snapshot per week for the next 4 weeks (so older daily snapshots get removed).
  • 12m: After a month, the policy will keep only monthly snapshots for the next 12 months.

The option snapshot_preserve_minensures that all daily snapshots from the last 7 days are preserved, regardless of the other retention rules. It's a safety net to guarantee that no daily snapshot from the past week is deleted prematurely.

services.btrbk.instances."remote_myhost" = {
  onCalendar = "daily";
  settings = {
      snapshot_preserve = "7d 4w 12m";
      snapshot_preserve_min = "7d";
      target_preserve = "7d 4w 12m";
  };
};

This retention policy will ensure you have a balance between recent, frequent backups (daily) and older, more spaced-out backups (weekly/monthly) while preserving space.

Manual usage

Manually dry running and testing a btrbk configuration

btrbk -c /etc/btrbk/remote_myhost.conf --dry-run --progress --verbose run

The filename remote_myhost.conf references the instance name choosen in the example configuration above.