NixOS Wiki

ZFS: Difference between revisions

← Older editNewer edit →
VisualWikitext
imported>Mic92
imported>Vater
No edit summary
Line 1: Line 1:
NixOS has native support for ZFS. It uses the code from the [http://zfsonlinux.org/ ZFS on Linux project], including kernel modules and userspace utilities. The installer also comes with zfs.
[[NixOS]] has native support for ZFS ([[wikipedia:ZFS]]). It uses the code from the [http://zfsonlinux.org/ ZFS on Linux project], including kernel modules and userspace utilities. The installation CD also comes with zfs.
 


== What works ==
== What works ==
Line 7: Line 6:
* Using ZFS as the root filesystem (using either MS-DOS or GPT partitions)
* Using ZFS as the root filesystem (using either MS-DOS or GPT partitions)
* Encrypted ZFS pools (using either native encryption or Linux's dm-crypt)
* Encrypted ZFS pools (using either native encryption or Linux's dm-crypt)
* All the other ZFS goodies (cheap snapshotting, checksumming, compression, RAID-Z, ...)
* All the other ZFS goodies (cheap snapshotting, checksumming, compression, RAID-Z, …)
* Auto-snapshotting service
* Auto-snapshotting service


Line 19: Line 18:


Just add the following to your <code>configuration.nix</code> file:
Just add the following to your <code>configuration.nix</code> file:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
boot.supportedFilesystems = [ "zfs" ];
boot.supportedFilesystems = [ "zfs" ];
</syntaxhighlight>
</syntaxhighlight>


Be sure to also set networking.hostId, see https://nixos.org/nixos/manual/options.html#opt-networking.hostId
Be sure to also set <code>networking.hostId</code>, see https://nixos.org/nixos/manual/options.html#opt-networking.hostId


To activate the configuration and load the ZFS kernel module, run:
To activate the configuration and load the ZFS kernel module, run:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
nixos-rebuild switch
nixos-rebuild switch
Line 38: Line 35:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
zfs set mountpoint=legacy <pool>/<fs>
zfs set mountpoint=legacy <pool>/<fs>
</syntaxhighlight>
<syntaxhighlight lang="bash">
mount -t zfs <pool>/<fs> <mountpoint>
mount -t zfs <pool>/<fs> <mountpoint>
</syntaxhighlight>


# This will regenerate your /etc/nixos/hardware-configuration.nix file:
This will regenerate your /etc/nixos/hardware-configuration.nix file:
<syntaxhighlight lang="bash">
nixos-generate-config
nixos-generate-config
</syntaxhighlight>


<syntaxhighlight lang="bash">
nixos-rebuild switch
nixos-rebuild switch
</syntaxhighlight>
</syntaxhighlight>


NixOS will now make sure that your filesystem is always mounted during boot.
NixOS will now make sure that your filesystem is always mounted during boot.
The <code>nixos-generate-config</code> command regenerates your <code>/etc/nixos/hardware-configuration.nix</code> file, which includes the list of filesystems for NixOS to mount during boot, e.g.:
The <code>nixos-generate-config</code> command regenerates your <code>/etc/nixos/hardware-configuration.nix</code> file, which includes the list of filesystems for NixOS to mount during boot, e.g.:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
Line 67: Line 72:


To change the maximum size of the ARC cache to (for example) 12 GB, add this to your NixOS configuration:
To change the maximum size of the ARC cache to (for example) 12 GB, add this to your NixOS configuration:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
boot.kernelParams = ["zfs.zfs_arc_max=12884901888"];
boot.kernelParams = ["zfs.zfs_arc_max=12884901888"];
Line 85: Line 89:
== Reservations ==
== Reservations ==


Since zfs is a copy-on-write filesystem even for deleting files disk space is needed. Therefore it should be avoided
Since zfs is a copy-on-write filesystem even for deleting files disk space is needed. Therefore it should be avoided to run out of disk space. Luckily it is possible to reserve disk space for datasets to prevent this.
to run out of disk space. Luckily it is possible to reserve disk space for datasets to prevent this.
 
To enable reservations pick any dataset of your and do:
To enable reservations pick any dataset of your and do:
 
: reserves enough disk space to have room for cleanups/deletion
<syntaxhighlight lang="console">
<syntaxhighlight lang="bash">
$ zfs set reservation=1G zroot # reserves enough disk space to have room for cleanups/deletion
zfs set reservation=1G zroot
</syntaxhighlight>
</syntaxhighlight>


Line 100: Line 104:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
$ zfs set com.sun:auto-snapshot=true <pool>/<fs>
zfs set com.sun:auto-snapshot=true <pool>/<fs>
</syntaxhighlight>
</syntaxhighlight>


Line 106: Line 110:


Then, to enable the auto-snapshot service, add this to your <code>configuration.nix</code>:
Then, to enable the auto-snapshot service, add this to your <code>configuration.nix</code>:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
services.zfs.autoSnapshot.enable = true;
services.zfs.autoSnapshot.enable = true;
Line 126: Line 129:
You can also disable a given type of snapshots on a per-dataset basis by setting a ZFS property, like this:
You can also disable a given type of snapshots on a per-dataset basis by setting a ZFS property, like this:


<syntaxhighlight lang="console">
<syntaxhighlight lang="bash">
$ zfs set com.sun:auto-snapshot:weekly=false <pool>/<fs>
zfs set com.sun:auto-snapshot:weekly=false <pool>/<fs>
</syntaxhighlight>
</syntaxhighlight>


Line 307: Line 310:
Assuming that a zpool named <code>zroot</code> has been already created as described.
Assuming that a zpool named <code>zroot</code> has been already created as described.
Encrypted datasets can be added on top as follow:
Encrypted datasets can be added on top as follow:
 
: posixacl are needed for journald
<syntaxHighlight lang=console>
<syntaxhighlight lang="bash">
# posixacl are needed for journald
zfs create -o  acltype=posixacl -o xattr=sa -o encryption=aes-256-gcm -o keyformat=passphrase -o mountpoint=none zroot/root
$ zfs create -o  acltype=posixacl -o xattr=sa -o encryption=aes-256-gcm -o keyformat=passphrase -o mountpoint=none zroot/root
</syntaxHighlight>
</syntaxHighlight>


Instead of encrypting just a dataset (and all its child datasets) you can also directly encrypt the whole pool upon creation:
Instead of encrypting just a dataset (and all its child datasets) you can also directly encrypt the whole pool upon creation:
<syntaxHighlight lang=console>
<syntaxhighlight lang="bash">
$ zpool create -o ashift=12 -o altroot="/mnt" -O encryption=aes-256-gcm -O keyformat=passphrase zroot /dev/sdxy
zpool create -o ashift=12 -o altroot="/mnt" -O encryption=aes-256-gcm -O keyformat=passphrase zroot /dev/sdxy
</syntaxHighlight>
</syntaxHighlight>


All child datasets will inherit the encryption.
All child datasets will inherit the encryption.
Note that using grub to boot directly from zfs with encryption enabled might not work at the moment,  
 
so a separate boot partition is required.
Note that using grub to boot directly from zfs with encryption enabled might not work at the moment, so a separate boot partition is required.
 
A full encrypted nixos installation on an UEFI system could look like this:
A full encrypted nixos installation on an UEFI system could look like this:
<syntaxhighlight lang="bash">
zfs create -o mountpoint=legacy -o sync=disabled zroot/root/tmp
zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=true zroot/root/home
zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=true zroot/root/nixos
</syntaxHighlight>


<syntaxHighlight lang=console>
<syntaxhighlight lang="bash">
$ zfs create -o mountpoint=legacy -o sync=disabled zroot/root/tmp
mount -t zfs zroot/root/nixos /mnt
$ zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=true zroot/root/home
mkdir /mnt/{home,tmp,boot}
$ zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=true zroot/root/nixos
</syntaxHighlight>
$ mount -t zfs zroot/root/nixos /mnt
: assuming that /dev/sda1 is the boot partition
$ mkdir /mnt/{home,tmp,boot}
<syntaxhighlight lang="bash">
$ # assuming that /dev/sda1 is the boot partition
mkfs.vfat /dev/sda1
$ mkfs.vfat /dev/sda1
mount /dev/sda1 /mnt/boot/
$ mount /dev/sda1 /mnt/boot/
</syntaxHighlight>
$ mount -t zfs zroot/root/home /mnt/home/
 
$ mount -t zfs zroot/root/tmp /mnt/tmp/
<syntaxhighlight lang="bash">
$ nixos-generate-config  --root /mnt
mount -t zfs zroot/root/home /mnt/home/
mount -t zfs zroot/root/tmp /mnt/tmp/
</syntaxHighlight>
 
<syntaxhighlight lang="bash">
nixos-generate-config  --root /mnt
</syntaxHighlight>
</syntaxHighlight>


To unlock the zfs dataset at root also the <code>boot.zfs.requestEncryptionCredentials</code> option must be set to <code>true</code>. Note that at the moment one can only use passphrases (<code>keylocation=prompt</code>) for pools that are mounted as the root fs. Data pools are mounted by a background systemd service and need a key (<code>keylocation=file://</code>). A key file could be for example put on a root filesystem if it is encrypted.
To unlock the zfs dataset at root also the <code>boot.zfs.requestEncryptionCredentials</code> option must be set to <code>true</code>. Note that at the moment one can only use passphrases (<code>keylocation=prompt</code>) for pools that are mounted as the root fs. Data pools are mounted by a background systemd service and need a key (<code>keylocation=file://</code>). A key file could be for example put on a root filesystem if it is encrypted.
If the key is not on the root filesystem, you will also need to set <code>zfs-import-poolname.serviceConfig.RequiresMountsFor=/path/to/key</code>, where <code>poolname</code> is the name of the data pool. This makes sure that systemd will mount the filesystem for <code>/path/to/key</code> first before importing the zfs pool.
If the key is not on the root filesystem, you will also need to set <code>zfs-import-poolname.serviceConfig.RequiresMountsFor=/path/to/key</code>, where <code>poolname</code> is the name of the data pool. This makes sure that systemd will mount the filesystem for <code>/path/to/key</code> first before importing the zfs pool.


=== Unlock encrypted zfs via ssh on boot ===
=== Unlock encrypted zfs via ssh on boot ===


In case you want unlock a machine remotely (after an update),
In case you want unlock a machine remotely (after an update), having a dropbear ssh service in initrd for the password prompt is handy:
having a dropbear ssh service in initrd for the password prompt
is handy:


<syntaxHighlight lang=nix>
<syntaxhighlight lang="nix">
  boot = {
  boot = {
   initrd.network = {
   initrd.network = {
Line 385: Line 397:
The following example follows the remote unlocking with dropbear, but imports another pool also and prompts for unlocking (either when at the machine itself or when logging in remotely:
The following example follows the remote unlocking with dropbear, but imports another pool also and prompts for unlocking (either when at the machine itself or when logging in remotely:


<syntaxHighlight lang=nix>
<syntaxhighlight lang="nix">
  boot = {
  boot = {
   initrd.network = {
   initrd.network = {
Line 405: Line 417:
When you login by SSH into dropbear or when you have physical access to the machine itself, you will be prompted to supply the unlocking password for your zroot and tankXXX pools.
When you login by SSH into dropbear or when you have physical access to the machine itself, you will be prompted to supply the unlocking password for your zroot and tankXXX pools.


==Regarding installation of NixOS to ZFS direct from installation media==
== Regarding installation of NixOS to ZFS direct from installation media ==


* Since [https://github.com/NixOS/nixpkgs/pull/51090 18.09] the installation iso comes with zfs by default again.
* Since [https://github.com/NixOS/nixpkgs/pull/51090 18.09] the installation iso comes with zfs by default again.
* For older versions it is still possible to enable it in the existing ISO at runtime adding:
* For older versions it is still possible to enable it in the existing ISO at runtime adding:


<syntaxHighlight lang=nix>
<syntaxhighlight lang="nix">
{   
{   
   boot.supportedFilesystems = [ "zfs" ];
   boot.supportedFilesystems = [ "zfs" ];
Line 416: Line 428:
</syntaxHighlight>
</syntaxHighlight>


to the iso's configuration.nix followed by a <code>nixos-rebuild switch</code>
to the iso's configuration.nix followed by a <code>nixos-rebuild switch</code> ([https://discourse.nixos.org/t/install-report-from-new-user/1390/9 source])
([https://discourse.nixos.org/t/install-report-from-new-user/1390/9 source])
 


== Need more info? ==
== Need more info? ==
Retrieved from "https://wiki.nixos.org/wiki/ZFS"