|
|
| Line 1: |
Line 1: |
| [[NixOS]] has native support for ZFS ([[wikipedia:ZFS]]). It uses the code from the [http://zfsonlinux.org/ ZFS on Linux project], including kernel modules and userspace utilities. The installation isos also come with zfs.
| | == Notes == |
| | * Newest kernels might not be supported by ZFS yet. If you are running an newer kernel which is not yet officially supported by zfs, the zfs module will refuse to evaluate and show up as ''broken''. Use <code>boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;</code> |
|
| |
|
| == What works == | | * ZFS does not support swap. Hibernation must be either disabled with <code><nowiki>boot.kernelParams = [ "nohibernate" ];</nowiki></code>, or enabled with a separate, non-ZFS swap partition. |
|
| |
|
| All functionality supported by ZFS on Linux, including:
| | * By default, all ZFS pools available to the system will be forcibly imported during boot. This behaviour can be disabled by setting <syntaxhighlight lang="nix" inline>boot.zfs.forceImportAll = false;</syntaxhighlight>. |
| * Using ZFS as the root filesystem (using either MS-DOS or GPT partitions)
| |
| * Encrypted ZFS pools (using either native encryption or Linux's dm-crypt)
| |
| * All the other ZFS goodies (cheap snapshotting, checksumming, compression, RAID-Z, …)
| |
| * Auto-snapshotting service
| |
|
| |
|
| == Known issues ==
| |
| {{note|Setting <code><nowiki>boot.zfs.enableUnstable = true;</nowiki></code> is required if you are running an newer kernel which is not yet officially supported by zfs, otherwise the zfs module will refuse to evaluate and show up as ''broken''. This will install a pre-release of zfs. This might be not as stable as a released version. However, in the past this has rarely led to problems/stability issues}}
| |
|
| |
| * Using NixOS on a ZFS root file system might result in the boot error ''external pointer tables not supported'' when the number of hardlinks in the nix store gets very high. This can be avoided by adding this option to your <code>configuration.nix</code> file:
| |
| <syntaxhighlight lang="nix">
| |
| boot.loader.grub.copyKernels = true;
| |
| </syntaxhighlight>
| |
|
| |
| * In contrast to many native Linux filesystems, ZFS misses support for freeze/thaw operations. This means that using ZFS together with hibernation (suspend to disk) may cause filesystem corruption. See https://github.com/openzfs/zfs/issues/260 (closed in favour of follow-up issues https://github.com/openzfs/zfs/issues/12842 and https://github.com/openzfs/zfs/issues/12843).
| |
|
| |
| {{note|For now, setting <code><nowiki>boot.kernelParams = [ "nohibernate" ];</nowiki></code>is necessary to avoid the issue described above.}}
| |
|
| |
|
| |
| == Caveats ==
| |
|
| |
| * (ZFS, unrelated to Nix- see https://github.com/openzfs/zfs/issues/7734) You shouldn't use a ZVol as a swap device, as it can deadlock under memory pressure.
| |
| * You should set the <code>mountpoint</code> property of your ZFS filesystems to be <code>legacy</code> and let NixOS mount them like any other filesystem (such as ''ext4'' or ''btrfs''), otherwise some filesystems may fail to mount due to ordering issues.
| |
| * By default, all ZFS pools available to the system will be forcibly imported during boot, regardless if you had imported them before or not. You should be careful not to have any other system accessing them at the same time, otherwise it will corrupt your pools. Normally (for the common desktop user) this should not be a problem, as a hard disk is usually only directly connected to one machine. This behaviour can be disabled by setting <syntaxhighlight lang="nix" inline>boot.zfs.forceImportAll = false;</syntaxhighlight>.
| |
| * If you create a zpool in the installer, make sure you run <code>zpool export <pool name></code> after <code>nixos-install</code>, or else when you reboot into your new system, zfs will fail to import the zpool.
| |
| * If you are running within a VM and NixOS fails to import the zpool on reboot, you may need to add <syntaxhighlight lang="nix" inline>boot.zfs.devNodes = "/dev/disk/by-path";</syntaxhighlight> to your configuration.nix file. | | * If you are running within a VM and NixOS fails to import the zpool on reboot, you may need to add <syntaxhighlight lang="nix" inline>boot.zfs.devNodes = "/dev/disk/by-path";</syntaxhighlight> to your configuration.nix file. |
|
| |
|
|
| |
|
| == How to use it == | | == Enable ZFS support == |
|
| |
|
| {{warning|Add all mounts necessary for booting to your configuration as legacy mounts as described in this article instead of zfs's own mount mechanism. Otherwise mounts might be not mounted in the correct order during boot!}}
| | Common ZFS installation guides are now maintained at [https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/index.html OpenZFS Documentation] website. Visit there for details and if an issue arises, submit an issue or pull request. |
|
| |
|
| Just add the following to your <code>configuration.nix</code> file:
| | == Root on ZFS == |
| <syntaxhighlight lang="nix">
| |
| # boot.initrd.supportedFilesystems = [ "zfs" ]; # Not required if zfs is root-fs (extracted from filesystems)
| |
| # boot.supportedFilesystems = [ "zfs" ]; # Not required if zfs is root-fs (extracted from filesystems)
| |
| services.udev.extraRules = ''
| |
| ACTION=="add|change", KERNEL=="sd[a-z]*[0-9]*|mmcblk[0-9]*p[0-9]*|nvme[0-9]*n[0-9]*p[0-9]*", ENV{ID_FS_TYPE}=="zfs_member", ATTR{../queue/scheduler}="none"
| |
| ''; # zfs already has its own scheduler. without this my(@Artturin) computer froze for a second when i nix build something.
| |
| </syntaxhighlight>
| |
| | |
| Be sure to also set <code>networking.hostId</code>, see https://nixos.org/nixos/manual/options.html#opt-networking.hostId (Why- https://discourse.nixos.org/t/feedback-on-a-user-guide-i-created-on-installing-nixos-with-zfs/5986/4?u=srgom)
| |
| | |
| To activate the configuration and load the ZFS kernel module, run:
| |
| <syntaxhighlight lang="console">
| |
| # nixos-rebuild switch
| |
| </syntaxhighlight>
| |
| | |
| All ZFS functionality should now be available.
| |
|
| |
|
| If you want NixOS to auto-mount your ZFS filesystems during boot, you should set their <code>mountpoint</code> property to <code>legacy</code> and treat it like if it were any other filesystem, i.e.: mount the filesystem manually and regenerate your list of filesystems, as such:
| | Root on ZFS guide is now maintained at [https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/Root%20on%20ZFS.html OpenZFS Documentation] website. Visit there for details and if an issue arises, submit an issue or pull request. |
|
| |
|
| <syntaxhighlight lang="console">
| | == Mount datasets at boot == |
| # zfs set mountpoint=legacy <pool>/<fs>
| | zfs-mount service is enabled by default on NixOS 22.05. |
| </syntaxhighlight>
| |
|
| |
|
| <syntaxhighlight lang="console"> | | To automatically mount a dataset at boot, you only need to set <code>canmount=on</code> and <code>mountpoint=/mount/point</code> on the respective datasets. |
| # mount -t zfs <pool>/<fs> <mountpoint>
| |
| </syntaxhighlight> | |
|
| |
|
| This will regenerate your /etc/nixos/hardware-configuration.nix file:
| | == Changing the Adaptive Replacement Cache size == |
| <syntaxhighlight lang="console">
| |
| # nixos-generate-config
| |
| </syntaxhighlight>
| |
|
| |
|
| <syntaxhighlight lang="console">
| | To change the maximum size of the ARC to (for example) 12 GB, add this to your NixOS configuration: |
| # nixos-rebuild switch
| |
| </syntaxhighlight>
| |
| | |
| NixOS will now make sure that your filesystem is always mounted during boot.
| |
| | |
| The <code>nixos-generate-config</code> command regenerates your <code>/etc/nixos/hardware-configuration.nix</code> file, which includes the list of filesystems for NixOS to mount during boot, e.g.:
| |
| <syntaxhighlight lang="nix">
| |
| fileSystems."/home" =
| |
| { device = "rpool/home";
| |
| fsType = "zfs";
| |
| };
| |
| | |
| fileSystems."/backup" =
| |
| { device = "rpool/backup";
| |
| fsType = "zfs";
| |
| };
| |
| </syntaxhighlight>
| |
| | |
| Alternatively, if you do not mind maintaining some manual tweaks to your Nix hardware configuration, you can avoid using the ZFS legacy mounting option if you add <syntaxhighlight lang="nix" inline>options = [ "zfsutil" ];</syntaxhighlight> to your filesystem definitions. e.g. the above would become.
| |
| <syntaxhighlight lang="nix">
| |
| fileSystems."/home" =
| |
| { device = "rpool/home";
| |
| fsType = "zfs";
| |
| options = [ "zfsutil" ];
| |
| };
| |
| | |
| fileSystems."/backup" =
| |
| { device = "rpool/backup";
| |
| fsType = "zfs";
| |
| options = [ "zfsutil" ];
| |
| };
| |
| </syntaxhighlight>
| |
| | |
| Keep your filesystem defintions in a file separate from <code>/etc/nixos/hardware-configuration.nix</code>, since this is overwritten whenever you run <code>nixos-generate-config</code>.
| |
| | |
| == Changing the cache size ==
| |
| | |
| ZFS has a complicated cache system. The cache you're most likely to want to fiddle with is the called Adaptive Replacement Cache, usually abbreviated ARC. This is the first-level (fastest) of ZFS's caches.
| |
| | |
| You can increase or decrease a parameter which represents approximately the maximum size of the ARC cache. You can't set its actual size (ZFS does that adaptively according to its workload), nor can you set its exact maximum size.
| |
| | |
| To change the maximum size of the ARC cache to (for example) 12 GB, add this to your NixOS configuration: | |
| <syntaxhighlight lang="nix"> | | <syntaxhighlight lang="nix"> |
| boot.kernelParams = [ "zfs.zfs_arc_max=12884901888" ]; | | boot.kernelParams = [ "zfs.zfs_arc_max=12884901888" ]; |
| </syntaxhighlight> | | </syntaxhighlight> |
|
| |
| In some versions of ZFS, you can change the maximum size of the ARC on the fly, but in NixOS 18.03 this is not possible. (Nor is it possible in other versions of ZFS on Linux yet, according to Stack Exchange.)
| |
|
| |
|
| == Automatic scrubbing == | | == Automatic scrubbing == |
