Official NixOS Wiki

ZFS: Difference between revisions

← Older edit
Kacper (talk | contribs)
4 edits
VisualWikitext
Typo
Kacper (talk | contribs)
4 edits
→Remote unlock: Remove a note to a closed issue, assuming this has been resolved already.
 
(16 intermediate revisions by 7 users not shown)
Line 49: Line 49:
==== Partial support for swap on ZFS ====
==== Partial support for swap on ZFS ====


ZFS does not support swapfiles. swap devices can be used instead. Additionally, hibernation is disabled by default due to a [https://github.com/NixOS/nixpkgs/pull/208037 high risk] of data corruption. Note that even if that pull request is merged, it does not fully mitigate the risk. If you wish to enable hibernation regardless and make sure that swapfiles on ZFS are not used, set <code>boot.zfs.allowHibernation = true</code>.
ZFS does not support swapfiles. swap devices can be used instead. Additionally, hibernation is disabled by default due to a [https://github.com/NixOS/nixpkgs/pull/208037 high risk] of data corruption. Note that even if that pull request is merged, it does not fully mitigate the risk. If you wish to enable hibernation regardless and made sure that swapfiles on ZFS are not used, set <code>boot.zfs.allowHibernation = true</code>.


==== Zpool not found ====
==== Zpool not found ====
Line 62: Line 62:


Disable the mount service with <code>systemd.services.zfs-mount.enable = false;</code> or remove the <code>fileSystems</code> entries in hardware-configuration.nix. Otherwise, use legacy mountpoints (created with e.g. <code>zfs create -o mountpoint=legacy</code>). Mountpoints must be specified with <code>fileSystems."/mount/point" = {};</code> or with <code>nixos-generate-config</code>.
Disable the mount service with <code>systemd.services.zfs-mount.enable = false;</code> or remove the <code>fileSystems</code> entries in hardware-configuration.nix. Otherwise, use legacy mountpoints (created with e.g. <code>zfs create -o mountpoint=legacy</code>). Mountpoints must be specified with <code>fileSystems."/mount/point" = {};</code> or with <code>nixos-generate-config</code>.
==== Nix builds and ZFS properties like normalization or utf8only ====
These options are often suggested in guides to setting up ZFS. <code>normalization</code> makes filenames compare the same in cases where there exists more than one UTF8 bytestring that represents the same characters. <code>utf8only</code> prevents the creation of files with non-UTF8 filenames, e.g. filenames using a Latin1 character set. These are non-POSIX and will make the tests for certain packages fail, which may interfere with builds. After nix 2.30, builds no longer happen in /tmp by default, instead they happen in <code>/nix/var/nix/builds</code>. On any system where you plan to run nix builds, you should ensure that this filesystem is POSIX-compliant. Either mounting a tmpfs in that directory (if you have lots of RAM + swap) or creating a zfs dataset there which does not have these or other non-POSIX settings like <code>noatime</code>, <code>snapdir=visible</code>, <code>acltype=nfsv4</code>, or <code>caseinsensitivity=insensitive</code>. Many of these cannot be changed after dataset creation so if this is your root filesystem, you will need to restore from a backup in order to recreate them.


== Guides ==
== Guides ==


=== OpenZFS Documentation for installing ===
=== Root on ZFS with disko ===
{{warning|This guide is not endorsed by NixOS and some features like immutable root do not have upstream support and could break on updates. If an issue arises while following this guide, please consult the guides support channels.}}
 
One guide for a NixOS installation with ZFS is maintained at [https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/ OpenZFS Documentation (''Getting Started'' for ''NixOS'')]


It is about:
disko[https://github.com/nix-community/disko/blob/master/example/zfs.nix] can partition disks declaratively and handle mount points at install time.
* [https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/index.html#installation Enabling ZFS on an existing NixOS installation]
* [https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/#root-on-zfs (Installing NixOS with) Root on ZFS].


It is not about:
Don't follow the Root on ZFS guide found in OpenZFS documentation. It was abandoned and has not been updated in years. See commit log for the openzfs-docs repo for details.
* Giving understandable, easy to follow instructions which are close to the standard installation guide
* Integrating ZFS into your existing config


=== Simple NixOS ZFS on root installation ===
=== Simple NixOS ZFS on root installation ===
Line 330: Line 326:
</syntaxhighlight>
</syntaxhighlight>


You can confirm whether any specified configuration/tuning got applied via commands like <code>arc_summary</code> and <code>arcstat -a -s " "</code>.
You can confirm whether any specified configuration/tuning got applied via commands like <code>zarcsummary</code> and <code>zarcstat -a -s " "</code>.


== Automatic scrubbing ==
== Automatic scrubbing ==
Line 343: Line 339:
=== Unlock encrypted ZFS via SSH on boot ===
=== Unlock encrypted ZFS via SSH on boot ===


{{note|As of 22.05, rebuilding your config with the below directions may result in a situation where, if you want to revert the changes, you may need to do some pretty hairy nix-store manipulation to be able to successfully rebuild, see https://github.com/NixOS/nixpkgs/issues/101462#issuecomment-1172926129}}
{{merge|Remote_disk_unlocking}}In case you want unlock a machine remotely (after an update), having an ssh service in initrd for the password prompt is handy:
 
In case you want unlock a machine remotely (after an update), having an ssh service in initrd for the password prompt is handy:


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
Line 426: Line 420:
== Take snapshots automatically ==
== Take snapshots automatically ==


See {{nixos:option|services.sanoid}} section in <code>man configuration.nix</code>.
See {{nixos:option|services.zfs.autoSnapshot}} or {{nixos:option|services.sanoid}} section in <code>man configuration.nix</code>.


== NFS share ==
== NFS share ==
Line 441: Line 435:
</syntaxhighlight>
</syntaxhighlight>
Only this line is needed. Configure firewall if necessary, as described in [[NFS]] article.
Only this line is needed. Configure firewall if necessary, as described in [[NFS]] article.
{{warning|<code>zfs share</code> or <code>sharenfs</code> does not work if the <code>mountpoint</code> is set to <code>legacy</code> (or <code>none</code>, of course). I was unable to find a source for this behaviour, but I was stuck on the problem for days, until I realized the problem.  ::Reply: sharenfs controlls what
is written into <code>/etc/exports</code>.  If ZFS does not know the mountpoint, as is the case in
mountpoint legacy or none, the contents of <code>/etc/exports</code> would be wrong}}


Then, set <code>sharenfs</code> property:
Then, set <code>sharenfs</code> property:
Line 453: Line 451:


ZFS Event Daemon (zed) monitors events generated by the ZFS Kernel module and runs configured tasks. It can be configured to send an email when a pool scrub is finished or a disk has failed. [https://search.nixos.org/options?query=services.zfs.zed zed options]
ZFS Event Daemon (zed) monitors events generated by the ZFS Kernel module and runs configured tasks. It can be configured to send an email when a pool scrub is finished or a disk has failed. [https://search.nixos.org/options?query=services.zfs.zed zed options]
=== Option A: enable mail notifications without re-compliation ===


First, we need to configure a mail transfer agent, the program that sends email:
First, we need to configure a mail transfer agent, the program that sends email:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
{
{
  age.secrets.msmtp = {
    file = "${inputs.self.outPath}/secrets/msmtp.age";
  };
  # for zed enableMail, enable sendmailSetuidWrapper
  services.mail.sendmailSetuidWrapper.enable = true;
   programs.msmtp = {
   programs.msmtp = {
     enable = true;
     enable = true;
Line 464: Line 467:
     defaults = {
     defaults = {
       aliases = "/etc/aliases";
       aliases = "/etc/aliases";
       port = 465;
       port = 587;
       tls_trust_file = "/etc/ssl/certs/ca-certificates.crt";
       auth = "plain";
       tls = "on";
       tls = "on";
      auth = "login";
       tls_starttls = "on";
       tls_starttls = "off";
     };
     };
     accounts = {
     accounts = {
       default = {
       default = {
         host = "mail.example.com";
         host = "smtp.mail.example.com";
         passwordeval = "cat /etc/emailpass.txt";
         passwordeval = "cat ${config.age.secrets.msmtp.path}";
         user = "user@example.com";
         user = "myname@example.com";
         from = "user@example.com";
         from = "myname@example.com";
       };
       };
     };
     };
Line 487: Line 489:
{
{
   environment.etc.aliases.text = ''
   environment.etc.aliases.text = ''
     root: you@example.com
     root: admin@example.com
   '';
   '';
}
}
</syntaxhighlight>
</syntaxhighlight>


Finally, override default zed settings with a custom one:
Finally, enable zed mail notification:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
{
{
   services.zfs.zed.settings = {
   services.zfs.zed = {
     ZED_DEBUG_LOG = "/tmp/zed.debug.log";
     enableMail = true;
     ZED_EMAIL_ADDR = [ "root" ];
     settings = {
    ZED_EMAIL_PROG = "${pkgs.msmtp}/bin/msmtp";
      ZED_EMAIL_ADDR = [ "root" ];
    ZED_EMAIL_OPTS = "@ADDRESS@";
      # send notification if scrub succeeds
 
      ZED_NOTIFY_VERBOSE = true;
    ZED_NOTIFY_INTERVAL_SECS = 3600;
     };
    ZED_NOTIFY_VERBOSE = true;
 
     ZED_USE_ENCLOSURE_LEDS = true;
    ZED_SCRUB_AFTER_RESILVER = true;
   };
   };
  # this option does not work; will return error
  services.zfs.zed.enableMail = false;
}
}
</syntaxhighlight>
</syntaxhighlight>
Line 517: Line 513:
</syntaxhighlight>
</syntaxhighlight>


=== Option B: Rebuild ZFS with mail support ===
The <code>zfs</code> package can be rebuilt with mail features. However, please note that this will cause Nix to recompile the entire ZFS package on the computer, and on every Kernel update, which could be very time-consuming on lower-end NAS systems.
An alternative solution that does not involve recompliation can be found above.
The following override is needed as <code>zfs</code>is implicitly used in partition mounting:
<syntaxhighlight lang="nix">
nixpkgs.config.packageOverrides = pkgs: {
  zfsStable = pkgs.zfsStable.override { enableMail = true; };
};
</syntaxhighlight>
A mail sender like [[msmtp]] or [[postfix]] is required.
A minimal, testable ZED configuration example:
<syntaxhighlight lang="nix">
services.zfs.zed.enableMail = true;
services.zfs.zed.settings = {
  ZED_EMAIL_ADDR = [ "root" ];
  ZED_NOTIFY_VERBOSE = true;
};
</syntaxhighlight>
Above, <code>ZED_EMAIL_ADDR</code> is set to <code>root</code>, which most people will have an alias for in their mailer. You can change it to directly mail you: <code>ZED_EMAIL_ADDR = [ "you@example.com" ];</code>
ZED pulls in <code>mailutils</code> and runs <code>mail</code> by default, but you can override it with <code>ZED_EMAIL_PROG</code>. If using msmtp, you may need <code>ZED_EMAIL_PROG = "${pkgs.msmtp}/bin/msmtp";</code>.
You can customize the mail command with <code>ZED_EMAIL_OPTS</code>. For example, if your upstream mail server requires a certain FROM address: <code>ZED_EMAIL_OPTS = "-r 'noreply@example.com' -s '@SUBJECT@' @ADDRESS@";</code>


[[Category:Guide]]
[[Category:Guide]]
Retrieved from "https://wiki.nixos.org/wiki/ZFS"