NFS: Difference between revisions

From NixOS Wiki
imported>Makefu
initial batch of nixos-users
 
Nh2 (talk | contribs)
add TODO regarding local_lock=all on Nix store
 
(25 intermediate revisions by 14 users not shown)
Line 1: Line 1:
__FORCETOC__
== Server ==
== Server ==


The setup is very similar as it would be done in regular config file. I will use my setup as an example.
Let's say that we've got one server-machine with 4 directories that we want to share: <code>/mnt/kotomi</code>, <code>/mnt/mafuyu</code>, <code>/mnt/sen</code> and <code>/mnt/tomoyo</code>.


I wish to share 4 mount-points (/mnt/kotomi, /mnt/mafuyu, /mnt/sen, /mnt/tomoyo) with my other computers which will run NFS clients.
First, we have to create a dedicated directory from which our NFS server will access the data:


First I created a separate directory for NFS shares:
<syntaxhighlight lang="console">
$ mkdir /export
</syntaxhighlight>


<pre class="console">$ mkdir /export</pre>
You may need to change ownership of the <code>/export</code> directory to <code>nobody:nogroup</code>
Then I mount (bind) the locations inside of /export from my config. Normally one would put it in /etc/fstab but nix generates that for us:


<pre class="nix">{
Then we have to either move our already-existing directories inside <code>/export</code> (using <code>mv</code> from the command line) or bind-mount them there:
     fileSystems.&quot;/export/mafuyu&quot; = {
 
       device = &quot;/mnt/mafuyu&quot;;
<syntaxhighlight lang="nix">
      options = &quot;bind&quot;;
{
    };
  fileSystems."/export/mafuyu" = {
     device = "/mnt/mafuyu";
    options = [ "bind" ];
  };
 
  fileSystems."/export/sen" = {
    device = "/mnt/sen";
    options = [ "bind" ];
  };
 
  fileSystems."/export/tomoyo" = {
    device = "/mnt/tomoyo";
    options = [ "bind" ];
  };
 
  fileSystems."/export/kotomi" = {
    device = "/mnt/kotomi";
    options = [ "bind" ];
  };
}
</syntaxhighlight>
 
Having the filesystem ready, we can proceed to configure the NFS server itself:
 
<syntaxhighlight lang="nix">{
  services.nfs.server.enable = true;
  services.nfs.server.exports = ''
    /export        192.168.1.10(rw,fsid=0,no_subtree_check) 192.168.1.15(rw,fsid=0,no_subtree_check)
    /export/kotomi  192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/mafuyu 192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/sen    192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/tomoyo  192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
  '';
}</syntaxhighlight>
This configuration exposes all our shares to 2 local IPs; you can find more examples at Gentoo's wiki [https://wiki.gentoo.org/wiki/NFSv4].
 
Other options are available on the [https://search.nixos.org/options?query=nfs NixOS option page] or via the <code>nixos-option</code> command.
 
=== Firewall ===
If your server-machine has a firewall turned on (as NixOS does by default, for instance), don't forget to open appropriate ports; e.g. for NFSv4:
<syntaxhighlight lang="nix">
networking.firewall.allowedTCPPorts = [ 2049 ];
</syntaxhighlight>
 
Many clients only support NFSv3, which requires the server to have fixed ports:
<syntaxhighlight lang="nix">
  services.nfs.server = {
    enable = true;
    # fixed rpc.statd port; for firewall
    lockdPort = 4001;
    mountdPort = 4002;
    statdPort = 4000;
    extraNfsdConfig = '''';
  };
  networking.firewall = {
    enable = true;
       # for NFSv3; view with `rpcinfo -p`
    allowedTCPPorts = [ 111  2049 4000 4001 4002 20048 ];
    allowedUDPPorts = [ 111 2049 4000 4001  4002 20048 ];
  };
</syntaxhighlight>
 
== Client ==
 
Continuing the server example, mounting the now-exposed ''tomoyo'' share on another box (on a client) is as simple as:
 
<syntaxhighlight lang="nix">
{
  fileSystems."/mnt/tomoyo" = {
    device = "server:/tomoyo";
    fsType = "nfs";
  };
}
</syntaxhighlight>
Replace "server" in the above device attribute with the IP address or DNS entry of the NFS server. Note that clients see exposed shares as if they were exposed at the root level - i.e. <code>/export/foo</code> becomes <code>/foo</code> (in the <code>device</code> option). Other, regular '''fileSystems''' options apply.
 
=== Specifying NFS version ===
 
You can specify NFS version by adding the <code>"nfsvers="</code> option:
<syntaxhighlight lang="nix">
{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "nfsvers=4.2" ];
  };
}
</syntaxhighlight>
 
=== Lazy-mounting ===
 
By default, all shares will be mounted right when your machine starts - apart from being simply unwanted sometimes, this may also cause issues when your computer doesn't have a stable network connection or uses WiFi; you can fix this by telling systemd to mount your shares the first time they are ''accessed'' (instead of keeping them mounted at all times):
 
<syntaxhighlight lang="nix">
{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "x-systemd.automount" "noauto" ];
  };
}
</syntaxhighlight>
 
=== Auto-disconnecting ===
 
You can tell systemd to disconnect your NFS-client from the NFS-server when the directory has not been accessed for some time:
 
<syntaxhighlight lang="nix">
{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "x-systemd.idle-timeout=600" ]; # disconnects after 10 minutes (i.e. 600 seconds)
  };
}
</syntaxhighlight>
 
=== Using systemd.mounts and systemd.automounts ===
 
Here is an example with auto-disconnecting and lazy-mounting implemented, and the <code>noatime</code> mount option added.
 
Note that <code>wantedBy = [ "multi-user.target" ];</code> is required for the automount unit to start at boot.


    fileSystems.&quot;/export/sen&quot; = {
Also note that <code>x-systemd</code> mount options are unneeded, as they are a representation of systemd options in <code>fstab(5)</code> format. They get parsed and converted to unit files by <code>systemd-fstab-generator(8)</code> as mentioned in <code>systemd.mount(5)</code>.
      device = &quot;/mnt/sen&quot;;
      options = &quot;bind&quot;;
    };


    fileSystems.&quot;/export/tomoyo&quot; = {
<syntaxhighlight lang="nix">
      device = &quot;/mnt/tomoyo&quot;;
{
       options = &quot;bind&quot;;
  services.rpcbind.enable = true; # needed for NFS
  systemd.mounts = [{
    type = "nfs";
    mountConfig = {
       Options = "noatime";
     };
     };
    what = "server:/tomoyo";
    where = "/mnt/tomoyo";
  }];


    fileSystems.&quot;/export/kotomi&quot; = {
  systemd.automounts = [{
      device = &quot;/mnt/kotomi&quot;;
    wantedBy = [ "multi-user.target" ];
       options = &quot;bind&quot;;
    automountConfig = {
       TimeoutIdleSec = "600";
     };
     };
}</pre>
    where = "/mnt/tomoyo";
Next we have to tell nix how we want to export these and to whom:
  }];
}
</syntaxhighlight>


<pre class="nix">{
Multiple mounts with the exact same options can benefit from abstraction.
    services.nfs.server.enable = true;
    services.nfs.server.exports = ''
      /export                192.168.1.10(rw,fsid=0,no_subtree_check) 192.168.1.15(rw,fsid=0,no_subtree_check)
      /export/kotomi          192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
      /export/mafuyu          192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
      /export/sen            192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
      /export/tomoyo          192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    '';
}</pre>
Here I export all my bound shares to 2 local IPs. For various flags, you can check the [https://wiki.gentoo.org/wiki/NFSv4#Server Gentoo wiki NFSv4 article] which has a nice coverage.


Other options are available on the [https://nixos.org/nixos/options.html#nfs NixOS option page] or via the <code>nixos-option</code> command
<syntaxhighlight lang="nix">
{
  services.rpcbind.enable = true; # needed for NFS
  systemd.mounts = let commonMountOptions = {
    type = "nfs";
    mountConfig = {
      Options = "noatime";
    };
  };


Please remember that NixOS by default has a firewall turned on! Add rules to allow NFS traffic or switch it off if you don't need it.
  in
 
== Client ==


Setting up the client is very easy. To follow from the server example, say I want to mount the now exposed ''tomoyo'' share on another box, call it ''server'', to ''/mnt/tomoyo''.
  [
    (commonMountOptions // {
      what = "server:/tomoyo";
      where = "/mnt/tomoyo";
    })


All I have to do is to put
    (commonMountOptions // {
      what = "server:/oyomot";
      where = "/mnt/oyomot";
    })
  ];


<pre class="nix">{
  systemd.automounts = let commonAutoMountOptions = {
     fileSystems.&quot;/mnt/tomoyo&quot; = {
     wantedBy = [ "multi-user.target" ];
       device = &quot;server:/tomoyo&quot;;
    automountConfig = {
      fsType = &quot;nfs&quot;;
       TimeoutIdleSec = "600";
     };
     };
}</pre>
  };
Note that clients see the exposed shares as if they were exposed at the root level: ''/export/foo'' becomes ''/foo'' when client is concerned with mounting it. Regular '''fileSystems''' options apply.


If you experience trouble with NFS mounts failing on boot because the network is not ready, try adding the following line in your fileSystems mount definition:
  in


<pre class="nix">{
  [
   # ...
    (commonAutoMountOptions // { where = "/mnt/tomoyo"; })
  options = &quot;x-systemd.automount,noauto&quot;;
    (commonAutoMountOptions // { where = "/mnt/oyomot"; })
}</pre>
   ];
That way, the NFS mount action won't actually be performed until the first time the mountpoint is accessed.
}
</syntaxhighlight>


= Nix store on NFS =
== Nix store on NFS ==


In a single-user setup ('''not on Nixos''') the nix store can be also exported over NFS (common in HPC clusters) to share package over the networks. The only requirement is to also pass <code>local_lock=flock</code> or <code>local_lock=all</code> as mount option to allow the nix packages to take locks on modifications. Example entry in <code>fstab</code>:
In a single-user setup ('''not on Nixos''') the Nix store can be also exported over NFS (common in HPC clusters) to share package over the networks. The only requirement is to also pass <code>local_lock=flock</code> or <code>local_lock=all</code> as mount option to allow the nix packages to take locks on modifications. Example entry in <code>fstab</code>:


<pre>&lt;host_or_ip&gt;/nix /nix nfs nofail,x-systemd.device-timeout=4,local_lock=all 0 0</pre>
<syntaxhighlight lang="console"><host_or_ip>/nix /nix nfs nofail,x-systemd.device-timeout=4,local_lock=all 0 0</syntaxhighlight>'''TODO:''' Why this? That seems extremely unsafe. This disables NFS locks (which apply to all NFS clients), and makes locks ''local'', meaning a lock taken by one NFS client isn't seen by another, and both can take their locks. So this removes protection against concurrent writes, which Nix assumes.
[[Category:Filesystem]]

Latest revision as of 01:41, 22 July 2024

Server

Let's say that we've got one server-machine with 4 directories that we want to share: /mnt/kotomi, /mnt/mafuyu, /mnt/sen and /mnt/tomoyo.

First, we have to create a dedicated directory from which our NFS server will access the data:

$ mkdir /export

You may need to change ownership of the /export directory to nobody:nogroup

Then we have to either move our already-existing directories inside /export (using mv from the command line) or bind-mount them there:

{
  fileSystems."/export/mafuyu" = {
    device = "/mnt/mafuyu";
    options = [ "bind" ];
  };

  fileSystems."/export/sen" = {
    device = "/mnt/sen";
    options = [ "bind" ];
  };

  fileSystems."/export/tomoyo" = {
    device = "/mnt/tomoyo";
    options = [ "bind" ];
  };

  fileSystems."/export/kotomi" = {
    device = "/mnt/kotomi";
    options = [ "bind" ];
  };
}

Having the filesystem ready, we can proceed to configure the NFS server itself:

{
  services.nfs.server.enable = true;
  services.nfs.server.exports = ''
    /export         192.168.1.10(rw,fsid=0,no_subtree_check) 192.168.1.15(rw,fsid=0,no_subtree_check)
    /export/kotomi  192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/mafuyu  192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/sen     192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
    /export/tomoyo  192.168.1.10(rw,nohide,insecure,no_subtree_check) 192.168.1.15(rw,nohide,insecure,no_subtree_check)
  '';
}

This configuration exposes all our shares to 2 local IPs; you can find more examples at Gentoo's wiki [1].

Other options are available on the NixOS option page or via the nixos-option command.

Firewall

If your server-machine has a firewall turned on (as NixOS does by default, for instance), don't forget to open appropriate ports; e.g. for NFSv4:

networking.firewall.allowedTCPPorts = [ 2049 ];

Many clients only support NFSv3, which requires the server to have fixed ports:

  services.nfs.server = {
    enable = true;
    # fixed rpc.statd port; for firewall
    lockdPort = 4001;
    mountdPort = 4002;
    statdPort = 4000;
    extraNfsdConfig = '''';
  };
  networking.firewall = {
    enable = true;
      # for NFSv3; view with `rpcinfo -p`
    allowedTCPPorts = [ 111  2049 4000 4001 4002 20048 ];
    allowedUDPPorts = [ 111 2049 4000 4001  4002 20048 ];
  };

Client

Continuing the server example, mounting the now-exposed tomoyo share on another box (on a client) is as simple as:

{
  fileSystems."/mnt/tomoyo" = {
    device = "server:/tomoyo";
    fsType = "nfs";
  };
}

Replace "server" in the above device attribute with the IP address or DNS entry of the NFS server. Note that clients see exposed shares as if they were exposed at the root level - i.e. /export/foo becomes /foo (in the device option). Other, regular fileSystems options apply.

Specifying NFS version

You can specify NFS version by adding the "nfsvers=" option:

{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "nfsvers=4.2" ];
  };
}

Lazy-mounting

By default, all shares will be mounted right when your machine starts - apart from being simply unwanted sometimes, this may also cause issues when your computer doesn't have a stable network connection or uses WiFi; you can fix this by telling systemd to mount your shares the first time they are accessed (instead of keeping them mounted at all times):

{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "x-systemd.automount" "noauto" ];
  };
}

Auto-disconnecting

You can tell systemd to disconnect your NFS-client from the NFS-server when the directory has not been accessed for some time:

{
  fileSystems."/mnt/tomoyo" = {
    # ...
    options = [ "x-systemd.idle-timeout=600" ]; # disconnects after 10 minutes (i.e. 600 seconds)
  };
}

Using systemd.mounts and systemd.automounts

Here is an example with auto-disconnecting and lazy-mounting implemented, and the noatime mount option added.

Note that wantedBy = [ "multi-user.target" ]; is required for the automount unit to start at boot.

Also note that x-systemd mount options are unneeded, as they are a representation of systemd options in fstab(5) format. They get parsed and converted to unit files by systemd-fstab-generator(8) as mentioned in systemd.mount(5).

{
  services.rpcbind.enable = true; # needed for NFS
  systemd.mounts = [{
    type = "nfs";
    mountConfig = {
      Options = "noatime";
    };
    what = "server:/tomoyo";
    where = "/mnt/tomoyo";
  }];

  systemd.automounts = [{
    wantedBy = [ "multi-user.target" ];
    automountConfig = {
      TimeoutIdleSec = "600";
    };
    where = "/mnt/tomoyo";
  }];
}

Multiple mounts with the exact same options can benefit from abstraction.

{
  services.rpcbind.enable = true; # needed for NFS
  systemd.mounts = let commonMountOptions = {
    type = "nfs";
    mountConfig = {
      Options = "noatime";
    };
  };

  in

  [
    (commonMountOptions // {
      what = "server:/tomoyo";
      where = "/mnt/tomoyo";
    })

    (commonMountOptions // {
      what = "server:/oyomot";
      where = "/mnt/oyomot";
    })
  ];

  systemd.automounts = let commonAutoMountOptions = {
    wantedBy = [ "multi-user.target" ];
    automountConfig = {
      TimeoutIdleSec = "600";
    };
  };

  in

  [
    (commonAutoMountOptions // { where = "/mnt/tomoyo"; })
    (commonAutoMountOptions // { where = "/mnt/oyomot"; })
  ];
}

Nix store on NFS

In a single-user setup (not on Nixos) the Nix store can be also exported over NFS (common in HPC clusters) to share package over the networks. The only requirement is to also pass local_lock=flock or local_lock=all as mount option to allow the nix packages to take locks on modifications. Example entry in fstab:

<host_or_ip>/nix /nix nfs nofail,x-systemd.device-timeout=4,local_lock=all 0 0

TODO: Why this? That seems extremely unsafe. This disables NFS locks (which apply to all NFS clients), and makes locks local, meaning a lock taken by one NFS client isn't seen by another, and both can take their locks. So this removes protection against concurrent writes, which Nix assumes.