Distributed build: Difference between revisions
Malteneuss (talk | contribs) m Simplify SSH connection table |
m →Modify the local machine's Nix config to know about the remote machine.: Remove erroneous bold header |
||
| (7 intermediate revisions by 5 users not shown) | |||
| Line 27: | Line 27: | ||
! SSH Connection Requirements | ! SSH Connection Requirements | ||
|- | |- | ||
| '''Multi-user''' || '''Multi-user''' || '''Local:''' {{ic|root}} user | | '''Multi-user''' || '''Multi-user''' || '''Local:''' {{ic|root}} user ------------SSH----> '''Remote''': ''any'' user '''(most frequent case)''' | ||
|- | |- | ||
| Single-user || Multi-user || '''Local:''' ''Your'' single-user | | Single-user || Multi-user || '''Local:''' ''Your'' single-user -----SSH----> '''Remote''': ''any'' user | ||
|- | |- | ||
| Multi-user || Single-user || '''Local:''' {{ic|root}} user | | Multi-user || Single-user || '''Local:''' {{ic|root}} user ------------SSH----> '''Remote''': ''your'' single-user for which Nix is installed with their UID (see [https://nixos.org/manual/nix/stable/installation/single-user.html Nix manual page]). | ||
|- | |- | ||
| Single-user || Single-user || '''Local:''' ''Your'' single-user | | Single-user || Single-user || '''Local:''' ''Your'' single-user -----SSH----> '''Remote''': ''your single-user'' each of which Nix is installed for with their UID (see [https://nixos.org/manual/nix/stable/installation/single-user.html Nix manual page]). | ||
|} | |} | ||
The thing to know about the '''"Multi-user"''' installation is that '''Nix is installed with a "nix-daemon" background process that runs as root''' and actually manages the builds on your behalf. So when you call '''"nix build ..." as a non-root user, this is delegated to the nix-daemon''' process, which runs as root. And this process can further delegate the build to a remote builder; that's why the '''local machine's root user''' needs the SSH access. | The thing to know about the '''"Multi-user"''' installation is that '''Nix is installed with a "nix-daemon" background process that runs as root''' and actually manages the builds on your behalf. So when you call '''"nix build ..." as a non-root user, this is delegated to the nix-daemon''' process, which runs as root. And this process can further delegate the build to a remote builder; that's why the '''local machine's root user''' needs the SSH access. | ||
{{Tip|The best | {{Tip|The best test to check that the SSH access works for Nix is to run on your local machine: | ||
nix store ping --store ssh://<REMOTE-BUILDER> | <code>nix store ping --store ssh://<REMOTE-BUILDER></code> | ||
Where <REMOTE-BUILDER> is the remote builder's IP address, host address or whatever name you configure in ~/.ssh/config or /root/.ssh/config, including the user@ prefix.}} | Where <code><REMOTE-BUILDER></code> is the remote builder's IP address, host address or whatever name you configure in ~/.ssh/config or /root/.ssh/config, including the user@ prefix.}} | ||
An alternative check is: | An alternative check is: | ||
| Line 66: | Line 66: | ||
For the common case where your local Nix is installed system-wide in multi-user mode, create a user on the '''remote''' machine that will have an unwriteable home directory, with a {{ic|~/.ssh/authorized_keys}} in it, that will allow SSH access to that user without a passphrase. The steps are: | For the common case where your local Nix is installed system-wide in multi-user mode, create a user on the '''remote''' machine that will have an unwriteable home directory, with a {{ic|~/.ssh/authorized_keys}} in it, that will allow SSH access to that user without a passphrase. The steps are: | ||
* {{ic|ssh}} to the remote builder. | |||
* Run (requires privileges) {{ic|useradd -m nixremote}}; {{ic|-m}} makes sure a home directory is created for the {{ic|nixremote}} user. | |||
* Run (requires privileges) {{ic|usermod nixremote -L}}; {{ic|-L}} locks the user such that nobody will be able to {{ic|su}} to it | |||
* Run (requires privileges) {{ic|mkdir ~nixremote/.ssh}}. Make sure to run this command as {{ic|nixremote}} user or {{ic|chown}} it afterwards | |||
If your '''remote builder''' has Nix installed system-wide in multi-user mode, but you're not running NixOS, '''you may need to add something like the following to your''' {{ic|/etc/ssh/sshd_config}} on this remote machine: | If your '''remote builder''' has Nix installed system-wide in multi-user mode, but you're not running NixOS, '''you may need to add something like the following to your''' {{ic|/etc/ssh/sshd_config}} on this remote machine: | ||
| Line 111: | Line 112: | ||
You may also want to make nix on '''the remote machine''' trust that new user by adding it to {{ic|nix.settings.trusted-users}} if it's using NixOS, or by manually adding <code><nowiki>trusted-users = nixremote</nowiki></code> to {{ic|/etc/nix/nix.conf}}. | You may also want to make nix on '''the remote machine''' trust that new user by adding it to {{ic|nix.settings.trusted-users}} if it's using NixOS, or by manually adding <code><nowiki>trusted-users = nixremote</nowiki></code> to {{ic|/etc/nix/nix.conf}}. | ||
== | == Modify the local machine's Nix config to know about the remote machine. == | ||
The Nix package manager '''on your local machine''' '''needs to know that the remote builder exists''' and what its ''supported features'' are. See [https://nixos.org/manual/nix/stable/command-ref/conf-file#conf-system-features official supportedFeatures documentation]. | The Nix package manager '''on your local machine''' '''needs to know that the remote builder exists''' and what its ''supported features'' are. See [https://nixos.org/manual/nix/stable/command-ref/conf-file#conf-system-features official supportedFeatures documentation]. | ||
| Line 140: | Line 141: | ||
# systems = ["x86_64-linux" "aarch64-linux" "riscv64-linux"]; | # systems = ["x86_64-linux" "aarch64-linux" "riscv64-linux"]; | ||
system = "x86_64-linux"; | system = "x86_64-linux"; | ||
# Nix custom ssh-variant that avoids lots of "trusted-users" settings pain | |||
protocol = "ssh-ng"; | protocol = "ssh-ng"; | ||
# default is 1 but may keep the builder idle in between builds | # default is 1 but may keep the builder idle in between builds | ||
| Line 151: | Line 153: | ||
nix.distributedBuilds = true; | nix.distributedBuilds = true; | ||
# optional, useful when the builder has a faster internet connection than yours | # optional, useful when the builder has a faster internet connection than yours | ||
nix. | nix.settings = { | ||
builders-use-substitutes = true | builders-use-substitutes = true; | ||
}; | |||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
| Line 249: | Line 251: | ||
== See also == | == See also == | ||
* [https://github.com/NixOS/nix/blob/ | * [https://github.com/NixOS/nix/blob/a6e6da3b0c579fc540acb00748fe3fd1858b9d99/tests/nixos/remote-builds.nix#L11-L21 The NixOS Remote Builds Test Case] | ||
* [https://nixos.org/nix-dev/2015-September/018255.html Mail to nixos-dev about setting up remote builds by Russell O'Connor] | * [https://nixos.org/nix-dev/2015-September/018255.html Mail to nixos-dev about setting up remote builds by Russell O'Connor] | ||
* [https://gist.github.com/danbst/09c3f6cd235ae11ccd03215d4542f7e7 A step-by-step guide on remote Firefox building through bastion host] | * [https://gist.github.com/danbst/09c3f6cd235ae11ccd03215d4542f7e7 A step-by-step guide on remote Firefox building through bastion host] | ||