Binary Cache: Difference between revisions

From NixOS Wiki
imported>Tfc
m add note about where the pubkey comes from
imported>Tfc
Incorporate musicmatze
Line 5: Line 5:
This tutorial assumes:
This tutorial assumes:


* NixOS
* NixOS on the serving machine
* You want to serve the nix store via port 80 and already have an nginx service running
* You want to serve the nix store via port 80 and already have an nginx service running (See here how to configure nginx [https://nixos.wiki/wiki/Nginx])
* The store is served with signing activated
* The store is served with signing activated
* Firewall rules (port 80) are already set up
* Firewall rules (port 80) are already set up (See firewall settings in NixOS manual[https://nixos.org/nixos/manual/index.html#sec-firewall])
* The hostname "binarycache.mydomain.com" points to the server
* The hostname "binarycache.mydomain.com" points to the server (see <code>networking.hostName</code>[https://nixos.org/nixos/manual/options.html#opt-networking.hostName])


The following steps are needed:
This tutorial explains the following steps one by one in the following:


# Generate a private/public keypair
# Generate a private/public keypair for signing packages
#* This step is not mandatory, but we are going to serve signed nix packages
#* This step is not mandatory, but we are going to serve signed nix packages
# Activate <code>nix-serve</code>
# Activate <code>nix-serve</code>
Line 65: Line 65:
</syntaxhighlight>
</syntaxhighlight>


Add HTTPS settings to this config if possible. The tutorial will simply continue with unsecure HTTP.
Add HTTPS settings to this config if possible. The tutorial will simply continue with unsecure HTTP. (See NixOS manual on how to set up HTTPS with letsencrypt[https://nixos.org/nixos/manual/index.html#module-security-acme-nginx])


=== 4. Rebuild the NixOS config and test ===
=== 4. Rebuild the NixOS config and test ===

Revision as of 11:57, 14 April 2019

Every machine with nix installed can be a configured to be a binary cache for another nix machine - no matter if it is a NixOS, other Linux distro, or MacOS machine.

Setting up a NixOS Machine as Binary Cache for Others

This tutorial assumes:

  • NixOS on the serving machine
  • You want to serve the nix store via port 80 and already have an nginx service running (See here how to configure nginx [1])
  • The store is served with signing activated
  • Firewall rules (port 80) are already set up (See firewall settings in NixOS manual[2])
  • The hostname "binarycache.mydomain.com" points to the server (see networking.hostName[3])

This tutorial explains the following steps one by one in the following:

  1. Generate a private/public keypair for signing packages
    • This step is not mandatory, but we are going to serve signed nix packages
  2. Activate nix-serve
    • This is the service that speaks the binary cache protocol via HTTP.
    • It will use the private key to sign the packages
  3. Create a virtual hostname in nginx
    • It will redirect the HTTP(s) traffic from port 80 to nix-serve
  4. Rebuild the NixOS config and test

1. Generate a private/public keypair

$ nix-store --generate-binary-cache-key binarycache.mydomain.com1 cache-priv-key.pem cache-pub-key.pem
$ sudo mv cache-priv-key.pem /var/cache-priv-key.pem
$ sudo chown nix-store /var/cache-priv-key.pem
$ sudo chmod 600 /var/cache-priv-key.pem

It is important that only nix-serve can access the private key. The location /var/cache-priv-key.pem is just an example.

2. Activate nix-serve

services.nix-serve = {
  enable = true;
  secretKeyFile = "/var/cache-priv-key.pem";
};

nix-serve will by default serve on port 5000. We are not going to open a firewall port for it, because we will let nginx redirect to it.

3. Create a virtual hostname in nginx

services.nginx = {
  enable = true;
  virtualHosts = {
    # ... existing hosts config etc. ...
    "binarycache.mydomain.com" = {
      serverAliases = [ "binarycache" ];
      locations."/".extraConfig = ''
        proxy_pass http://localhost:${toString config.services.nix-serve.port};
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      '';
    };
  };
};

Add HTTPS settings to this config if possible. The tutorial will simply continue with unsecure HTTP. (See NixOS manual on how to set up HTTPS with letsencrypt[4])

4. Rebuild the NixOS config and test

sudo nixos-rebuild switch

Check the general availability:

$ curl http://binarycache.mydomain.com/nix-cache-info
StoreDir: /nix/store
WantMassQuery: 1
Priority: 30

On the binary cache server, build some package:

$ nix-build '<nixpkgs>' -A pkgs.hello
/nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10

On some other machine, test if nix can realize the same package via the binary cache (the content of the trusted-public-keys is the content of the file cache-pub-key.pem which you should share with users of the binary cache!):

$ nix-store -r /nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10 --option substituters http://binarycache.mydomain.com --option trusted-public-keys binarycache.mydomain.com1:dsafdafDFW123fdasfa123124FADSAD
these paths will be fetched (0.00 MiB download, 24.04 MiB unpacked):
  /nix/store/7gx4kiv5m0i7d7qkixq2cwzbr10lvxwc-glibc-2.27
  /nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10
copying path '/nix/store/7gx4kiv5m0i7d7qkixq2cwzbr10lvxwc-glibc-2.27' from 'http://binarycache.mydomain.com'...
copying path '/nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10' from 'http://binarycache.mydomain.com'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10

It works, great! Now you can configure your nix machines to look up packages always/also on this server. For that, have a look into the Nix manual [5] and the following settings:

  • substituters
  • extra-substituters
  • trusted-public-keys