Official NixOS Wiki

Binary Cache: Difference between revisions

← Older edit
Axka (talk | contribs)
92 edits
VisualWikitext
Drupol (talk | contribs)
13 edits
mNo edit summary
Axka (talk | contribs)
92 edits
 
(4 intermediate revisions by 2 users not shown)
Line 2: Line 2:


A binary cache builds Nix packages and caches the result for other machines. Any machine with Nix installed can be a binary cache for another one, no matter the operating system.
A binary cache builds Nix packages and caches the result for other machines. Any machine with Nix installed can be a binary cache for another one, no matter the operating system.
== Setting up a binary cache with attic and caddy ==
Here's a snippet enabling [https://github.com/zhaofengli/attic Attic] and [https://caddyserver.com/ Caddy].
Please refer to the [https://docs.attic.rs/ Attic documentation] to set it up correctly. The goal here is to show how those two services can be used together to provide a solid solution.<syntaxhighlight lang="nix" line="1">
{
  networking.firewall = {
    allowedTCPPorts = [ 8080 ];
  };
  services = {
    atticd = {
      enable = true;
      settings = {
        listen = "127.0.0.1:8081";
      };
      # Path to an EnvironmentFile containing required environment variables:
      # ATTIC_SERVER_TOKEN_RS256_SECRET_BASE64: The base64-encoded RSA PEM PKCS1 of the RS256 JWT secret. Generate it with openssl genrsa -traditional 4096 | base64 -w0.
      environmentFile = "/root/.attic-env-file";
    };
    # Inspired from:
    # 1. https://github.com/phanirithvij/system/blob/main/nixos/applications/nix/selfhosted/proxy-cache.nix
    # 2. https://github.com/rnl-dei/nixrnl/blob/master/profiles/proxy-cache.nix
    caddy = {
      enable = true;
      package = pkgs.caddy.withPlugins {
        plugins = [ "github.com/caddyserver/cache-handler@v0.16.0" ];
        hash = "sha256-CecAx6KelOHEDiOKDTKLlDcnWtRNnDzBw1AzgN5JaFw=";
      };
      globalConfig = ''
        order cache before rewrite
        cache {
          # Global default cache duration (if not overridden below)
          ttl 1h
          log_level debug
        }
      '';
      virtualHosts.":8080" = {
        extraConfig = ''
          log {
            format console
          }
          # Nix cache info endpoint
          @nix_cache_info path /nix-cache-info
          handle @nix_cache_info {
            header Cache-Control "public, max-age=300"
            # 2. Tell Caddy's internal cache to hold this for 5 minutes
            cache {
              ttl 300s
            }
            reverse_proxy https://cache.nixos.org {
              header_up Host cache.nixos.org
            }
          }
          # NAR files (the actual packages)
          @nar path /nar/*
          handle @nar {
            header Cache-Control "public, max-age=31536000, immutable"
            # Cache the actual nar packages for a year
            cache {
              ttl 8760h
            }
            reverse_proxy https://cache.nixos.org {
              header_up Host cache.nixos.org
            }
          }
          # Narinfo files (metadata about packages)
          @narinfo path_regexp ^/[^/]+\.narinfo$
          handle @narinfo {
            header Cache-Control "public, max-age=86400"
            # Narinfo can change, so cache them locally for 24 hours
            cache {
              ttl 24h
            }
            reverse_proxy https://cache.nixos.org {
              header_up Host cache.nixos.org
            }
          }
          # Fallback for other requests
          handle {
            # We omit the `cache` directive here so Caddy doesn't interfere
            # with Attic's API operations or package pushing (PUT/POST requests).
            reverse_proxy 127.0.0.1:8081
          }
        '';
      };
    };
  };
};
</syntaxhighlight>


== Setting up a binary cache with nix-serve and nginx ==
== Setting up a binary cache with nix-serve and nginx ==
Line 7: Line 112:
This tutorial explains how to setup a machine as a binary cache for other machines, serving the nix store on TCP port 80 with signing turned on. It assumes that an {{ic|[[nginx]]}} service is already running, that port 80 is open,<ref group="cf."> {{manual:nixos|sec=#sec-firewall|chapter=11.5. Firewall}}</ref> and that the hostname {{ic|binarycache.example.com}} resolves to the server.<ref group="cf.">{{nixos:option|networking.hostName}}</ref>
This tutorial explains how to setup a machine as a binary cache for other machines, serving the nix store on TCP port 80 with signing turned on. It assumes that an {{ic|[[nginx]]}} service is already running, that port 80 is open,<ref group="cf."> {{manual:nixos|sec=#sec-firewall|chapter=11.5. Firewall}}</ref> and that the hostname {{ic|binarycache.example.com}} resolves to the server.<ref group="cf.">{{nixos:option|networking.hostName}}</ref>


=== 1. Generating a private/public keypair ===
=== 1. Generating a signing key ===
 
A keypair is necessary to sign Nix packages. Replace <code>binarycache.example.com</code> with your domain.


{{bc|
Follow the generation process at [[Signing store paths#Signing Key]], replacing the example hostname with your own. In this tutorial, {{ic|nix-serve}} signs packages on the fly when it serves them so any [[Nix (package manager)|Nix]] configuration isn't required.
cd /var
nix-store --generate-binary-cache-key binarycache.example.com cache-priv-key.pem cache-pub-key.pem
chown nix-serve cache-priv-key.pem
chmod 600 cache-priv-key.pem
cat cache-pub-key.pem
}}
 
The packages can be signed before adding them to the binary cache, or on the fly as they are served.
In this tutorial we'll set up {{ic|nix-serve}} to sign packages on the fly when it serves them.
In this case it is important that only {{ic|nix-serve}} can access the private key.
The location {{ic|/var/cache-priv-key.pem}} is just an example.


=== 2. Activating {{ic|nix-serve}} ===
=== 2. Activating {{ic|nix-serve}} ===


{{ic|nix-serve}} is the service that speaks the binary cache protocol via HTTP.
{{ic|nix-serve}} is the service that serves the binary cache protocol via HTTP.


To start it on NixOS:
To start it on NixOS:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">services.nix-serve = {
services.nix-serve = {
   enable = true;
   enable = true;
   secretKeyFile = "/var/cache-priv-key.pem";
 
};
  # Note: You don't need to give nix-serve ownership of the file because systemd reads it.
</syntaxhighlight>
   secretKeyFile = "/var/secrets/nix-cache-priv-key";
};</syntaxhighlight>


To start it on a non-NixOS machine at boot, add to {{ic|/etc/crontab}}:
To start it on a non-NixOS machine at boot, add to {{ic|/etc/crontab}}:
<syntaxhighlight lang="crontab">
<syntaxhighlight>
NIX_SECRET_KEY_FILE=/var/cache-priv-key.pem
NIX_SECRET_KEY_FILE=/var/secrets/nix-cache-priv-key
@reboot /home/USER/.nix-profile/bin/nix-serve --listen :5000 --error-log /var/log/nix-serve.log --pid /var/run/nix-serve.pid --user USER --daemonize
@reboot /home/USER/.nix-profile/bin/nix-serve --listen :5000 --error-log /var/log/nix-serve.log --pid /var/run/nix-serve.pid --user USER --daemonize
</syntaxhighlight>
</syntaxhighlight>
Line 78: Line 170:
=== 4. Testing ===
=== 4. Testing ===


To apply the previous settings to your NixOS machine, run:
To apply the previous settings to your NixOS machine, run:<syntaxhighlight lang="shell-session">
{{bc|# nixos-rebuild switch}}
# nixos-rebuild switch
 
</syntaxhighlight>Check the general availability:
Check the general availability:
<syntaxhighlight lang="shell-session">
<syntaxhighlight lang="bash">
$ curl http://binarycache.example.com/nix-cache-info
$ curl http://binarycache.example.com/nix-cache-info
StoreDir: /nix/store
StoreDir: /nix/store
Line 90: Line 181:


On the binary cache server, build some package:
On the binary cache server, build some package:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="shell-session">
$ nix-build '<nixpkgs>' -A pkgs.hello
$ nix-build '<nixpkgs>' -A pkgs.hello
/nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10
/nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10
Line 96: Line 187:


To verify the signing on the fly, make sure the following request contains a {{ic|Sig:}} line:
To verify the signing on the fly, make sure the following request contains a {{ic|Sig:}} line:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="shell-session">
$ curl http://binarycache.example.com/gdh8165b7rg4y53v64chjys7mbbw89f9.narinfo
$ curl http://binarycache.example.com/gdh8165b7rg4y53v64chjys7mbbw89f9.narinfo
StorePath: /nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10
StorePath: /nix/store/gdh8165b7rg4y53v64chjys7mbbw89f9-hello-2.10
Line 134: Line 225:
     };
     };
   };
   };
</syntaxhighlight>As described on [https://search.nixos.org/options?show=nix.settings.substituters&type=packages&query=substituters search.nixos.org] by default <nowiki>https://cache.nixos.org/</nowiki> is added to the substituters. You may need to use lib.mkForce to override this and ensure your substituter is the primary choice.<syntaxhighlight>
</syntaxhighlight>As described on [https://search.nixos.org/options?show=nix.settings.substituters&type=packages&query=substituters search.nixos.org] by default <nowiki>https://cache.nixos.org/</nowiki> is added to the substituters. You may need to use lib.mkForce to override this and ensure your substituter is the primary choice.<syntaxhighlight lang="nix">
# /etc/nixos/configuration.nix
# /etc/nixos/configuration.nix


Line 192: Line 283:
You can place a hint to your binary cache in your Flake so when someone builds an output of your Flake, the nix command will ask interactively to trust the specified binary cache.
You can place a hint to your binary cache in your Flake so when someone builds an output of your Flake, the nix command will ask interactively to trust the specified binary cache.


<syntaxHighlight lang=nix>
<syntaxHighlight lang="nix">
{
{
   nixConfig = {
   nixConfig = {
Retrieved from "https://wiki.nixos.org/wiki/Binary_Cache"