CCache

CCache is useful for Packaging large packages with Incremental builds.

With CCache, recompile time can be reduced from many hours to a few minutes.

NixOS

On NixOS, the programs.ccache module can be used to partially enable CCache.

programs.ccache.enable = true;

However, without specifying programs.ccache.packageNames the CCache wrapper is not configured. The wrapper configuration can be added to your Nix overlays.

nixpkgs.overlays = [
  (self: super: {
    ccacheWrapper = super.ccacheWrapper.override {
      extraConfig = ''
        export CCACHE_COMPRESS=1
        export CCACHE_DIR="${config.programs.ccache.cacheDir}"
        export CCACHE_UMASK=007
        if [ ! -d "$CCACHE_DIR" ]; then
          echo "====="
          echo "Directory '$CCACHE_DIR' does not exist"
          echo "Please create it with:"
          echo "  sudo mkdir -m0770 '$CCACHE_DIR'"
          echo "  sudo chown root:nixbld '$CCACHE_DIR'"
          echo "====="
          exit 1
        fi
        if [ ! -w "$CCACHE_DIR" ]; then
          echo "====="
          echo "Directory '$CCACHE_DIR' is not accessible for user $(whoami)"
          echo "Please verify its access permissions"
          echo "====="
          exit 1
        fi
      '';
    };
  })
];

The CCache directory also needs to be added to the builder sandboxes.

nix.settings.extra-sandbox-paths = [ config.programs.ccache.cacheDir ];

Run sudo nixos-rebuild switch to enable these options before attempting to use CCache for a derivation.

Derivation CCache

Packages can built with CCache by overriding stdenv in the derivation.

nixpkgs.overlays = [
  (self: super: {
    ffmpeg = super.ffmpeg.override { stdenv = super.ccacheStdenv; };
  })
];

Note, that if the package is a top-level package, you can also add it to the programs.ccache.packageNames list.

programs.ccache.packageNames = [ "ffmpeg" ];

Some packages do not use stdenv directly. You may need to plumb it through other dependencies first.

System CCache

todo

Monitor CCache

The NixOS module creates a script that can be used to monitor the CCache directory without sudo.

nix-ccache --show-stats

Non-NixOS

Ceate the cache folder:

sudo mkdir -m0770 -p /nix/var/cache/ccache

# Linux
sudo chown --reference=/nix/store /nix/var/cache/ccache

# macOS workaround for chown --reference
nix-shell -p coreutils --run 'sudo chown --reference=/nix/store /nix/var/cache/ccache'

Add the path to the derivation sandbox by adding extra-sandbox-paths to nix.conf

extra-sandbox-paths = /nix/var/cache/ccache

Then configure the CCache wrapper script.

nixpkgs.overlays = [
  (self: super: {
    ccacheWrapper = super.ccacheWrapper.override {
      extraConfig = ''
        export CCACHE_COMPRESS=1
        export CCACHE_DIR="/nix/var/cache/ccache"
        export CCACHE_UMASK=007
        if [ ! -d "$CCACHE_DIR" ]; then
          echo "====="
          echo "Directory '$CCACHE_DIR' does not exist"
          echo "Please create it with:"
          echo "  sudo mkdir -m0770 '$CCACHE_DIR'"
          echo "  sudo chown root:nixbld '$CCACHE_DIR'"
          echo "====="
          exit 1
        fi
        if [ ! -w "$CCACHE_DIR" ]; then
          echo "====="
          echo "Directory '$CCACHE_DIR' is not accessible for user $(whoami)"
          echo "Please verify its access permissions"
          echo "====="
          exit 1
        fi
      '';
    };
  })
];

Derivation CCache

Packages can built with CCache by overriding stdenv in the derivation.

nixpkgs.overlays = [
  (self: super: {
    ffmpeg = super.ffmpeg.override { stdenv = super.ccacheStdenv; };
  })
];

Some packages do not use stdenv directly. You may need to plumb it through other dependencies first.

Monitor CCache status

# watch ccache size
sudo watch du -sh /nix/var/cache/ccache

# watch ccache stats
sudo watch ccache --dir /nix/var/cache/ccache --show-stats

Uncacheable

TODO: why are some compilations "uncacheable" according to ccache --show-stats

See also

• https://leanprover.github.io/lean4/doc/make/nix.html
• sandbox-paths in nix.conf