Flakes: Difference between revisions

Revision as of 05:01, 31 August 2023

Nix flakes are an experimental feature of the Nix package manager.

Introduction

Flakes introduce a policy for managing dependencies between Nix expressions. It improves reproducibility, composability and usability in the Nix ecosystem. Although it's still an experimental feature, flakes have been widely used by the Nix community.

Technically, a flake is a file system tree that contains a Nix file named flake.nix in its root directory.

Flakes add the following behavior to Nix:

A flake.nix file offers a uniform schema, where:
- Other flakes can be referenced as dependencies providing Nix language code or other files.
- The values produced by the Nix expression in flake.nix are structured according to pre-defined use cases.
References to other flakes can be specified using a dedicated URL-like syntax. A flake registry allows using symbolic identifiers for further brevity. References can be automatically locked to their current specific version and later updated programmatically.
A new command line interface, implemented as a separate experimental feature, leverages flakes by accepting flake references in order to build, run, or deploy software defined as a flake.

Enable flakes

Flakes were introduced with Nix 2.4 (release notes).

Temporary

Add --experimental-features 'nix-command flakes' when using any nix3 commands.

Permanent

NixOS

Add the following to your system configuration:

  nix.settings.experimental-features = [ "nix-command" "flakes" ];

Other Distros: With Home-Manager

If you use HM, add the following to your home-manager config:

  nix = {
    package = pkgs.nix;
    settings.experimental-features = [ "nix-command" "flakes" ];
  };

Other Distros: Without Home-Manager

Note: Flakes are already enabled if you installed Nix using the Determinate Installer.

Add the following to ~/.config/nix/nix.conf or /etc/nix/nix.conf:

experimental-features = nix-command flakes

Basic project usage

Warning: Files that are part of the flake will be copied to the world-readable nix store when the flake is evaluated; avoid putting unencrypted secrets in your flake.

In your project, run nix flake init to generate the flake.nix file.

For flakes in git repos, only files in the working tree will be copied to the store.

Therefore, if you use git for your flake, ensure to git add any project files after you first create them.

Flake schema

The flake.nix file is a Nix file but that has special restrictions (more on that later).

It has 4 top-level attributes:

description is a string describing the flake.
inputs is an attribute set of all the dependencies of the flake. The schema is described below.
outputs is a function of one argument that takes an attribute set of all the realized inputs, and outputs another attribute set whose schema is described below.
nixConfig is an attribute set of values which reflect the values given to nix.conf. This can extend the normal behavior of a user's nix experience by adding flake-specific configuration, such as a binary cache.

Input schema

This is not a complete schema but should be enough to get you started:

{
  inputs = {
    # use release 23.05 branch of the GitHub repository as input, this is the most common input format
    nixpkgs.url = "github:NixOS/nixpkgs/release-23.05";
    # Git URL, can be used for any Git repository based on https/ssh protocol
    git-example.url = "git+https://git.somehost.tld/user/path?ref=branch&rev=fdc8ef970de2b4634e1b3dca296e1ed918459a9e";
    # You've created a fork of a git and are working on a new branch
    forked-git-flake.url = "git+file:/home/user/forked-flake?branch=feat/myNewFeature"
    # The example above will also copy .git, use this for (shallow) local Git repos
    git-directory-example.url = "git+file:/path/to/repo?shallow=1";

    # Local directories (for absolute paths you can omit 'path:')
    directory-example.url = "path:/path/to/repo";

    bar = {
      url = "github:foo/bar/branch";
      # if the input is not a flake, you need to set flake=false
      flake = false;
    };

    sops-nix = {
      url = "github:Mic92/sops-nix";
      # The `follows` keyword in inputs is used for inheritance.
      # Here, `inputs.nixpkgs` of sops-nix is kept consistent with the `inputs.nixpkgs` of
      # the current flake, to avoid problems caused by different versions of nixpkgs.
      inputs.nixpkgs.follows = "nixpkgs";
    };

    # Pin flakes to a specific revision
    nix-doom-emacs = {
      url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3";
      flake = false;
    };

    # To use a subdirectory of a repo, pass `dir=xxx`
    nixpkgs.url = "github:foo/bar?dir=shu";
  }
}

Also see the nix flake manual.

The bar input is then passed to the output schema

Output schema

Once the inputs are resolved, they're passed to the function `outputs` along with with `self`, which is the directory of this flake in the store. `outputs` returns the outputs of the flake, according to the following schema.

This is described in the nix package manager src/nix/flake.cc in CmdFlakeCheck.

Where:

<system> is something like "x86_64-linux", "aarch64-linux", "i686-linux", "x86_64-darwin"
<name> is an attribute name like "hello".
<flake> is a flake name like "nixpkgs".
<store-path> is a /nix/store.. path

{ self, ... }@inputs:
{
  # Executed by `nix flake check`
  checks."<system>"."<name>" = derivation;
  # Executed by `nix build .#<name>`
  packages."<system>"."<name>" = derivation;
  # Executed by `nix build .`
  packages."<system>".default = derivation;
  # Executed by `nix run .#<name>`
  apps."<system>"."<name>" = {
    type = "app";
    program = "<store-path>";
  };
  # Executed by `nix run . -- <args?>`
  apps."<system>".default = { type = "app"; program = "..."; };

  # Formatter (alejandra, nixfmt or nixpkgs-fmt)
  formatter."<system>" = derivation;
  # Used for nixpkgs packages, also accessible via `nix build .#<name>`
  legacyPackages."<system>"."<name>" = derivation;
  # Overlay, consumed by other flakes
  overlays."<name>" = final: prev: { };
  # Default overlay
  overlays.default = final: prev: { };
  # Nixos module, consumed by other flakes
  nixosModules."<name>" = { config }: { options = {}; config = {}; };
  # Default module
  nixosModules.default = { config }: { options = {}; config = {}; };
  # Used with `nixos-rebuild --flake .#<hostname>`
  # nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
  nixosConfigurations."<hostname>" = {};
  # Used by `nix develop .#<name>`
  devShells."<system>"."<name>" = derivation;
  # Used by `nix develop`
  devShells."<system>".default = derivation;
  # Hydra build jobs
  hydraJobs."<attr>"."<system>" = derivation;
  # Used by `nix flake init -t <flake>#<name>`
  templates."<name>" = {
    path = "<store-path>";
    description = "template description goes here?";
  };
  # Used by `nix flake init -t <flake>`
  templates.default = { path = "<store-path>"; description = ""; };
}

You can also define additional arbitrary attributes, but these are the outputs that Nix knows about.

nix run

When output apps.<system>.myapp is not defined, nix run myapp runs <packages or legacyPackages.<system>.myapp>/bin/<myapp.meta.mainProgram or myapp.pname or myapp.name (the non-version part)>

Using flakes with stable Nix

There exists the flake-compat library that you can use to shim default.nix and shell.nix files. It will download the inputs of the flake, pass them to the flake’s outputs function and return an attribute set containing defaultNix and shellNix attributes. The attributes will contain the output attribute set with an extra default attribute pointing to current platform’s defaultPackage (resp. devShell for shellNix).

Place the following into default.nix (for shell.nix, replace defaultNix with shellNix) to use the shim:

(import (
  fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/12c64ca55c1014cdc1b16ed5a804aa8576601ff2.tar.gz";
    sha256 = "0jm6nzb83wa6ai17ly9fzpqc40wg1viib8klq8lby54agpl213w5"; }
) {
  src =  ./.;
}).defaultNix

You can also use the lockfile to make updating the hashes easier using nix flake lock --update-input flake-compat. Add the following to your flake.nix:

  inputs.flake-compat = {
    url = "github:edolstra/flake-compat";
    flake = false;
  };

and add flake-compat to the arguments of outputs attribute. Then you will be able to use default.nix like the following:

(import (
  let
    lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  in fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
    sha256 = lock.nodes.flake-compat.locked.narHash; }
) {
  src =  ./.;
}).defaultNix

Accessing flakes from Nix expressions

If you want to access a flake from within a regular Nix expression on a system that has flakes enabled, you can use something like (builtins.getFlake "path:/path/to/directory").packages.x86_64-linux.default, where 'directory' is the directory that contains your flake.nix.

Making your evaluations pure

Nix flakes run in pure evaluation mode, which is underdocumented. Some tips for now:

fetchurl and fetchtar require a sha256 argument to be considered pure.
builtins.currentSystem is non-hermetic and impure. This can usually be avoided by passing the system (i.e., x86_64-linux) explicitly to derivations requiring it.
Imports from channels like <nixpkgs> can be made pure by instead importing from the output function in flake.nix, where the arguments provide the store path to the flake's inputs:

 outputs = { self, nixpkgs, ... }:
  {
    nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
      # Note that you cannot put arbitrary configuration here: the configuration must be placed in the files loaded via modules
      system = "x86_64-linux";
      modules = [
        (nixpkgs + "/nixos/modules/<some-module>.nix")
        ./machine.nix
      ];
    };
  };

The nix flakes command

The nix flake subcommand is described in command reference page of the unstable manual.

Install packages with `nix profile`

You can install Nix packages into your local profile from any flakes with `nix profile install. For example from `nixpkgs`` flake :

nix profile install nixpkgs#hello

On NixOS you can syncronize your system and your profile references to `nixpkgs` with:

nix registry pin nixpkgs github:NixOS/nixpkgs/$(nixos-version --revision)

Using nix flakes with NixOS

nixos-rebuild switch will read its configuration from /etc/nixos/flake.nix if it is present.

A basic nixos flake.nix could look like this:

{
  outputs = { self, nixpkgs }: {
    # replace 'joes-desktop' with your hostname here.
    nixosConfigurations.joes-desktop = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [ ./configuration.nix ];
    };
  };
}

If you want to pass on the flake inputs to external configuration files, you can use the specialArgs attribute:

{
  inputs.nixpkgs.url = github:NixOS/nixpkgs;
  inputs.home-manager.url = github:nix-community/home-manager;

  outputs = { self, nixpkgs, ... }@attrs: {
    nixosConfigurations.fnord = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      specialArgs = attrs;
      modules = [ ./configuration.nix ];
    };
  };
}

Then, you can access the flake inputs from the file configuration.nix like this:

{ config, lib, nixpkgs, home-manager, ... }: {
  # do something with home-manager here, for instance:
  imports = [ home-manager.nixosModules.default ];
  ...
}

nixos-rebuild also allows to specify different flake using the --flake flag (# is optional):

$ sudo nixos-rebuild switch --flake '.#'

By default nixos-rebuild will use the currents system hostname to lookup the right nixos configuration in nixosConfigurations. You can also override this by using appending it to the flake parameter:

$ sudo nixos-rebuild switch --flake '/etc/nixos#joes-desktop'

To switch a remote configuration, use:

$ nixos-rebuild --flake .#mymachine \
  --target-host mymachine-hostname --build-host localhost \
  switch

Warning: Remote building seems to be broken at the moment, which is why the build host is set to “localhost”.

Super fast nix-shell

A feature of the nix Flake edition is that Nix evaluations are cached.

Let’s say that your project has a shell.nix file that looks like this:

{ pkgs ? import <nixpkgs> { } }:
with pkgs;
mkShell {
  buildInputs = [
    nixpkgs-fmt
  ];

  shellHook = ''
    # ...
  '';
}

Running nix-shell can be a bit slow and take 1-3 seconds.

Now create a flake.nix file in the same repository:

{
  description = "my project description";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let pkgs = nixpkgs.legacyPackages.${system}; in
        {
          devShells.default = import ./shell.nix { inherit pkgs; };
        }
      );
}

Run git add flake.nix so that Nix recognizes it.

And finally, run nix develop. This is what replaces the old nix-shell invocation.

Exit and run again, this command should now be super fast.

Warning: TODO: there is an alternative version where the defaultPackage is a pkgs.buildEnv that contains all the dependencies. And then nix shell is used to open the environment.

Direnv integration

Assuming that the flake defines a devShell output attribute and that you are using direnv. Here is how to replace the old use nix stdlib function with the faster flake version:

use_flake() {
  watch_file flake.nix
  watch_file flake.lock
  eval "$(nix print-dev-env --profile "$(direnv_layout_dir)/flake-profile")"
}

Copy this in ~/.config/direnv/lib/use_flake.sh or in ~/.config/direnv/direnvrc or directly in your project specific .envrc.

Note: You may not need to create use_flake() yourself; as of direnv 2.29, use flake is part of direnv's standard library.

With this in place, you can now replace the use nix invocation in the .envrc file with use flake:

# .envrc
use flake

The nice thing about this approach is that evaluation is cached.

Optimize the reloads

Nix Flakes has a Nix evaluation caching mechanism. Is it possible to expose that somehow to automatically trigger direnv reloads?

With the previous solution, direnv would only reload if the flake.nix or flake.lock files have changed. This is not completely precise as the flake.nix file might import other files in the repository.

Setting the bash prompt like nix-shell

A new experimental feature of flakes allow to setup a bash-prompt per flake:

{
  description = "...";
  nixConfig.bash-prompt = "\[nix-develop\]$ ";
  ...
}

Otherwise it's also possible to set the nix develop bash prompt system wide using the nix.conf option bash-prompt. (On nixos I think it is set in nix.extraOptions)

Pushing Flakes to Cachix

https://docs.cachix.org/pushing#flakes

To push all flake outputs automatically, use devour-flake.

Build specific attributes in a flake repository

When in the repository top-level, run nix build .#<attr>. It will look in the legacyPackages and packages output attributes for the corresponding derivation.

Eg, in nixpkgs:

$ nix build .#hello

Building flakes from a Git repo url with submodules

As per nix 2.9.1, git submodules in package srcs won't get copied to the nix store, this may cause the build to fail. To workaround this, use:

$ nix build .?submodules=1#hello

See: https://github.com/NixOS/nix/pull/5434

Importing packages from multiple channels

A NixOS config flake skeleton could be as follows:

{
  description = "NixOS configuration with two or more channels";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-21.11";
    nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs, nixpkgs-unstable }:
    let
      system = "x86_64-linux";
      overlay-unstable = final: prev: {
        unstable = nixpkgs-unstable.legacyPackages.${prev.system};
        # use this variant if unfree packages are needed:
        # unstable = import nixpkgs-unstable {
        #   inherit system;
        #   config.allowUnfree = true;
        # };

      };
    in {
      nixosConfigurations."<hostname>" = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          # Overlays-module makes "pkgs.unstable" available in configuration.nix
          ({ config, pkgs, ... }: { nixpkgs.overlays = [ overlay-unstable ]; })
          ./configuration.nix
        ];
      };
    };
}

# NixOS configuration.nix, can now use "pkgs.package" or "pkgs.unstable.package"
{ config, pkgs, ... }: {
  environment.systemPackages = [pkgs.firefox pkgs.unstable.chromium];
  # ...
}

Same can be done with the NURs, as it already has an overlay attribute in the flake.nix of the project, you can just add

nixpkgs.overlays = [ nur.overlay ];

If the variable nixpkgs points to the flake, you can also define pkgs with overlays with:

pkgs = import nixpkgs { overlays = [ /*the overlay in question*/ ]; };

Getting Instant System Flakes Repl

How to get a nix repl out of your system flake:

# nix repl
>> :lf /etc/nixos
>> nixosConfigurations.myhost.config
{ ... }

Or out of your current flake:

# nix repl
>> :lf .#

You can then access to the inputs, outputs… For instance if you would like to check the default version of the kernel present in nixpgs:

nix-repl> inputs.nixpkgs.legacyPackages.x86_64-linux.linuxPackages.kernel.version
"5.15.74"

However, this won't be instant upon evaluation if any file changes have been done since your last configuration rebuild. Instead, if one puts:

nix.nixPath = let path = toString ./.; in [ "repl=${path}/repl.nix" "nixpkgs=${inputs.nixpkgs}" ];

In their system flake.nix configuration file, and includes the following file in their root directory flake as repl.nix:

let
  flake = builtins.getFlake (toString ./.);
  nixpkgs = import <nixpkgs> { };
in
{ inherit flake; }
// flake
// builtins
// nixpkgs
// nixpkgs.lib
// flake.nixosConfigurations

(Don't forget to git add repl.nix && nixos-rebuild switch --flake "/etc/nixos") Then one can run (or bind a shell alias):

source /etc/set-environment && nix repl $(echo $NIX_PATH | perl -pe 's|.*(/nix/store/.*-source/repl.nix).*|\1|')

This will launch a repl with access to nixpkgs, lib, and the flake options in a split of a second.

An alternative approach to the above shell alias is omitting repl from nix.nixPath and creating a shell script:

nix.nixPath = [ "nixpkgs=${inputs.nixpkgs}" ];
environment.systemPackages = let
  repl_path = toString ./.;
  my-nix-fast-repl = pkgs.writeShellScriptBin "my-nix-fast-repl" ''
    source /etc/set-environment
    nix repl "${repl_path}/repl.nix" "$@"
  '';
in [
  my-nix-fast-repl
];

Enable unfree software

Refer to Unfree Software.

Development tricks

How to add a file locally in git but not include it in commits

When a git folder exists, flake will only copy files added in git to maximize reproducibility (this way if you forgot to add a local file in your repo, you will directly get an error when you try to compile it). However, for development purpose you may want to create an alternative flake file, for instance containing configuration for your preferred editors as described here… of course without committing this file since it contains only your own preferred tools. You can do so by doing something like that (say for a file called extra/flake.nix):

git add --intent-to-add extra/flake.nix
git update-index --skip-worktree extra/flake.nix

Rapid iteration of a direct dependency

One common pain point with using Nix as a development environment is the need to completely rebuild dependencies and re-enter the dev shell every time they are updated. The nix develop --redirect <flake> <directory> command allows you to provide a mutable dependency to your shell as if it were built by Nix.

Consider a situation where your executable, consumexe, depends on a library, libdep. You're trying to work on both at the same time, where changes to libdep are reflected in real time for consumexe. This workflow can be achieved like so:

cd ~/libdep-src-checkout/
nix develop # Or `nix-shell` if applicable.
export prefix="./install" # configure nix to install it here
buildPhase   # build it like nix does
installPhase # install it like nix does

Now that you've built the dependency, consumexe can take it as an input. In another terminal:

cd ~/consumexe-src-checkout/
nix develop --redirect libdep ~/libdep-src-checkout/install
echo $buildInputs | tr " " "\n" | grep libdep
# Output should show ~/libdep-src-checkout/ so you know it worked

If Nix warns you that your redirected flake isn't actually used as an input to the evaluated flake, try using the --inputs-from . flag. If all worked well you should be able to buildPhase && installPhase when the dependency changes and rebuild your consumer with the new version without exiting the development shell.