Flakes: Difference between revisions

Revision as of 22:23, 20 March 2023

Nix Flakes are an upcoming feature of the Nix package manager.

Introduction

Flakes allow you to specify your code's dependencies (e.g. remote Git repositories) in a declarative way, simply by listing them inside a flake.nix file:

{
  inputs = {
    home-manager.url = "github:nix-community/home-manager";
  };
}

Each dependency gets pinned, that is: its commit hash gets automatically stored into a file - named flake.lock - making it easy to, say, upgrade it:

$ nix flake lock --update-input home-manager

(if you're familiar with modern packages managers like cargo or npm, then the overall mechanism shouldn't surprise you - Nix works in a similar way, although without a centralized repository.)

Flakes replace the nix-channels command and things like ad-hoc invocations of builtins.fetchgit - no more worrying about keeping your channels in sync, no more worrying about forgetting about a dependency deep down in your tree: everything's at hand right inside flake.lock.

Enable flakes

Flakes have been introduced with Nix 2.4 (release notes).

See Nix package versions to find a channel or nixpkgs revision providing the required Nix version.

Temporary

Add --experimental-features 'nix-command flakes' when calling the nix command.

Permanent

Edit either ~/.config/nix/nix.conf or /etc/nix/nix.conf and add:

experimental-features = nix-command flakes

Here's a handy copy-paste:

mkdir -p ~/.config/nix
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf

If the Nix installation is in multi-user mode, don’t forget to restart the nix-daemon.

NixOS

On NixOS set the following options in configuration.nix and run nixos-rebuild.

{ pkgs, ... }: {
  nix.settings.experimental-features = [ "nix-command" "flakes" ];
}

Basic project usage

Warning: All files tracked by the version control system (e.g. git or mercurial) will be copied to the nix store when the flake is evaluated. So be careful when putting secrets in version control (which is not optimal by itself) around a flake.

In your repo, run nix flake init to generate the flake.nix file. Then run git add flake.nix to add it to the git staging area, otherwise nix will not recognize that the file exists.

To install a flake when using nix outside of nixOS, use nix profile install /path/to/flake. The path may also be an URL (e.g.: nix profile install git+https://example.com/my-repo?dir=subdirectory).

Flake schema

The flake.nix file is a Nix file but that has special restrictions (more on that later).

It has 4 top-level attributes:

description is a string describing the flake.
inputs is an attribute set of all the dependencies of the flake. The schema is described below.
outputs is a function of one argument that takes an attribute set of all the realized inputs, and outputs another attribute set which schema is described below.
nixConfig is an attribute set of values which reflect the values given to nix.conf. This can extend the normal behavior of a user's nix experience by adding flake-specific configuration, such as a binary cache.

Input schema

This is not a complete schema but should be enough to get you started:

{
  inputs = {
    # GitHub example, also supports GitLab:
    nixpkgs.url = "github:Mic92/nixpkgs/master";
    # Git urls
    git-example.url = "git+https://git.somehost.tld/user/path?ref=branch&rev=fdc8ef970de2b4634e1b3dca296e1ed918459a9e";
    # Local directories (for absolute paths you can omit 'path:')
    directory-example.url = "path:/path/to/repo";
    # The above url will also copy .git, use this for (shallow) local Git repos
    git-directory-example.url = "git+path:/path/to/repo?shallow=true"
    # Use this for non-flakes
    bar.url = "github:foo/bar/branch";
    bar.flake = false;
    # Overwrite inputs in a flake
    # This is useful to use the same nixpkgs version in both flakes
    sops-nix.url = "github:Mic92/sops-nix";
    sops-nix.inputs.nixpkgs.follows = "nixpkgs";
    # Pin flakes to a specific revision
    nix-doom-emacs.url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3";
    nix-doom-emacs.flake = false;
    # To use a subdirectory of a repo, pass dir=
    nixpkgs.url = "github:foo/bar?dir=shu";
  }
}

Also see the nix flake manual.

The bar input is then passed to the output schema

Output schema

This is described in the nix package manager src/nix/flake.cc in CmdFlakeCheck.

Where:

<system> is something like "x86_64-linux", "aarch64-linux", "i686-linux", "x86_64-darwin"
<attr> is an attribute name like "hello".
<flake> is a flake name like "nixpkgs".
<store-path> is a /nix/store.. path

{ self, ... }@inputs:
{
  # Executed by `nix flake check`
  checks."<system>"."<name>" = derivation;
  # Executed by `nix build .#<name>`
  packages."<system>"."<name>" = derivation;
  # Executed by `nix build .`
  packages."<system>".default = derivation;
  # Executed by `nix run .#<name>`
  apps."<system>"."<name>" = {
    type = "app";
    program = "<store-path>";
  };
  # Executed by `nix run . -- <args?>`
  apps."<system>".default = { type = "app"; program = "..."; };

  # Formatter (alejandra, nixfmt or nixpkgs-fmt)
  formatter."<system>" = derivation;
  # Used for nixpkgs packages, also accessible via `nix build .#<name>`
  legacyPackages."<system>"."<name>" = derivation;
  # Overlay, consumed by other flakes
  overlays."<name>" = final: prev: { };
  # Default overlay
  overlays.default = {};
  # Nixos module, consumed by other flakes
  nixosModules."<name>" = { config }: { options = {}; config = {}; };
  # Default module
  nixosModules.default = {};
  # Used with `nixos-rebuild --flake .#<hostname>`
  # nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
  nixosConfigurations."<hostname>" = {};
  # Used by `nix develop .#<name>`
  devShells."<system>"."<name>" = derivation;
  # Used by `nix develop`
  devShells."<system>".default = derivation;
  # Hydra build jobs
  hydraJobs."<attr>"."<system>" = derivation;
  # Used by `nix flake init -t <flake>#<name>`
  templates."<name>" = {
    path = "<store-path>";
    description = "template description goes here?";
  };
  # Used by `nix flake init -t <flake>`
  templates.default = { path = "<store-path>"; description = ""; };
}

You can also define additional arbitrary attributes, but these are the outputs that Nix knows about.

nix run

When output apps.<system>.myapp is not defined, nix run myapp runs <packages or legacyPackages.<system>.myapp>/bin/<myapp.meta.mainProgram or myapp.pname or myapp.name (the non-version part)>

Using flakes project from a legacy Nix

If you want to use a flake from a 'regular' derivation on a system that does have flakes enabled, you can use something like (builtins.getFlake "path:/path/to/directory").packages.x86_64-linux.default, where 'directory' is the directory that contains your flake.nix.

There is also a flake-compat library you can use to shim legacy default.nix and shell.nix files. It will download the inputs of the flake, pass them to the flake’s outputs function and return an attribute set containing defaultNix and shellNix attributes. The attributes will contain the output attribute set with an extra default attribute pointing to current platform’s defaultPackage (resp. devShell for shellNix).

Place the following into default.nix (for shell.nix, replace defaultNix with shellNix) to use the shim:

(import (
  fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/12c64ca55c1014cdc1b16ed5a804aa8576601ff2.tar.gz";
    sha256 = "0jm6nzb83wa6ai17ly9fzpqc40wg1viib8klq8lby54agpl213w5"; }
) {
  src =  ./.;
}).defaultNix

You can also use the lockfile to make updating the hashes easier using nix flake lock --update-input flake-compat. Add the following to your flake.nix:

  inputs.flake-compat = {
    url = "github:edolstra/flake-compat";
    flake = false;
  };

and add flake-compat to the arguments of outputs attribute. Then you will be able to use default.nix like the following:

(import (
  let
    lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  in fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
    sha256 = lock.nodes.flake-compat.locked.narHash; }
) {
  src =  ./.;
}).defaultNix

Making your evaluations pure

Nix flakes run in pure evaluation mode, which is underdocumented. Some tips for now:

fetchurl and fetchtar require a sha256 argument to be considered pure.
builtins.currentSystem is non-hermetic and impure. This can usually be avoided by passing the system (i.e., x86_64-linux) explicitly to derivations requiring it.
Imports from channels like <nixpkgs> can be made pure by instead importing from the output function in flake.nix, where the arguments provide the store path to the flake's inputs:

 outputs = { self, nixpkgs, ... }:
  {
    nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
      # Note that you cannot put arbitrary configuration here: the configuration must be placed in the files loaded via modules
      system = "x86_64-linux";
      modules = [
        (nixpkgs + "/nixos/modules/<some-module>.nix")
        ./machine.nix
      ];
    };
  };

The nix flakes command

The nix flake subcommand is described in command reference page of the unstable manual.

Using nix flakes with NixOS

nixos-rebuild switch will read its configuration from /etc/nixos/flake.nix if it is present.

A basic nixos flake.nix could look like this:

{
  outputs = { self, nixpkgs }: {
    # replace 'joes-desktop' with your hostname here.
    nixosConfigurations.joes-desktop = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [ ./configuration.nix ];
    };
  };
}

If you want to pass on the flake inputs to external configuration files, you can use the specialArgs attribute:

{
  inputs.nixpkgs.url = github:NixOS/nixpkgs;
  inputs.home-manager.url = github:nix-community/home-manager;
  
  outputs = { self, nixpkgs, ... }@attrs: {
    nixosConfigurations.fnord = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      specialArgs = attrs;
      modules = [ ./configuration.nix ];
    };
  };
}

Then, you can access the flake inputs from the file configuration.nix like this:

{ config, lib, nixpkgs, home-manager, ... }: {
  # do something with home-manager here, for instance:
  imports = [ home-manager.nixosModule ];
  ...
}

nixos-rebuild also allows to specify different flake using the --flake flag (# is optional):

$ sudo nixos-rebuild switch --flake '.#'

By default nixos-rebuild will use the currents system hostname to lookup the right nixos configuration in nixosConfigurations. You can also override this by using appending it to the flake parameter:

$ sudo nixos-rebuild switch --flake '/etc/nixos#joes-desktop'

To switch a remote configuration, use:

$ nixos-rebuild --flake .#mymachine \
  --target-host mymachine-hostname --build-host localhost \
  switch

Warning: Remote building seems to be broken at the moment, which is why the build host is set to “localhost”.

Super fast nix-shell

One of the nix feature of the Flake edition is that Nix evaluations are cached.

Let’s say that your project has a shell.nix file that looks like this:

{ pkgs ? import <nixpkgs> { } }:
with pkgs;
mkShell {
  buildInputs = [
    nixpkgs-fmt
  ];

  shellHook = ''
    # ...
  '';
}

Running nix-shell can be a bit slow and take 1-3 seconds.

Now create a flake.nix file in the same repository:

{
  description = "my project description";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let pkgs = nixpkgs.legacyPackages.${system}; in
        {
          devShells.default = import ./shell.nix { inherit pkgs; };
        }
      );
}

Run git add flake.nix so that Nix recognizes it.

And finally, run nix develop. This is what replaces the old nix-shell invocation.

Exit and run again, this command should now be super fast.

Warning: TODO: there is an alternative version where the defaultPackage is a pkgs.buildEnv that contains all the dependencies. And then nix shell is used to open the environment.

Direnv integration

Assuming that the flake defines a devShell output attribute and that you are using direnv. Here is how to replace the old use nix stdlib function with the faster flake version:

use_flake() {
  watch_file flake.nix
  watch_file flake.lock
  eval "$(nix print-dev-env --profile "$(direnv_layout_dir)/flake-profile")"
}

Copy this in ~/.config/direnv/lib/use_flake.sh or in ~/.config/direnv/direnvrc or directly in your project specific .envrc.

Note: You may not need to create use_flake() yourself; as of direnv 2.29, use flake is part of direnv's standard library.

With this in place, you can now replace the use nix invocation in the .envrc file with use flake:

# .envrc
use flake

The nice thing about this approach is that evaluation is cached.

Optimize the reloads

Nix Flakes has a Nix evaluation caching mechanism. Is it possible to expose that somehow to automatically trigger direnv reloads?

With the previous solution, direnv would only reload if the flake.nix or flake.lock files have changed. This is not completely precise as the flake.nix file might import other files in the repository.

Setting the bash prompt like nix-shell

A new experimental feature of flakes allow to setup a bash-prompt per flake:

{
  description = "...";
  nixConfig.bash-prompt = "\[nix-develop\]$ ";
  ...
}

Otherwise it's also possible to set the nix develop bash prompt system wide using the nix.conf option bash-prompt. (On nixos I think it is set in nix.extraOptions)

Pushing Flakes to Cachix

https://docs.cachix.org/pushing#flakes

Build specific attributes in a flake repository

When in the repository top-level, run nix build .#<attr>. It will look in the legacyPackages and packages output attributes for the corresponding derivation.

Eg, in nixpkgs:

$ nix build .#hello

Building flakes from a Git repo url with submodules

As per nix 2.9.1, git submodules in package srcs won't get copied to the nix store, this may cause the build to fail. To workaround this, use:

$ nix build .?submodules=1#hello

See: https://github.com/NixOS/nix/pull/5434

Importing packages from multiple channels

A NixOS config flake skeleton could be as follows:

{
  description = "NixOS configuration with two or more channels";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-21.11"; 
    nixpkgs-unstable.url = "nixpkgs/nixos-unstable"; 
  };

  outputs = { self, nixpkgs, nixpkgs-unstable }:
    let
      system = "x86_64-linux";
      overlay-unstable = final: prev: {
        unstable = nixpkgs-unstable.legacyPackages.${prev.system};
        # use this variant if unfree packages are needed:
        # unstable = import nixpkgs-unstable {
        #   inherit system;
        #   config.allowUnfree = true;
        # };

      };
    in {
      nixosConfigurations."<hostname>" = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          # Overlays-module makes "pkgs.unstable" available in configuration.nix
          ({ config, pkgs, ... }: { nixpkgs.overlays = [ overlay-unstable ]; })
          ./configuration.nix
        ];
      };
    };
}

# NixOS configuration.nix, can now use "pkgs.package" or "pkgs.unstable.package"
{ config, pkgs, ... }: {
  environment.systemPackages = [pkgs.firefox pkgs.unstable.chromium];
  # ...
}

Same can be done with the NURs, as it already has an overlay attribute in the flake.nix of the project, you can just add

nixpkgs.overlays = [ nur.overlay ];

If the variable nixpkgs points to the flake, you can also define pkgs with overlays with:

pkgs = import nixpkgs { overlays = [ /*the overlay in question*/ ]; };

Getting Instant System Flakes Repl

How to get a nix repl out of your system flake:

# nix repl 
>> :lf /etc/nixos
>> nixosConfigurations.myhost.config
{ ... }

Or out of your current flake:

# nix repl 
>> :lf .#

You can then access to the inputs, outputs… For instance if you would like to check the default version of the kernel present in nixpgs:

nix-repl> inputs.nixpkgs.legacyPackages.x86_64-linux.linuxPackages.kernel.version        
"5.15.74"

However, this won't be instant upon evaluation if any file changes have been done since your last configuration rebuild. Instead, if one puts:

nix.nixPath = let path = toString ./.; in [ "repl=${path}/repl.nix" "nixpkgs=${inputs.nixpkgs}" ];

In their system flake.nix configuration file, and includes the following file in their root directory flake as repl.nix:

let
  flake = builtins.getFlake (toString ./.);
  nixpkgs = import <nixpkgs> { };
in
{ inherit flake; }
// flake
// builtins
// nixpkgs
// nixpkgs.lib
// flake.nixosConfigurations

(Don't forget to git add repl.nix && nixos-rebuild switch --flake "/etc/nixos") Then one can run (or bind a shell alias):

source /etc/set-environment && nix repl $(echo $NIX_PATH | perl -pe 's|.*(/nix/store/.*-source/repl.nix).*|\1|')

This will launch a repl with access to nixpkgs, lib, and the flake options in a split of a second.

An alternative approach to the above shell alias is omitting repl from nix.nixPath and creating a shell script:

nix.nixPath = [ "nixpkgs=${inputs.nixpkgs}" ];
environment.systemPackages = let
  repl_path = toString ./.;
  my-nix-fast-repl = pkgs.writeShellScriptBin "my-nix-fast-repl" ''
    source /etc/set-environment
    nix repl "${repl_path}/repl.nix" "$@"
  '';
in [
  my-nix-fast-repl
];

Enable unfree software

Because flake evalutations are hermetic, they will ignore the system configuration on nonfree software and the NIXPKGS_ALLOW_UNFREE environment variable by default.

To use nonfree software with CLI tools like nix shell or nix run, the --impure flag must be used for Nixpkgs to access the current environment variables:

$ NIXPKGS_ALLOW_UNFREE=1 nix run --impure nixpkgs#discord

To use nonfree software in a flake, add nixpkgs as an input in your flake and import it with the allowUnfree option:

pkgs = import nixpkgs { config = { allowUnfree = true; }; };

Enable unfree software in home-manager

If you want to install software using home-manager via nix flakes in non NixOS systems (like darwin) you can use the home-manager nixpkgs.config option for example

nixpkgs.config.allowUnfree = true;

Development tricks

How to add a file locally in git but not include it in commits

When a git folder exists, flake will only copy files added in git to maximize reproducibility (this way if you forgot to add a local file in your repo, you will directly get an error when you try to compile it). However, for development purpose you may want to create an alternative flake file, for instance containing configuration for your preferred editors as described here… of course without committing this file since it contains only your own preferred tools. You can do so by doing something like that (say for a file called extra/flake.nix):

git add --intent-to-add extra/flake.nix
git update-index --assume-unchanged extra/flake.nix

Rapid iteration of a direct dependency

One common pain point with using Nix as a development environment is the need to completely rebuild dependencies and re-enter the dev shell every time they are updated. The nix develop --redirect <flake> <directory> command allows you to provide a mutable dependency to your shell as if it were built by Nix.

Consider a situation where your executable, consumexe, depends on a library, libdep. You're trying to work on both at the same time, where changes to libdep are reflected in real time for consumexe. This workflow can be achieved like so:

cd ~/libdep-src-checkout/
nix develop # Or `nix-shell` if applicable.
export prefix="./install" # configure nix to install it here
buildPhase   # build it like nix does
installPhase # install it like nix does

Now that you've built the dependency, consumexe can take it as an input. In another terminal:

cd ~/consumexe-src-checkout/
nix develop --redirect libdep ~/libdep-src-checkout/install
echo $buildInputs | tr " " "\n" | grep libdep
# Output should show ~/libdep-src-checkout/ so you know it worked

If Nix warns you that your redirected flake isn't actually used as an input to the evaluated flake, try using the --inputs-from . flag. If all worked well you should be able to buildPhase && installPhase when the dependency changes and rebuild your consumer with the new version without exiting the development shell.