Derivations

Derivations are the Nix ecosystem way of describing any reproducible build process. While NixOS comes with a plethora of packages, applications and options, there will inevitably come a time when you need to build an application, a library, a package, etc. that is not available off the shelf already — those are all derivations under the hood. This makes the build process reproducible and predictable; without changing the derivation's input configuration, the output will remain the same. In essence, a derivation is a pure function of an executable, and a set of input configuration, that produces exactly the same output for every invocation, in unique locations on the filesystem.

Motivation

While the need to build software, package libraries and execute build processes is clear to anyone using any operating system, the natural question that may arise is Why go out of our way dealing with this complicated process when I can just run a few terminal commands? For most distributions, the answer is they don't do things this way. Most Linux distributions, and most operating systems for that matter, are designed to change over time; the same build process will yield different results each time it's invoked. For example, remember trying to build a package twice; the first time you build it the installation will be successful, but the second time it's built you might get an error about the paths it's trying to write to already existing. Build processes in most Linux distributions are stateful, the context in which they're run might change as you're using that system.

However, the Nix ecosystem is fundamentally different in this regard; when you build a derivation, a unique path in the Nix store is assigned, and all possible outputs (including filesystem operations) produced by it will be persisted under that path. No other derivation can modify those files; the result of the derivation is uniquely determined by its input configuration, and subsequent reruns will produce exactly the same outputs, under different Nix store paths. Any potential issues regarding being able to reproduce a build process are addressed by design, if a derivation was successful once, it will always build successfully as long as its inputs don't change.

Derivations are a powerful fundamental part of Nix and provide the core platform for managing packages in NixOS. Every package and library you include in your NixOS configuration is a derivation.

Definition

A derivation is defined as a specification for running an executable on precisely defined input files to repeatably produce output files at uniquely determined file system paths. Simply put, it describes a set of steps to take some input and produce some output in a deterministic manner.

Derivations can be written manually using the derivation function; this is the most fundamental way in which they can be defined. However, since this low-level function is quite simple, building derivations this way can easily become unwieldy and repetitive. To aid in the process of creating derivations, Nixpkgs contains the standard environment (also known as the stdenv), which provides a set of utility functions for the most common derivation types (e.g. a Python package, a shell script, a Docker container, etc.)

Getting Started

Derivations take an input and produce an output through a series of steps. For most packages, the inputs refer to the source files, the steps refer to the compilation process and are called phases, and the outputs refer to finalized executable binaries written to some file/directory. This sequence of events can be well described within a standard environment.

src

This attribute points towards the inputs used within the derivation. This can be source code, but it can also be a pre-build binary. Usually depending on what the source is, a special series of steps is done to ensure its correctness in a Nix environment.

This usually is downloaded via a fetcher, but it can also be a local path:

stdenv.mkDerivation {

  src = ./relative-path/to/src;
  # OR
  src = fetchFromGitHub {
    owner = "torvalds";
    repo = "linux";
    rev = "refs/tags/v6.11";

    hash = "...";
  };
}

out

Unlike src, this is not an attribute you set but rather an environment variable which points to the finalized location of the derivation's contents. The actual directory of $out is an implementation detail abstracted away by Nix and the stdenv builder. Anything placed within $out will then be part of the final derivation. This follows an FHS-inspired like structure, where $out/bin contains binaries, $out/lib contains shared objects, $out/include contains headers, and so forth. These paths will become part of the derivation and are the resulting entries within the /nix/store.

This attribute must point to either a file or directory, even if they are empty! Failure to create this path will result in the builder failing the entire build process.

This step is handled largely by the installPhase(see #Phases for more details):

stdenv.mkDerivation {
  src = ...;

  installPhase = ''
    mkdir -p $out/bin

    install -Dm755 -t $out/bin ./my-binary
  '';
}

meta

Unlike the previous two, this attribute has no significant relevance to building and largely contains a loose set of attributes useful with the context of Nixpkgs. Therefore, if you're writing a derivation that is not intended for Nixpkgs, this entire attribute set can be safely omitted.

Attributes in here relate to the upstream source, or the finalized derivation result. Some attributes, like meta.homepage and meta.description, are used to describe and link to relevant information about the upstream source this derivation builds. Attributes like meta.platforms and meta.licenses are also semantically relevant to upstream, but these are also useful for Nixpkgs to determine whether a package can be built on a different system (and whether to allow so in evaluation!), and if a package has a suitable license that allows for re-distribution (caching in the official binary cache).

There are also some other attributes that are only relevant within the context of Nixpkgs/Nix alone, such as meta.mainProgram, which describes the binary that can be considered the "main" program. For example, the cmake derivation would have meta.mainProgram set to cmake, which would be resolved as $out/bin/cmake when needed.

A fully exhaustive documentation on all meta-attributes can be found in the Nixpkgs manual.

Phases

☶︎

This article or section needs to be expanded. Further information may be found in the related discussion page. Please consult the pedia article metapage for guidelines on contributing.

A phase can be described as a set of steps used to transform an input into an output suitable for the next phase. Each step in the stdenv builder controls a distinct part of the build process and is largely inspired from GNU Autoconf convention of ./configure, make, and make install.

Each phase is written in bash syntax and can use any program defined within the stdenv dependencies, alongside a very minimal set of packages automatically declared within the stdenv. This minimal set include the core-utils, gcc, gnumake, bash, gnuinstall, and more that are not exhaustively documented anywhere^{[citation needed]}.