Derivations: Difference between revisions
No edit summary |
|||
| (6 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
{{dock|{{external|The Nix Reference Manual|§ 5.4.1. {{manual|nix|language/derivations|Derivations}}}}{{external|The Nixpkgs Reference Manual|{{manual|nixpkgs|sec-functions-library-derivations|lib.derivations}}}}{{external|The Nixpkgs Reference Manual|{{manual|nixpkgs|sec-using-stdenv|Using stdenv}}}}}} | |||
'''Derivations''' are the [[Nix ecosystem]] way of describing any reproducible build process. While [[NixOS]] comes with a plethora of packages, applications and options, there will inevitably come a time when you need to build an application, a library, a package, etc. that is not available ''off the shelf'' already — those are all derivations ''under the hood''. This makes the build process ''reproducible'' and ''predictable''; without changing the derivation's input configuration, the output will remain the same. In essence, a derivation is a ''pure function'' of an executable, and a set of input configuration, that produces exactly the same output for every invocation, in unique locations on the filesystem. | '''Derivations''' are the [[Nix ecosystem]] way of describing any reproducible build process. While [[NixOS]] comes with a plethora of packages, applications and options, there will inevitably come a time when you need to build an application, a library, a package, etc. that is not available ''off the shelf'' already — those are all derivations ''under the hood''. This makes the build process ''reproducible'' and ''predictable''; without changing the derivation's input configuration, the output will remain the same. In essence, a derivation is a ''pure function'' of an executable, and a set of input configuration, that produces exactly the same output for every invocation, in unique locations on the filesystem. | ||
| Line 4: | Line 5: | ||
While the need to build software, package libraries and execute build processes is clear to anyone using any operating system, the natural question that may arise is ''Why go out of our way dealing with this complicated process when I can just run a few terminal commands?'' For most distributions, the answer is ''they don't do things this way''. Most Linux distributions, and most operating systems for that matter, are designed to change over time; the same build process will yield different results each time it's invoked. For example, remember trying to build a package twice; the first time you build it the installation will be successful, but the second time it's built you might get an error about the paths it's trying to write to already existing. Build processes in most Linux distributions are ''stateful'', the context in which they're run might change as you're using that system. | While the need to build software, package libraries and execute build processes is clear to anyone using any operating system, the natural question that may arise is ''Why go out of our way dealing with this complicated process when I can just run a few terminal commands?'' For most distributions, the answer is ''they don't do things this way''. Most Linux distributions, and most operating systems for that matter, are designed to change over time; the same build process will yield different results each time it's invoked. For example, remember trying to build a package twice; the first time you build it the installation will be successful, but the second time it's built you might get an error about the paths it's trying to write to already existing. Build processes in most Linux distributions are ''stateful'', the context in which they're run might change as you're using that system. | ||
However, the Nix ecosystem is ''fundamentally'' different in this regard; when you build a derivation, a unique path in the [[Nix store]] is assigned, and all possible outputs (including filesystem operations) produced by it will be persisted under that path. No other derivation can modify those files; the result of the derivation is '''uniquely determined''' by its input configuration, and subsequent reruns will produce ''exactly'' the same outputs, | However, the Nix ecosystem is ''fundamentally'' different in this regard; when you build a derivation, a unique path in the [[Nix store]] is assigned, and all possible outputs (including filesystem operations) produced by it will be persisted under that path. No other derivation can modify those files; the result of the derivation is '''uniquely determined''' by its input configuration, and subsequent reruns will produce ''exactly'' the same outputs {{foot|In actuality, if the derivation build was successful, Nix will recognise that the inputs haven't changed, and dynamically link to the already build Nix store path.}}. Any potential issues regarding being able to reproduce a build process are addressed ''by design'', if a derivation was successful once, it will always build successfully as long as its inputs don't change. | ||
Derivations are a powerful fundamental part of [[Nix]] and provide the core platform for managing packages in NixOS. Every package and library you include in your NixOS configuration is a derivation. | Derivations are a powerful fundamental part of [[Nix]] and provide the core platform for managing packages in NixOS. Every package and library you include in your NixOS configuration is a derivation. | ||
== Definition == | == Definition == | ||
A '''derivation''' is defined as ''a specification for running an executable on precisely defined input files to repeatably produce output files at uniquely determined file system paths'' | A '''derivation''' is defined as ''a specification for running an executable on precisely defined input files to repeatably produce output files at uniquely determined file system paths''{{cite manual|nix|language/derivations|5.4.1|Derivations}}. Simply put, it describes a set of steps to take some input and produce some output in a deterministic manner. | ||
Derivations can be written manually using the <code>derivation</code> function; this is the most fundamental way in which they can be defined. However, since this low-level function is quite simple, building derivations this way can easily become unwieldy and repetitive. To aid in the process of creating derivations, [[Nixpkgs]] contains [https://nixos.org/manual/nixpkgs/unstable/#chap-stdenv the standard environment] (also known as the <code>stdenv</code>), which provides a set of utility functions for the most common derivation types (e.g. a [[Python libraries|Python package]], a shell script, a [[Docker]] container, etc.) | Derivations can be written manually using the <code>derivation</code> function; this is the most fundamental way in which they can be defined. However, since this low-level function is quite simple, building derivations this way can easily become unwieldy and repetitive. To aid in the process of creating derivations, [[Nixpkgs]] contains [https://nixos.org/manual/nixpkgs/unstable/#chap-stdenv the standard environment] (also known as the <code>stdenv</code>), which provides a set of utility functions for the most common derivation types (e.g. a [[Python libraries|Python package]], a shell script, a [[Docker]] container, etc.) | ||
| Line 112: | Line 113: | ||
== Phases == | == Phases == | ||
{{ | {{Wip|date=29 June 2025|scope=section}} | ||
The vast majority of derivations follow a fairly established set of stages to their building—for example, configuration, building, installation, etc. Thus, the vast majority of derivations that use the standard environment's <code>stdenv.mkDerivation</code>, and any language-specific builders that make use of it, follow a convention of grouping the derivation steps in '''phases''', which follow a common established pattern. However, this setup can be overridden for specific packaging systems (e.g. docker, etc.); to reduce complexity, this section will start with the generic phase setup first, expanding on how these can be overridden or modified in later sections. | |||
Each phase is written in | By default, derivations follow a standard pre-established set of phases, that are meant to mirror how the vast majority of packages are built. Each phase is meant to control a distinct part of the build process, an approach largely inspired by GNU's Autoconf convention of <code>./configure</code>, <code>make</code>, and <code>make install</code>. | ||
Taking the basic definition of a derivation to heart, that of being ''a specification for supplying a set of steps to a builder'', each phase is defined to be a sequence of steps written in shell syntax. The phase scripts can use any program or package provided by the standard environment, such as <code>core-utils</code>, <code>gcc</code>, <code>gnumake</code>, <code>bash</code>, <code>gnuinstall</code>, etc. | |||
By common convention, standard environment derivations contain the following phases, executed in this sequence: | |||
# Unpack: handles the preparation of the build environment (e.g. extracting archives, touching files, etc.); | |||
# Patch: handles changes to the underlying source code (e.g. patching bugs, adapting the source code to work in a Nix environment, etc.); | |||
# Configure: handles the configuration of the build environment, for example detecting system capabilities and setting build parameters; | |||
# Build: compiles the source code into binaries, bytecode, or otherwise a distributable form of the source code; | |||
# Check: performs any tests on the compiled package, for example the package's test suite; | |||
# Install: copies the build artefacts to the output directory, handling any needed changes (e.g. directory structure reorganisation); | |||
# Fixup: process the output artefacts to work in a Nix environment (e.g.: strip binaries, override ELF paths, handle dynamic library linking, etc.); | |||
# Install Check: performs any tests on the final output, essentially acting as a integration test into the Nix environment; | |||
# Dist: creates distribution archives (rarely used). | |||
{{references|group=footnotes|heading=Notes}} | |||
{{references}} | {{references}} | ||
[[Category:Nix]] | [[Category:Nix]] | ||