Low-level derivations: Difference between revisions

(15 intermediate revisions by the same user not shown)

Line 1:

{{external|The Nix Reference Manual|§ 5.4.1. {{manual|nix|language/derivations|Derivations}}}}

Writing '''low-level derivations''' without the convenience of the [[~~Standard~~ environment]] requires understanding the core fundamentals of how [[Nix]] works. While writing derivations in this manner is not common, they can be useful when no utility present in the standard environment matches the required use case, as well as being an essential learning tool for understanding why <code>stdenv.mkDerivation</code> is needed in most cases.

Writing '''low-level derivations''' without the convenience of the [[standard environment]] requires understanding the core fundamentals of how [[Nix]] works. While writing [[derivations]] in this manner is not common, they can be useful when no utility present in the standard environment matches the required use case, as well as being an essential learning tool for understanding why <code>stdenv.mkDerivation</code> is needed in most cases.

In the section below, we'll start by writing a simple derivation that creates a script which displays the message ''"Hello, world"'' when run. While developing the Nix file, we'll be encountering the same challenges that led the Nix community to develop more ergonomic tooling. Each obstacle we hit - from missing system dependencies to path management - illustrates a core principle of Nix's purely functional build model. By the end, you'll understand both what makes Nix powerful and why most derivations use abstraction layers.

Line 8:

{{File|example.sh|bash|#!/bin/bash

echo "Hello, world!"}}

When running this script (e.g. <code>$ ./example.sh</code>) we'll get exactly the output we want. However, we're trying to build the script using Nix. Our first approach might be to write a simple derivation similar to:

When running this script (e.g. <code>$ ./example.sh</code>) we'll get exactly the output we want. However, we're trying to build the script using Nix. Our first approach might be to write a simple derivation{{cite manual|nix|language/derivations|number=5.4.1|title=Derivations}} similar to:

{{File|example.nix|nix|<nowiki>derivation {

name = "hello-world";

Line 48:

{{File|example.nix|nix|highlight=1-3,7,11|<nowiki>

let

pkgs = import '</nowiki><<nowiki>nixpkgs</nowiki>><nowiki>'

pkgs = import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> { };

in

derivation {

Line 66:

Building the package again, we'll come across another surprising error:

{{code|~~lines~~=no|lang=console|highlight=8|

{{code|line=no|lang=console|highlight=8|

$ nix-build example.nix

this derivation will be built:

Line 81:

Turns out <code>chmod</code> is not part of our build environment either; that's how minimal the low-level derivation environment is! However, before we fix the script in a similar manner, let's observe another important detail: the store path. Compare the build location of this example, and the previous one; the SHAs are different! That is because our derivation has different inputs to the one before, therefore it's built under a different location in the Nix store. And to highlight this, without fixing the script, let's build it again:

{{code|~~lines~~=no|lang=console|highlight=3|

{{code|line=no|lang=console|highlight=3|

$ nix-build example.nix

this derivation will be built:

Line 95:

The SHA is the same, because our inputs haven't changed. This is the essence to why derivations are ''pure functions'' of their inputs.

In rewriting our derivation, we need to import <code>chmod</code>, which is part of the <code>coreutils</code> package. We could point to the binary similarly to how we did it before, but for variation, let's modify our <code>PATH</code> variable instead:

{{File|example.nix|nix|highlight=11|<nowiki>

let

pkgs = import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> { };

in

derivation {

name = "hello-world";

system = builtins.currentSystem;

builder = "${pkgs.bash}/bin/bash";

args = [

"-c"

''

export PATH="$PATH:${pkgs.coreutils}/bin"

echo '#!${pkgs.bash}/bin/bash' </nowiki>><nowiki> $out

echo 'echo "Hello, World!"' </nowiki>>><nowiki> $out

chmod +x $out

''

];

}</nowiki>}}

Let's rebuild it again, noticing how the SHA will change again, as we changed our derivation's inputs:

{{code|lang=console|line=no|highlight=3|

$ nix-build example.nix

this derivation will be built:

/nix/store/ipjigfvbj371a5g966xbnj790sjvq5hp-hello-world.drv

building '/nix/store/ipjigfvbj371a5g966xbnj790sjvq5hp-hello-world.drv'...

/nix/store/lsm49a32nmc3ihi3f119p17lvwadr5ms-hello-world

}}

This time it built successfully. Finally, let's run our script:

{{code|lang=console|line=no|highlight=3|

$ ./result

Hello, world!

}}

As mentioned above, building derivations this way can be unwieldy. But working through this example should shed some light onto why the community has built the standard environment, providing utility functions to build derivations for the most common use cases.

=== Standard environment ===

For practicality purposes, let's compare this to the standard environment way of doing this. Using <code>stdenv.mkDerivation</code>{{cite manual|nixpkgs|sec-using-stdenv|title=Using stdenv}} gives us access to the common Linux utilities pre-included for us{{cite manual|nixpkgs|sec-tools-of-stdenv|title=Tools provided by stdenv}}, so we don't have to do the package importing ourselves:

{{File|example.nix|nix|highlight=3|

{ pkgs ? import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> {} }:

pkgs.stdenv.mkDerivation {

name = "hello-world";

buildCommand = ''

echo '#!/bin/bash' </nowiki>><nowiki> $out

echo 'echo "Hello, World!"' </nowiki>>><nowiki> $out

chmod +x $out

'';

}

</nowiki>

}}

=== Nixpkgs utilities ===

Even further up the chain of abstractions, [[Nixpkgs]] contains many pre-built utilities for us that handle some of the configuration involved in the standard environment as well. In our case, we can use the <code>writeShellScript</code>{{cite manual|nixpkgs|trivial-builder-writeShellScript|title=writeShellScript}}:

{{File|example.nix|nix|

{ pkgs ? import <nixpkgs> {} }:

pkgs.writeShellScript "hello-world" ''

echo "Hello, World!"

''

}}

@@ Line 1: / Line 1: @@
-{{wip|date=9 June 2025}}
+{{external|The Nix Reference Manual|§ 5.4.1. {{manual|nix|language/derivations|Derivations}}}}
-Writing '''low-level derivations''' without the convenience of the [[Standard environment]] requires understanding the core fundamentals of how [[Nix]] works. While writing derivations in this manner is not common, they can be useful when no utility present in the standard environment matches the required use case, as well as being an essential learning tool for understanding why <code>stdenv.mkDerivation</code> is needed in most cases.
+Writing '''low-level derivations''' without the convenience of the [[standard environment]] requires understanding the core fundamentals of how [[Nix]] works. While writing [[derivations]] in this manner is not common, they can be useful when no utility present in the standard environment matches the required use case, as well as being an essential learning tool for understanding why <code>stdenv.mkDerivation</code> is needed in most cases.
 In the section below, we'll start by writing a simple derivation that creates a script which displays the message ''"Hello, world"'' when run. While developing the Nix file, we'll be encountering the same challenges that led the Nix community to develop more ergonomic tooling. Each obstacle we hit - from missing system dependencies to path management - illustrates a core principle of Nix's purely functional build model. By the end, you'll understand both what makes Nix powerful and why most derivations use abstraction layers.
@@ Line 8: / Line 8: @@
 {{File|example.sh|bash|#!/bin/bash
 echo "Hello, world!"}}
-When running this script (e.g. <code>$ ./example.sh</code>) we'll get exactly the output we want. However, we're trying to build the script using Nix. Our first approach might be to write a simple derivation similar to:
+When running this script (e.g. <code>$ ./example.sh</code>) we'll get exactly the output we want. However, we're trying to build the script using Nix. Our first approach might be to write a simple derivation{{cite manual|nix|language/derivations|number=5.4.1|title=Derivations}} similar to:
 {{File|example.nix|nix|<nowiki>derivation {
    name = "hello-world";
@@ Line 48: / Line 48: @@
 {{File|example.nix|nix|highlight=1-3,7,11|<nowiki>
 let
-   pkgs = import '</nowiki><<nowiki>nixpkgs</nowiki>><nowiki>'
+   pkgs = import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> { };
 in
 derivation {
@@ Line 66: / Line 66: @@
 Building the package again, we'll come across another surprising error:
-{{code|lines=no|lang=console|highlight=8|
+{{code|line=no|lang=console|highlight=8|
 $ nix-build example.nix
 this derivation will be built:
@@ Line 81: / Line 81: @@
 Turns out <code>chmod</code> is not part of our build environment either; that's how minimal the low-level derivation environment is! However, before we fix the script in a similar manner, let's observe another important detail: the store path. Compare the build location of this example, and the previous one; the SHAs are different! That is because our derivation has different inputs to the one before, therefore it's built under a different location in the Nix store. And to highlight this, without fixing the script, let's build it again:
-{{code|lines=no|lang=console|highlight=3|
+{{code|line=no|lang=console|highlight=3|
 $ nix-build example.nix
 this derivation will be built:
@@ Line 95: / Line 95: @@
 The SHA is the same, because our inputs haven't changed. This is the essence to why derivations are ''pure functions'' of their inputs.
+In rewriting our derivation, we need to import <code>chmod</code>, which is part of the <code>coreutils</code> package. We could point to the binary similarly to how we did it before, but for variation, let's modify our <code>PATH</code> variable instead:
+{{File|example.nix|nix|highlight=11|<nowiki>
+let
+  pkgs = import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> { };
+in
+derivation {
+  name = "hello-world";
+  system = builtins.currentSystem;
+  builder = "${pkgs.bash}/bin/bash";
+  args = [
+    "-c"
+    ''
+      export PATH="$PATH:${pkgs.coreutils}/bin"
+      echo '#!${pkgs.bash}/bin/bash' </nowiki>><nowiki> $out
+      echo 'echo "Hello, World!"' </nowiki>>><nowiki> $out
+      chmod +x $out
+    ''
+  ];
+}</nowiki>}}
+Let's rebuild it again, noticing how the SHA will change again, as we changed our derivation's inputs:
+{{code|lang=console|line=no|highlight=3|
+$ nix-build example.nix
+this derivation will be built:
+  /nix/store/ipjigfvbj371a5g966xbnj790sjvq5hp-hello-world.drv
+building '/nix/store/ipjigfvbj371a5g966xbnj790sjvq5hp-hello-world.drv'...
+/nix/store/lsm49a32nmc3ihi3f119p17lvwadr5ms-hello-world
+}}
+This time it built successfully. Finally, let's run our script:
+{{code|lang=console|line=no|highlight=3|
+$ ./result
+Hello, world!
+}}
+As mentioned above, building derivations this way can be unwieldy. But working through this example should shed some light onto why the community has built the standard environment, providing utility functions to build derivations for the most common use cases.
+=== Standard environment ===
+For practicality purposes, let's compare this to the standard environment way of doing this. Using <code>stdenv.mkDerivation</code>{{cite manual|nixpkgs|sec-using-stdenv|title=Using stdenv}} gives us access to the common Linux utilities pre-included for us{{cite manual|nixpkgs|sec-tools-of-stdenv|title=Tools provided by stdenv}}, so we don't have to do the package importing ourselves:
+{{File|example.nix|nix|highlight=3|
+<nowiki>
+{ pkgs ? import </nowiki><<nowiki>nixpkgs</nowiki>><nowiki> {} }:
+pkgs.stdenv.mkDerivation {
+  name = "hello-world";
+  buildCommand = ''
+    echo '#!/bin/bash' </nowiki>><nowiki> $out
+    echo 'echo "Hello, World!"' </nowiki>>><nowiki> $out
+    chmod +x $out
+  '';
+}
+</nowiki>
+}}
+=== Nixpkgs utilities ===
+Even further up the chain of abstractions, [[Nixpkgs]] contains many pre-built utilities for us that handle some of the configuration involved in the standard environment as well. In our case, we can use the <code>writeShellScript</code>{{cite manual|nixpkgs|trivial-builder-writeShellScript|title=writeShellScript}}:
+{{File|example.nix|nix|
+{ pkgs ? import <nixpkgs> {} }:
+pkgs.writeShellScript "hello-world" ''
+  echo "Hello, World!"
+''
+}}
+{{references}}