Overlays: Difference between revisions

imported>Tim-uni
m replace self and super with final and prev
Separate python overlay args from nixpkgs overlay args
 
(17 intermediate revisions by 15 users not shown)
Line 1: Line 1:
<blockquote>Overlays are Nix functions which accept two arguments, conventionally called final and prev, and return a set of packages. ... Overlays are similar to other methods for customizing Nixpkgs, in particular the packageOverrides ... Indeed, packageOverrides acts as an overlay with only the prev argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute.</blockquote>
<blockquote>Overlays are Nix functions which accept two arguments, conventionally called <code>final</code> and <code>prev</code> (formerly also <code>self</code> and <code>super</code>), and return a set of packages. ... Overlays are similar to other methods for customizing Nixpkgs, in particular the packageOverrides ... Indeed, packageOverrides acts as an overlay with only the <code>prev</code> (<code>super</code>) argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute.</blockquote>


<cite>From the [https://nixos.org/manual/nixpkgs/stable/#sec-overlays-definition Nixpkgs manual]</cite>
<cite>From the [https://nixos.org/manual/nixpkgs/stable/#sec-overlays-definition Nixpkgs manual]</cite>
Line 18: Line 18:
== Data flow of overlays ==
== Data flow of overlays ==


The data flow around overlays, especially regarding <tt>prev</tt> and <tt>final</tt> arguments can be a bit confusing if you are not familiar with how overlays work. This graph shows the data flow:
The data flow of overlays, especially regarding <tt>prev</tt> and <tt>final</tt> arguments can be a bit confusing if you are not familiar with how overlays work. This graph shows the data flow:


[[File:Dram-overlay-self-super.png]]
[[File:Dram-overlay-final-prev.png]]


Here the main package set is extended with two overlays, ext-1 and ext-2. <tt>x // y</tt> is represented by a <tt>//</tt> box with <tt>x</tt> coming in from the left and <tt>y</tt> from above.
Here the main package set is extended with two overlays, ext-1 and ext-2. <tt>x // y</tt> is represented by a <tt>//</tt> box with <tt>x</tt> coming in from the left and <tt>y</tt> from above.


As you can see, <tt>final</tt> is the same for every stage, but <tt>prev</tt> comes from only the stage before. So when you define an attribute <tt>foo</tt> in the set to override it, within that overlay <tt>final.foo</tt> will be it's version, and <tt>prev.foo</tt> will be the non-overriden version. This is why you see patterns like <tt>foo = prev.foo.override { ... }</tt>.
As you can see, <tt>final</tt> is the same for every stage, but <tt>prev</tt> comes from only the stage before. So when you define an attribute <tt>foo</tt> in the set to override it, within that overlay <tt>final.foo</tt> will be its version, and <tt>prev.foo</tt> will be the non-overriden version. This is why you see patterns like <tt>foo = prev.foo.override { ... }</tt>.


The names <tt>final</tt> and <tt>prev</tt> might remind you of inheritance in object-oriented languages. In fact, overlays are exactly the same thing as subclasses, with regards to overriding and calling methods. This data flow is also how objects know which method to call. This is probably why the two arguments got their names, too.
The names <tt>final</tt> and <tt>prev</tt> might remind you of inheritance in object-oriented languages. In fact, overlays are exactly the same thing as subclasses, with regards to overriding and calling methods. This data flow is also how objects know which method to call. This is probably how the two arguments got their names, too.


== Data flow of overlays (alternative explanation) ==
== Data flow of overlays (alternative explanation) ==
Line 43: Line 43:




And <syntaxhighlight lang="nix">final: prev: firefox = final.firefox.override { ... };</syntaxhighlight> would cause infinite recursion.
And <syntaxhighlight lang="nix">final: prev: { firefox = final.firefox.override { ... }; }</syntaxhighlight> would cause infinite recursion.


== Using overlays ==
== Using overlays ==
Line 50: Line 50:


==== In standalone nix code ====
==== In standalone nix code ====
===== In a shell.nix =====
When writing standalone nix code, for example a <code>shell.nix</code> for a project, one usually starts by importing nixpkgs: <code>let pkgs = import <nixpkgs> {}</code>. To use an overlay in this context, replace that by:
When writing standalone nix code, for example a <code>shell.nix</code> for a project, one usually starts by importing nixpkgs: <code>let pkgs = import <nixpkgs> {}</code>. To use an overlay in this context, replace that by:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
import <nixpkgs> { overlays = [ overlay1 overlay2 ]; }
import <nixpkgs> { overlays = [ overlay1 overlay2 ]; }
</syntaxhighlight>
===== In a Nix flake =====
In a Nix flake, nixpkgs will be coming from the inputs. It is common to write something like
<syntaxhighlight lang="nix">
let pkgs = nixpkgs.legacyPackages.${system}
</syntaxhighlight>
where <code>system</code> is a variable containing eg. <code>"x86_64-linux"</code>. In order to apply overlays to this, one can do either of:
<syntaxhighlight lang="nix">
let pkgs = (nixpkgs.legacyPackages.${system}.extend overlay1).extend overlay2
</syntaxhighlight>
or, using the <code>import</code> function:
<syntaxhighlight lang="nix">
let pkgs = import nixpkgs { inherit system; overlays = [ overlay1 overlay2 ]; }
</syntaxhighlight>
</syntaxhighlight>


Line 130: Line 148:
</syntaxhighlight>
</syntaxhighlight>


Then, add the following contents to <tt>/etc/nixos/overlays-compat/overlays.nix</tt><ref>Based on [https://gitlab.com/samueldr/nixos-configuration/blob/3febd83b15210282d6435932944d426cd0a9e0ca/modules/overlays-compat/overlays.nix [[User:samueldr|@samueldr]]'s configuration: overlays-compat]</ref>:
Then, add the following contents to <tt>/etc/nixos/overlays-compat/overlays.nix</tt><ref>Based on [https://gitlab.com/samueldr/nixos-configuration/blob/3febd83b15210282d6435932944d426cd0a9e0ca/modules/overlays-compat/overlays.nix][[User:Samueldr|@samueldr]]<span>'s configuration: overlays-compat</span></ref>:


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
Line 159: Line 177:
       rev = "923e7d7ebc5c1f009755bdeb789ac25658ccce03";
       rev = "923e7d7ebc5c1f009755bdeb789ac25658ccce03";
       # If you don't know the hash, the first time, set:
       # If you don't know the hash, the first time, set:
       # sha256 = "0000000000000000000000000000000000000000000000000000";
       # hash = "";
       # then nix will fail the build with such an error message:
       # then nix will fail the build with such an error message:
       # hash mismatch in fixed-output derivation '/nix/store/m1ga09c0z1a6n7rj8ky3s31dpgalsn0n-source':
       # hash mismatch in fixed-output derivation '/nix/store/m1ga09c0z1a6n7rj8ky3s31dpgalsn0n-source':
       # wanted: sha256:0000000000000000000000000000000000000000000000000000
       # specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
       # got:    sha256:173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4
       # got:    sha256-173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4
       sha256 = "173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4";
       hash = "173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4";
     };
     };
   });
   });
Line 181: Line 199:
       (prev.fetchpatch {
       (prev.fetchpatch {
         url = "https://github.com/charlieLehman/sl/commit/e20abbd7e1ee26af53f34451a8f7ad79b27a4c0a.patch";
         url = "https://github.com/charlieLehman/sl/commit/e20abbd7e1ee26af53f34451a8f7ad79b27a4c0a.patch";
         sha256 = "07sx98d422589gxr8wflfpkdd0k44kbagxl3b51i56ky2wfix7rc";
         hash = "07sx98d422589gxr8wflfpkdd0k44kbagxl3b51i56ky2wfix7rc";
       })
       })
       # alternatively if you have a local patch,
       # alternatively if you have a local patch,
Line 207: Line 225:
final: prev: {
final: prev: {
   # elements of pkgs.gnome must be taken from gfinal and gprev
   # elements of pkgs.gnome must be taken from gfinal and gprev
   gnome = prev.gnome.overrideScope' (gfinal: gprev: {
   gnome = prev.gnome.overrideScope (gfinal: gprev: {
     mutter = gprev.mutter.overrideAttrs (oldAttrs: {
     mutter = gprev.mutter.overrideAttrs (oldAttrs: {
       patches = oldAttrs.patches ++ [
       patches = oldAttrs.patches ++ [
Line 213: Line 231:
         (prev.fetchpatch {
         (prev.fetchpatch {
           url = "https://salsa.debian.org/gnome-team/mutter/-/raw/91d9bdafd5d624fe1f40f4be48663014830eee78/debian/patches/x11-Add-support-for-fractional-scaling-using-Randr.patch";
           url = "https://salsa.debian.org/gnome-team/mutter/-/raw/91d9bdafd5d624fe1f40f4be48663014830eee78/debian/patches/x11-Add-support-for-fractional-scaling-using-Randr.patch";
           sha256 = "m6PKjVxhGVuzsMBVA82UyJ6Cb1s6SMI0eRooa+F2MY8=";
           hash = "m6PKjVxhGVuzsMBVA82UyJ6Cb1s6SMI0eRooa+F2MY8=";
         })
         })
     ];
     ];
Line 222: Line 240:
         (prev.fetchpatch {
         (prev.fetchpatch {
           url = "https://salsa.debian.org/gnome-team/gnome-control-center/-/raw/f185f33fb200cc963c062c7a82920a085f696978/debian/patches/ubuntu/display-Support-UI-scaled-logical-monitor-mode.patch";
           url = "https://salsa.debian.org/gnome-team/gnome-control-center/-/raw/f185f33fb200cc963c062c7a82920a085f696978/debian/patches/ubuntu/display-Support-UI-scaled-logical-monitor-mode.patch";
           sha256 = "XBMD0chaV6GGg3R9/rQnsBejXspomVZz/a4Bvv/AHCA=";
           hash = "XBMD0chaV6GGg3R9/rQnsBejXspomVZz/a4Bvv/AHCA=";
         })
         })
         # https://salsa.debian.org/gnome-team/gnome-control-center/-/blob/ubuntu/master/debian/patches/ubuntu/display-Allow-fractional-scaling-to-be-enabled.patch
         # https://salsa.debian.org/gnome-team/gnome-control-center/-/blob/ubuntu/master/debian/patches/ubuntu/display-Allow-fractional-scaling-to-be-enabled.patch
         (prev.fetchpatch {
         (prev.fetchpatch {
           url = "https://salsa.debian.org/gnome-team/gnome-control-center/-/raw/f185f33fb200cc963c062c7a82920a085f696978/debian/patches/ubuntu/display-Allow-fractional-scaling-to-be-enabled.patch";
           url = "https://salsa.debian.org/gnome-team/gnome-control-center/-/raw/f185f33fb200cc963c062c7a82920a085f696978/debian/patches/ubuntu/display-Allow-fractional-scaling-to-be-enabled.patch";
           sha256 = "Pm6PTmsL2bW9JAHD1u0oUEqD1PCIErOlcuqlwvP593I=";
           hash = "Pm6PTmsL2bW9JAHD1u0oUEqD1PCIErOlcuqlwvP593I=";
         })
         })
       ];
       ];
Line 235: Line 253:
</syntaxhighlight>
</syntaxhighlight>


=== Overriding a package inside an attribute set ===
=== Overriding a package inside an extensible attribute set ===


Here is an example of adding plugins to `vimPlugins`.
Here is an example of adding plugins to `vimPlugins`.
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
final: prev: {
final: prev: {
   vimPlugins = prev.vimPlugins // {
   vimPlugins = prev.vimPlugins.extend (final': prev': {
     indent-blankline-nvim-lua =
     indent-blankline-nvim-lua = prev.callPackage ../packages/indent-blankline-nvim-lua { };
      prev.pkgs.callPackage ../packages/indent-blankline-nvim-lua { };
   });
   };
}
}
</syntaxhighlight>
=== Overrding a package inside a plain attribute set ===
Here's an example of overriding the source of <code>obs-studio-plugins.obs-backgroundremoval</code>.
<syntaxhighlight lang="nix">
    final: prev: {
      obs-studio-plugins = prev.obs-studio-plugins // {
        obs-backgroundremoval =
          prev.obs-studio-plugins.obs-backgroundremoval.overrideAttrs (old: {
            version = "0.5.17";
            src = prev.fetchFromGitHub {
              owner = "royshil";
              repo = "obs-backgroundremoval";
              rev = "v0.5.17";
              hash = "";
            };
          });
      };
    };
</syntaxhighlight>
</syntaxhighlight>


Line 257: Line 295:
final: prev:
final: prev:
# Within the overlay we use a recursive set, though I think we can use `final` as well.
# Within the overlay we use a recursive set, though I think we can use `final` as well.
rec {
{
   # nix-shell -p python.pkgs.my_stuff
   # nix-shell -p python.pkgs.my_stuff
   python = prev.python.override {
   python = prev.python.override {
     # Careful, we're using a different final and prev here!
     # Careful, we're using a different final and prev here!
     packageOverrides = final: prev: {
     packageOverrides = pyfinal: pyprev: {
       my_stuff = prev.buildPythonPackage rec {
       my_stuff = pyprev.buildPythonPackage rec {
         pname = "pyaes";
         pname = "pyaes";
         version = "1.6.0";
         version = "1.6.0";
         src = prev.fetchPypi {
         src = pyprev.fetchPypi {
           inherit pname version;
           inherit pname version;
           sha256 = "0bp9bjqy1n6ij1zb86wz9lqa1dhla8qr1d7w2kxyn7jbj56sbmcw";
           hash = "0bp9bjqy1n6ij1zb86wz9lqa1dhla8qr1d7w2kxyn7jbj56sbmcw";
         };
         };
       };
       };
Line 273: Line 311:
   };
   };
   # nix-shell -p pythonPackages.my_stuff
   # nix-shell -p pythonPackages.my_stuff
   pythonPackages = python.pkgs;
   pythonPackages = final.python.pkgs;


   # nix-shell -p my_stuff
   # nix-shell -p my_stuff
   my_stuff = pythonPackages.buildPythonPackage rec {
   my_stuff = final.pythonPackages.buildPythonPackage rec {
     pname = "pyaes";
     pname = "pyaes";
     version = "1.6.0";
     version = "1.6.0";
     src = pythonPackages.fetchPypi {
     src = pythonPackages.fetchPypi {
       inherit pname version;
       inherit pname version;
       sha256 = "0bp9bjqy1n6ij1zb86wz9lqa1dhla8qr1d7w2kxyn7jbj56sbmcw";
       hash = "0bp9bjqy1n6ij1zb86wz9lqa1dhla8qr1d7w2kxyn7jbj56sbmcw";
     };
     };
   };
   };
Line 290: Line 328:
=== R Packages Overlay ===
=== R Packages Overlay ===


Here is an example of an R packages overlay, in which it can be seen how to provide different versions of packages then those available in the current R version. It should be noticed that in the case of R and Python the argument to <code>override</code> is named differently. Names of these can be find using <code>nix repl</code> and evaluating e.g. <code>python.override.__functionArgs</code>.
Here is an example of an R packages overlay, in which it can be seen how to provide different versions of packages then those available in the current R version. It should be noted that in the case of R and Python the argument to <code>override</code> is named differently. Names of these can be found using <code>nix repl</code> and evaluating e.g. <code>python.override.__functionArgs</code>.


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
Line 305: Line 343:
           url =
           url =
             "https://github.com/r-lib/rprojroot/archive/refs/tags/v2.0.2.tar.gz";
             "https://github.com/r-lib/rprojroot/archive/refs/tags/v2.0.2.tar.gz";
           sha256 = "1i0s1f7hla91yw1fdx0rn7c18dp6jwmg2mlww8dix1kk7qbxfjww";
           hash = "1i0s1f7hla91yw1fdx0rn7c18dp6jwmg2mlww8dix1kk7qbxfjww";
         };
         };
         nativeBuildInputs = [ prev.R ];
         nativeBuildInputs = [ prev.R ];
Line 315: Line 353:
         src = prev.fetchurl {
         src = prev.fetchurl {
           url = "https://github.com/r-lib/here/archive/refs/tags/v1.0.1.tar.gz";
           url = "https://github.com/r-lib/here/archive/refs/tags/v1.0.1.tar.gz";
           sha256 = "0ky6sq6n8px3b70s10hy99sccf3vcjjpdhamql5dr7i9igsf8nqy";
           hash = "0ky6sq6n8px3b70s10hy99sccf3vcjjpdhamql5dr7i9igsf8nqy";
         };
         };
         nativeBuildInputs = [ prev.R final.rPackages.rprojroot ];
         nativeBuildInputs = [ prev.R final.rPackages.rprojroot ];
Line 329: Line 367:
Due to https://github.com/NixOS/nixpkgs/issues/107070
Due to https://github.com/NixOS/nixpkgs/issues/107070


it is not possible to just override <code>cargoSha256</code>, instead cargoDeps has to be overriden
it is not possible to just override <code>cargoHash</code>, instead cargoDeps has to be overriden


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
Line 335: Line 373:
   rnix-lsp = prev.rnix-lsp.overrideAttrs (oldAttrs: rec {
   rnix-lsp = prev.rnix-lsp.overrideAttrs (oldAttrs: rec {
     version = "master";
     version = "master";
 
 
     src = prev.fetchFromGitHub {
     src = prev.fetchFromGitHub {
       owner = "nix-community";
       owner = "nix-community";
       repo = "rnix-lsp";
       repo = "rnix-lsp";
       rev = "1fdd7cf9bf56b8ad2dddcfd27354dae8aef2b453";
       rev = "1fdd7cf9bf56b8ad2dddcfd27354dae8aef2b453";
       sha256 = "sha256-w0hpyFXxltmOpbBKNQ2tfKRWELQzStc/ho1EcNyYaWc=";
       hash = "sha256-w0hpyFXxltmOpbBKNQ2tfKRWELQzStc/ho1EcNyYaWc=";
     };
     };
 
 
     cargoDeps = oldAttrs.cargoDeps.overrideAttrs (lib.const {
     cargoDeps = oldAttrs.cargoDeps.overrideAttrs (lib.const {
       name = "rnix-lsp-vendor.tar.gz";
       name = "rnix-lsp-vendor.tar.gz";
Line 351: Line 389:
}
}
</syntaxhighlight>
</syntaxhighlight>
== List of 3rd party overlays ==
This is an non-exhaustive list:
* [https://nixos.org/manual/nixpkgs/unstable/#using-community-maintained-rust-toolchains Details in the Nixpkgs manual for using Rust overlays]
* [https://github.com/peter-sa/nixos-rocm Overlay for Radeon Open-Compute packages]
* [https://github.com/garbas/nixpkgs-python Overlay by Rok Garbas for a set of python packages built by pypi2nix (archived)]


== See also ==
== See also ==
Line 356: Line 403:
* [https://nixos.org/nixpkgs/manual/#chap-overlays Overlays  in nixpkgs manual]
* [https://nixos.org/nixpkgs/manual/#chap-overlays Overlays  in nixpkgs manual]
* [https://blog.flyingcircus.io/2017/11/07/nixos-the-dos-and-donts-of-nixpkgs-overlays/ Blog post "The DOs and DON’Ts of nixpkgs overlays"]
* [https://blog.flyingcircus.io/2017/11/07/nixos-the-dos-and-donts-of-nixpkgs-overlays/ Blog post "The DOs and DON’Ts of nixpkgs overlays"]
* [https://www.youtube.com/watch?v=s2fkgkN55vk&list=PLgknCdxP89ReD6gxl755B6G_CI65z4J2e Nixpkgs Overlays – A place for all excluded packages] - Talk by Nicolas B. Pierron at NixCon 2017


==== References ====
==== References ====