Update a package: Difference between revisions

From NixOS Wiki
imported>Fadenb
→‎See also: removed dead link
Flukx (talk | contribs)
Note for newer packages under /by-name
 
(22 intermediate revisions by 14 users not shown)
Line 1: Line 1:
== Intro ==
== Introduction ==


This is the easiest and more straightforward way I know to modify and expand [https://nixos.org/nixpkgs/ nixpkgs]. I hope this article will help future newbies so the experts can focus on other stuff.
This is the easiest and more straightforward way I know to modify and expand [https://nixos.org/nixpkgs/ nixpkgs]. I hope this article will help future newbies so the experts can focus on other stuff.
Line 7: Line 7:
The official nixpkgs sources are managed on a [http://git-scm.com/ git] repository at [https://github.com GitHub] (https://github.com/NixOS/nixpkgs). And GitHub implements most of the tools for the nixpkgs development workflow.
The official nixpkgs sources are managed on a [http://git-scm.com/ git] repository at [https://github.com GitHub] (https://github.com/NixOS/nixpkgs). And GitHub implements most of the tools for the nixpkgs development workflow.


''All operations explained on this document, if tested, have been tested on a [https://nixos.org/nixos/ NixOS] system.''
This tutorial was designed for [https://nixos.org/nixos/ NixOS], and may not always apply on other systems. If you want reproduce the environment used in this tutorial exactly, you can <syntaxhighlight lang="bash" inline>git reset --hard 111c8db50038</syntaxhighlight> in your local copy of Nixpkgs.


== Cloning the "NixOS/nixpkgs" repository on GitHub ==
== Cloning the "NixOS/nixpkgs" repository on GitHub ==


* I had to sign up at GitHub. This is free (as in beer) as of 2013-03-20.
* I had to sign up at GitHub. This is free (as in beer).
* Then I visited [https://github.com/NixOS/nixpkgs the official nixpkgs repository] and hit the ''fork'' button.
* Then I visited [https://github.com/NixOS/nixpkgs the official nixpkgs repository] and hit the ''fork'' button.
* This results in your own somewhat-independent repository: <tt>https://github.com/your_username/nixpkgs</tt>
* This results in your own somewhat-independent repository: <tt>https://github.com/your_username/nixpkgs</tt>
Line 18: Line 18:


* Now you can clone your repository somewhere on your system:
* Now you can clone your repository somewhere on your system:
<pre>mkdir ~/devel/
<syntaxhighlight lang="bash">
mkdir ~/devel/
cd ~/devel/
cd ~/devel/
git clone https://github.com/your_username/nixpkgs.git
git clone https://github.com/your_username/nixpkgs.git \
    --depth=1 # Prevents cloning the entire history, which is multiple gigabytes
cd nixpkgs/
cd nixpkgs/
git remote add upstream https://github.com/NixOS/nixpkgs.git</pre>
git remote add upstream https://github.com/NixOS/nixpkgs.git</syntaxhighlight>
* When the clone is not recent or you suspect there have been changes upstream (in official nixpkgs):
* When the clone is not recent or you suspect there have been changes upstream (in official nixpkgs):
<pre>cd ~/devel/nixpkgs/
<syntaxhighlight lang="bash">
cd ~/devel/nixpkgs/
git fetch upstream
git fetch upstream
git merge upstream/master</pre>
git merge upstream/master
</syntaxhighlight>
* You should get a git repository with the whole nixpkgs tree under <tt>~/devel/nixpkgs/</tt>
* You should get a git repository with the whole nixpkgs tree under <tt>~/devel/nixpkgs/</tt>
* Unless explicitly stated, for the rest of the document we are working from this directory: <pre>cd ~/devel/nixpkgs/</pre>
* Unless explicitly stated, for the rest of the document we are working from this directory: <syntaxhighlight lang="bash" inline>cd ~/devel/nixpkgs/</syntaxhighlight>
* Just to ensure everything is clear, on this directory there should be two main entries: the <tt>default.nix</tt> file and the <tt>pkgs/</tt> directory.
* Just to ensure everything is clear, on this directory there should be two main entries: the <tt>default.nix</tt> file and the <tt>pkgs/</tt> directory.
* From now on, <tt>pkgs/</tt> will refer to this <tt>nixpkgs/pkgs/</tt> so if you keep the equivalent of <tt>~/devel/nixpkgs/</tt> as your CWD (current working directory), most commands and pathnames can be copied and pasted in real life.
* From now on, <tt>pkgs/</tt> will refer to this <tt>nixpkgs/pkgs/</tt> so if you keep the equivalent of <tt>~/devel/nixpkgs/</tt> as your CWD (current working directory), most commands and pathnames can be copied and pasted in real life.
Line 37: Line 41:
== Following the white rabbit ==
== Following the white rabbit ==


* This section does not apply for packages in the directory pkgs/by-name. For them, see the included [https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/README.md pkgs/by-name/README.md].
* I will center this tutorial on the nix expression that packages the [http://i3wm.org/ i3 window manager], as this expression is quite simple and it was contributing to this expression that I got to understand this whole workflow I am documenting.
* I will center this tutorial on the nix expression that packages the [http://i3wm.org/ i3 window manager], as this expression is quite simple and it was contributing to this expression that I got to understand this whole workflow I am documenting.
* In <tt>~/devel/nixpkgs/default.nix</tt> there is an ''import'' statement that reads <tt>import ./pkgs/top-level/all-packages.nix</tt>
* In <tt>~/devel/nixpkgs/default.nix</tt> there is an ''import'' statement that reads <tt>import ./pkgs/top-level/all-packages.nix</tt>
* I have the feeling that most references between nix sources (''nix expressions'') in ''nixpkgs'' are stated as relative paths pointing up and down the <tt>nixpkgs/</tt> directory hierarchy.
* I have the feeling that most references between nix sources (''nix expressions'') in ''nixpkgs'' are stated as relative paths pointing up and down the <tt>nixpkgs/</tt> directory hierarchy.
* Please have a look at this <tt>pkgs/top-level/all-packages.nix</tt>. There are many more or less complex definitions at the beginning, and then a lot of declarations such as:
* Please have a look at this <tt>pkgs/top-level/all-packages.nix</tt>. There are many more or less complex definitions at the beginning, and then a lot of declarations such as:
<pre>i3      = callPackage ../applications/window-managers/i3 { };
<syntaxhighlight lang="nix">
i3      = callPackage ../applications/window-managers/i3 { };
i3lock  = callPackage ../applications/window-managers/i3/lock.nix { };
i3lock  = callPackage ../applications/window-managers/i3/lock.nix { };
i3status = callPackage ../applications/window-managers/i3/status.nix { };</pre> and <pre>pango = callPackage ../development/libraries/pango/1.30.x.nix { };</pre>
i3status = callPackage ../applications/window-managers/i3/status.nix { };
</syntaxhighlight>
and <syntaxhighlight lang="nix" inline>pango = callPackage ../development/libraries/pango/1.30.x.nix { };</syntaxhighlight>
* Now let's have a look at the contents of <tt>pkgs/applications/window-managers/i3/</tt>:
* Now let's have a look at the contents of <tt>pkgs/applications/window-managers/i3/</tt>:
<pre>$ ls -1 pkgs/applications/window-managers/i3/
<syntaxhighlight lang="console">
$ ls -1 pkgs/applications/window-managers/i3/
default.nix
default.nix
lock.nix
lock.nix
status.nix</pre>
status.nix
</syntaxhighlight>
* The first assignment <tt>i3 = callPackage ...</tt> refers implicitly to <tt>default.nix</tt> on the <tt>pkgs/applications/window-managers/i3/</tt> directory.
* The first assignment <tt>i3 = callPackage ...</tt> refers implicitly to <tt>default.nix</tt> on the <tt>pkgs/applications/window-managers/i3/</tt> directory.
* The second and third assignments are explicitly referring to <tt>lock.nix</tt> and <tt>status.nix</tt> files on this <tt>pkgs/applications/window-managers/i3/</tt> directory.
* The second and third assignments are explicitly referring to <tt>lock.nix</tt> and <tt>status.nix</tt> files on this <tt>pkgs/applications/window-managers/i3/</tt> directory.
Line 56: Line 66:


* We will dive (not too deep, though) on <tt>pkgs/applications/window-managers/i3/default.nix</tt> which is the nix expression that corresponds to the package for the i3 window manager:
* We will dive (not too deep, though) on <tt>pkgs/applications/window-managers/i3/default.nix</tt> which is the nix expression that corresponds to the package for the i3 window manager:
<pre>{ fetchurl, stdenv, which, pkgconfig, libxcb, xcbutilkeysyms, xcbutil,
<syntaxhighlight lang="nix">
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   libXcursor, coreutils, perl, pango }:
   libXcursor, coreutils, perl, pango }:
Line 69: Line 80:
   };
   };


   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
     libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
     libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];


   patchPhase = ''
   postPatch = ''
     patchShebangs
     patchShebangs
   '';
   '';
Line 84: Line 95:
     license = stdenv.lib.licenses.bsd3;
     license = stdenv.lib.licenses.bsd3;
   };
   };
}</pre>
}
</syntaxhighlight>
* As an example of the trivial fixes and improvements I was talking about, you can have a look at two changesets (''commits'' in the ''git'' terminology) I did:
* As an example of the trivial fixes and improvements I was talking about, you can have a look at two changesets (''commits'' in the ''git'' terminology) I did:
** adding the ''pango'' dependency to [https://github.com/NixOS/nixpkgs/commit/f976962d7b7087920a6635ca90f07b06e8a44202 enable antialiased fonts on i3]
** adding the ''pango'' dependency to [https://github.com/NixOS/nixpkgs/commit/f976962d7b7087920a6635ca90f07b06e8a44202 enable antialiased fonts on i3]
Line 98: Line 110:
** Changed the <tt>version</tt> string.
** Changed the <tt>version</tt> string.
** Changed the checksum for the tarball on the <tt>sha256</tt> string.
** Changed the checksum for the tarball on the <tt>sha256</tt> string.
** Removed a line on the <tt>patchPhase</tt> string. This line was a <tt>sed</tt> command which was erasing some lines from the file <tt>common.mk</tt>, which is part of the sources in the tarball, before building the application.
** Removed a line on the <tt>postPatch</tt> string. This line was a <tt>sed</tt> command which was erasing some lines from the file <tt>common.mk</tt>, which is part of the sources in the tarball, before building the application.
* And in an attempt to extract something positive from the issue, we are going to review an additional [https://github.com/NixOS/nixpkgs/commit/1eb437784021521d57ae96980bf447e9162d4415 unfortunate change] I made on <tt>pkgs/top-level/all-packages.nix</tt>
* And in an attempt to extract something positive from the issue, we are going to review an additional [https://github.com/NixOS/nixpkgs/commit/1eb437784021521d57ae96980bf447e9162d4415 unfortunate change] I made on <tt>pkgs/top-level/all-packages.nix</tt>
* To figure out the checksum for the tarball you have the <tt>nix-prefetch-url</tt> utility. In addition to returning the ''sha256 checksum'' it will also leave the downloaded blob on your ''nix store'', thus avoiding additional downloads when you test the build process. You get the path to the downloaded blob and the ''sha256sum'' on the last two lines of the command's output:
* To figure out the checksum for the tarball you have the <tt>nix-prefetch-url</tt> utility. In addition to returning the ''sha256 checksum'' it will also leave the downloaded blob on your ''nix store'', thus avoiding additional downloads when you test the build process. You get the path to the downloaded blob and the ''sha256sum'' on the last two lines of the command's output:
<pre>$ nix-prefetch-url 'http://i3wm.org/downloads/i3-4.5.1.tar.bz2'
<syntaxhighlight lang="console">
$ nix-prefetch-url 'http://i3wm.org/downloads/i3-4.5.1.tar.bz2'
   % Total    % Received % Xferd  Average Speed  Time    Time    Time  Current
   % Total    % Received % Xferd  Average Speed  Time    Time    Time  Current
                                 Dload  Upload  Total  Spent    Left  Speed
                                 Dload  Upload  Total  Spent    Left  Speed
100  858k  100  858k    0    0  866k      0 --:--:-- --:--:-- --:--:--  867k
100  858k  100  858k    0    0  866k      0 --:--:-- --:--:-- --:--:--  867k
path is "/nix/store/clvs3jvyk24bskkn5pfd7fjpyaa1svsx-i3-4.5.1.tar.bz2"
path is "/nix/store/clvs3jvyk24bskkn5pfd7fjpyaa1svsx-i3-4.5.1.tar.bz2"
1wi3p1s4z7r6kvzwxlg59fx6bfiqc2mlybhqmqddf8aaghf5zrds</pre>
1wi3p1s4z7r6kvzwxlg59fx6bfiqc2mlybhqmqddf8aaghf5zrds
* Almost out of scope here, but you can easily review your changes. Go get some ''git'' documentation if you need:
</syntaxhighlight>
<pre>$ git diff
* Almost out of scope here, but you can easily review your changes. Go get some ''git'' documentation if you need: <syntaxhighlight lang="console" inline>$ git diff</syntaxhighlight>
<syntaxhighlight lang="diff">
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
index f8e4e16..4173660 100644
Line 114: Line 128:
+++ b/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
@@ -1,21 +1,20 @@
  { fetchurl, stdenv, which, pkgconfig, libxcb, xcbutilkeysyms, xcbutil,
  { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
-  libXcursor, coreutils, perl }:
Line 130: Line 144:
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
   
   
   patchPhase = ''
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
     patchShebangs .
Line 152: Line 166:
   
   
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
     cairo = cairo.override { xcbSupport = true; };</pre>
     cairo = cairo.override { xcbSupport = true; };
</syntaxhighlight>


== Testing the changes ==
== Testing the changes ==
Line 160: Line 175:
* If your current directory is, as we said, <tt>~/devel/nixpkgs/</tt>, you can request nix to build the modified package (see <tt>man nix-build</tt>):
* If your current directory is, as we said, <tt>~/devel/nixpkgs/</tt>, you can request nix to build the modified package (see <tt>man nix-build</tt>):
<pre>nix-build -A i3</pre>
<pre>nix-build -A i3</pre>
* Alternatively, you can request nix to build from expressions on an specific directory (<tt>-f .</tt>)and then to update your user environment so you will have the newly built binaries, man pages etc. available at hand for testing (see <tt>man nix-env</tt>):
* In this invocation the <tt>i3</tt> in <tt>-A i3</tt> refers to <tt>i3 = callPackage ...</tt> in <tt>pkgs/top-level/all-packages.nix</tt>. In the same way as the <tt>pango</tt> we talked about above, both names are attributes from the top-level Nix expression being evaluated. In this case <tt>~/devel/nixpkgs/default.nix</tt>
<pre>nix-env -f . -iA i3</pre>
* In these two invocations the <tt>i3</tt> in <tt>-A i3</tt> refers to <tt>i3 = callPackage ...</tt> in <tt>pkgs/top-level/all-packages.nix</tt>. In the same way as the <tt>pango</tt> we talked about above, both names are attributes from the top-level Nix expression being evaluated. In this case <tt>~/devel/nixpkgs/default.nix</tt>
* Here I would have detected the error I introduced on <tt>pkgs/top-level/all-packages.nix</tt>. Too bad I didn't still know how to test my changes:
* Here I would have detected the error I introduced on <tt>pkgs/top-level/all-packages.nix</tt>. Too bad I didn't still know how to test my changes:
<pre>$ nix-build -A i3
<syntaxhighlight lang="console">
$ nix-build -A i3
error: function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
error: function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
(use `--show-trace' to show detailed location information)</pre>
(use `--show-trace' to show detailed location information)
</syntaxhighlight>
* Let's follow the advice by adding <tt>--show-trace</tt> to the <tt>nix-build</tt> invocation:
* Let's follow the advice by adding <tt>--show-trace</tt> to the <tt>nix-build</tt> invocation:
<pre>$ nix-build --show-trace -A i3  
<syntaxhighlight lang="console">
$ nix-build --show-trace -A i3  
error: while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:98:35':
error: while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:98:35':
while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:59:24':
while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:59:24':
while evaluating the builtin function `isAttrs':
while evaluating the builtin function `isAttrs':
function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument</pre>
function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
</syntaxhighlight>
* Not that I got any more clues from this extra step. But it turned out that there is no mention to <tt>cairo</tt> on <tt>pkgs/applications/window-managers/i3/default.nix</tt> so it was illegal to invoke the <tt>cairo.override ...</tt> thing inside the <tt>i3 = callPackage ...</tt> on <tt>pkgs/top-level/all-packages.nix</tt>:
* Not that I got any more clues from this extra step. But it turned out that there is no mention to <tt>cairo</tt> on <tt>pkgs/applications/window-managers/i3/default.nix</tt> so it was illegal to invoke the <tt>cairo.override ...</tt> thing inside the <tt>i3 = callPackage ...</tt> on <tt>pkgs/top-level/all-packages.nix</tt>:
<pre>i3 = callPackage ../applications/window-managers/i3 {
 
<syntaxhighlight lang="nix">
i3 = callPackage ../applications/window-managers/i3 {
     cairo = cairo.override { xcbSupport = true; }; # WRONG!
     cairo = cairo.override { xcbSupport = true; }; # WRONG!
   };</pre>
   };
</syntaxhighlight>
* So I would have reverted my changes to <tt>pkgs/top-level/all-packages.nix</tt>. Take care with this command as it is quite eager to undo your changes. But that's out of scope here. See some ''git'' documentation:
* So I would have reverted my changes to <tt>pkgs/top-level/all-packages.nix</tt>. Take care with this command as it is quite eager to undo your changes. But that's out of scope here. See some ''git'' documentation:
<pre>git checkout pkgs/top-level/all-packages.nix</pre>
<pre>git checkout pkgs/top-level/all-packages.nix</pre>
* Now let's try again:
* Now let's try again:
<pre>$ nix-build -A i3
<syntaxhighlight lang="console">
$ nix-build -A i3
/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1
/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1
</pre>
</syntaxhighlight>
* When the build succeeds, <tt>nix-build</tt> will leave the result linked as <tt>./result</tt> on the current working directory:
* When the build succeeds, <tt>nix-build</tt> will leave the result linked as <tt>./result</tt> on the current working directory:
<pre>$ stat ./result | head -n 1
<syntaxhighlight lang="console">
   File: "./result" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1"</pre>
$ stat -c %N ./result
   File: "./result" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1"
</syntaxhighlight>
* If convenient, you can review and fiddle with the resulting stuff:
* If convenient, you can review and fiddle with the resulting stuff:
<pre>$ find result/ -type f
<syntaxhighlight lang="console">
$ find result/ -type f
result/include/i3/ipc.h
result/include/i3/ipc.h
result/bin/i3-input
result/bin/i3-input
Line 204: Line 228:
result/etc/i3/config
result/etc/i3/config
result/share/xsessions/i3.desktop
result/share/xsessions/i3.desktop
result/share/applications/i3.desktop</pre>
result/share/applications/i3.desktop
</syntaxhighlight>
* And for some real-life testing we would invoke <tt>nix-env</tt> as explained above:
* And for some real-life testing we would invoke <tt>nix-env</tt> as explained above:
<pre>$ nix-env -f . -iA i3
<syntaxhighlight lang="console">
$ nix-env -f . -iA i3
replacing old `i3-4.5.1'
replacing old `i3-4.5.1'
installing `i3-4.5.1'
installing `i3-4.5.1'
building path(s) `/nix/store/49ag28h77sjmg3rygpsm83jxkhn5jzyn-user-environment'
building path(s) `/nix/store/49ag28h77sjmg3rygpsm83jxkhn5jzyn-user-environment'
created 2006 symlinks in user environment</pre>
created 2006 symlinks in user environment
</syntaxhighlight>
* You might double-check that the binaries on <tt>$PATH</tt> are indeed the new ones:
* You might double-check that the binaries on <tt>$PATH</tt> are indeed the new ones:
<pre> stat `type -P i3` | head -n 1
<syntaxhighlight lang="console">$ stat `type -P i3` | head -n 1
   File: "~/.nix-profile/bin/i3" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1/bin/i3"</pre>
   File: "~/.nix-profile/bin/i3" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1/bin/i3"
</syntaxhighlight>
* It is quite annoying to test a window manager, <tt>i3</tt> in special as [http://bugs.i3wm.org/report/ticket/536 it does not implement the <tt>--replace</tt> feature].
* It is quite annoying to test a window manager, <tt>i3</tt> in special as [http://bugs.i3wm.org/report/ticket/536 it does not implement the <tt>--replace</tt> feature].
* And sometimes there are lots of other tricky possibilities. For example <tt>i3</tt> invokes other binaries from the same package, such as <tt>i3bar</tt>. I lost a great deal of time trying to figure out why the ''i3 status bar'' was refusing to render ''pango fonts''. That was because I was launching <tt>~/devel/nixpkgs/result/bin/i3</tt> instead of having the whole package replaced on my ''nix user environment''. And the old <tt>i3bar</tt> from the ''nix store'' was on my <tt>$PATH</tt>. Obviously that was before I learned the <tt>nix-env</tt> invocation shown above.
* And sometimes there are lots of other tricky possibilities. For example <tt>i3</tt> invokes other binaries from the same package, such as <tt>i3bar</tt>. I lost a great deal of time trying to figure out why the ''i3 status bar'' was refusing to render ''pango fonts''. That was because I was launching <tt>~/devel/nixpkgs/result/bin/i3</tt> instead of having the whole package replaced on my ''nix user environment''. And the old <tt>i3bar</tt> from the ''nix store'' was on my <tt>$PATH</tt>. Obviously that was before I learned the <tt>nix-env</tt> invocation shown above.
Line 223: Line 251:
* The first step is to commit your local changes to the local repository.
* The first step is to commit your local changes to the local repository.
* A quick overview of the affected stuff:
* A quick overview of the affected stuff:
<pre>$ git status
<syntaxhighlight lang="console">
$ git status
# On branch example
# On branch example
# Changes not staged for commit:
# Changes not staged for commit:
Line 232: Line 261:
#      modified:  pkgs/top-level/all-packages.nix
#      modified:  pkgs/top-level/all-packages.nix
#
#
no changes added to commit (use "git add" and/or "git commit -a")</pre>
no changes added to commit (use "git add" and/or "git commit -a")
</syntaxhighlight>
* As we have just seen, you can review the pending changes with <tt>git diff</tt>
* As we have just seen, you can review the pending changes with <tt>git diff</tt>
* You can limit the scope to review piece by piece:
* You can limit the scope to review piece by piece: <syntaxhighlight lang="console" inline>$ git diff pkgs/top-level/all-packages.nix</syntaxhighlight>
<pre>$ git diff pkgs/top-level/all-packages.nix
<syntaxhighlight lang="diff">
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0d3e4cc..a1c0d4b 100644
index 0d3e4cc..a1c0d4b 100644
Line 250: Line 280:
   
   
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
     cairo = cairo.override { xcbSupport = true; };</pre>
     cairo = cairo.override { xcbSupport = true; };
 
</syntaxhighlight>


* This change, at the time, looked good to me. So I went ahead committing it:
* This change, at the time, looked good to me. So I went ahead committing it:
<pre>$ git commit pkgs/top-level/all-packages.nix -m 'i3 window manager: cairo.override with xcbSupport enabled'
<syntaxhighlight lang="console">
$ git commit pkgs/top-level/all-packages.nix -m 'i3 window manager: cairo.override with xcbSupport enabled'
[example 731abb2] i3 window manager: cairo.override with xcbSupport enabled
[example 731abb2] i3 window manager: cairo.override with xcbSupport enabled
  1 file changed, 3 insertions(+), 1 deletion(-)</pre>
  1 file changed, 3 insertions(+), 1 deletion(-)
</syntaxhighlight>


* Let's review what's left:
* Let's review what's left: <syntaxhighlight lang="console" inline>$ git diff pkgs/applications/window-managers/i3/default.nix</syntaxhighlight>
<pre>$ git diff pkgs/applications/window-managers/i3/default.nix
<syntaxhighlight lang="diff">
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
index f8e4e16..4173660 100644
Line 264: Line 298:
+++ b/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
@@ -1,21 +1,20 @@
  { fetchurl, stdenv, which, pkgconfig, libxcb, xcbutilkeysyms, xcbutil,
  { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
-  libXcursor, coreutils, perl }:
Line 280: Line 314:
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
   
   
   patchPhase = ''
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
     patchShebangs .
   '';</pre>
   '';
</syntaxhighlight>
* These changes were made with two different purposes. I don't like commits mixing unrelated changes. It turns out that git can help here saving a lot of time and potential human error: <tt>git commit --patch pkgs/applications/window-managers/i3/default.nix</tt>.
* These changes were made with two different purposes. I don't like commits mixing unrelated changes. It turns out that git can help here saving a lot of time and potential human error: <tt>git commit --patch pkgs/applications/window-managers/i3/default.nix</tt>.
* ''git'' will sequentially show "groups of changes" (it calls them ''hunks''). For each ''hunk'' we are asked what we want to do; we can do several things than are displayed by typing a question mark. I will explain with my words the ones we are interested in right now. Obviously the explanation given by git prevails:
* ''git'' will sequentially show "groups of changes" (it calls them ''hunks''). For each ''hunk'' we are asked what we want to do; we can do several things than are displayed by typing a question mark. I will explain with my words the ones we are interested in right now. Obviously the explanation given by git prevails:
Line 295: Line 330:
* Note that this command is not destructive. Changes left out of the commit are not discarded.
* Note that this command is not destructive. Changes left out of the commit are not discarded.
* So I will do a first commit with all changes related to enabling pango support for i3. Please note how I explain git to <tt>s</tt> split the big hunk, <tt>y</tt> include the first small hunk on the commit, <tt>n</tt> skip the second hunk, <tt>n</tt> skip the third hunk, <tt>y</tt> and <tt>y</tt> to include the last two hunks:
* So I will do a first commit with all changes related to enabling pango support for i3. Please note how I explain git to <tt>s</tt> split the big hunk, <tt>y</tt> include the first small hunk on the commit, <tt>n</tt> skip the second hunk, <tt>n</tt> skip the third hunk, <tt>y</tt> and <tt>y</tt> to include the last two hunks:
<pre>$ git commit --patch pkgs/applications/window-managers/i3/default.nix -m 'i3 window manager: enable Pango support for anti-aliased fonts'
<syntaxhighlight lang="console" inline>$ git commit --patch pkgs/applications/window-managers/i3/default.nix -m 'i3 window manager: enable Pango support for anti-aliased fonts'</syntaxhighlight>
<syntaxhighlight lang="diff">
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
index f8e4e16..4173660 100644
Line 301: Line 337:
+++ b/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
@@ -1,21 +1,20 @@
  { fetchurl, stdenv, which, pkgconfig, libxcb, xcbutilkeysyms, xcbutil,
  { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
-  libXcursor, coreutils, perl }:
Line 317: Line 353:
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
   
   
   patchPhase = ''
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
     patchShebangs .
Line 329: Line 365:
Split into 5 hunks.
Split into 5 hunks.
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
  { fetchurl, stdenv, which, pkgconfig, libxcb, xcbutilkeysyms, xcbutil,
  { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
-  libXcursor, coreutils, perl }:
Line 355: Line 391:
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
@@ -12,6 +12,6 @@
@@ -12,6 +12,6 @@
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
   
   
   patchPhase = ''
   postPatch = ''
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? y
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? y
@@ -16,6 +16,5 @@
@@ -16,6 +16,5 @@
   
   
   patchPhase = ''
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
     patchShebangs .
Line 376: Line 412:


[example 729796b] i3 window manager: enable Pango support for anti-aliased fonts
[example 729796b] i3 window manager: enable Pango support for anti-aliased fonts
  1 file changed, 2 insertions(+), 3 deletions(-)</pre>
  1 file changed, 2 insertions(+), 3 deletions(-)
</syntaxhighlight>
* As expected, the skipped hunks are still on the local file, uncommitted:
* As expected, the skipped hunks are still on the local file, uncommitted:
<pre>$ git diff pkgs/applications/window-managers/i3/default.nix
<syntaxhighlight lang="console" inline>$ git diff pkgs/applications/window-managers/i3/default.nix</syntaxhighlight>
<syntaxhighlight lang="diff">
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index b977a6a..4173660 100644
index b977a6a..4173660 100644
Line 396: Line 434:
   };
   };
   
   
   buildInputs = [ which pkgconfig libxcb xcbutilkeysyms xcbutil xcbutilwm</pre>
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
</syntaxhighlight>
* As these changes are both related, we can issue a simple commit:
* As these changes are both related, we can issue a simple commit:
<pre>$ git commit pkgs/applications/window-managers/i3/default.nix -m "i3 window manager: version bump from 4.5 to 4.5.1"
<syntaxhighlight lang="console">
$ git commit pkgs/applications/window-managers/i3/default.nix -m "i3 window manager: version bump from 4.5 to 4.5.1"
[example 4c7daed] i3 window manager: version bump from 4.5 to 4.5.1
[example 4c7daed] i3 window manager: version bump from 4.5 to 4.5.1
  1 file changed, 2 insertions(+), 2 deletions(-)</pre>
  1 file changed, 2 insertions(+), 2 deletions(-)
</syntaxhighlight>
* Let's check if we're done:
* Let's check if we're done:
<pre>$ git status
<syntaxhighlight lang="console">
$ git status
# On branch example
# On branch example
# Your branch is ahead of 'origin/example' by 3 commit.
# Your branch is ahead of 'origin/example' by 3 commit.
#  (use "git push" to publish your local commits)
#  (use "git push" to publish your local commits)
#
#
nothing to commit, working directory clean</pre>
nothing to commit, working directory clean
</syntaxhighlight>


=== Pushing to GitHub ===
=== Pushing to GitHub ===


* Now we want to push the local commits to our clone of NixOS/nixpkgs at GitHub:
* Now we want to push the local commits to our clone of NixOS/nixpkgs at GitHub:
<pre>$ git push  
<syntaxhighlight lang="console">
$ git push  
Counting objects: 27, done.
Counting objects: 27, done.
Delta compression using up to 2 threads.
Delta compression using up to 2 threads.
Line 419: Line 463:
Total 19 (delta 14), reused 14 (delta 12)
Total 19 (delta 14), reused 14 (delta 12)
To https://github.com/your_username/nixpkgs.git
To https://github.com/your_username/nixpkgs.git
   1c86db4..4c7daed  example -> example</pre>
   1c86db4..4c7daed  example -> example
</syntaxhighlight>


== Sending a pull request to the official nixpkgs repository ==
== Sending a pull request to the official nixpkgs repository ==
Line 426: Line 471:
* Now there are 3 commits on the <tt>example</tt> branch of out nixpkgs repository at GitHub.
* Now there are 3 commits on the <tt>example</tt> branch of out nixpkgs repository at GitHub.
* The idea is to propose the official ''NixOS/nixpkgs'' maintainers to include our changes on the official repository, so all the ''nixpkgs'' users can eventually benefit of our improvements.
* The idea is to propose the official ''NixOS/nixpkgs'' maintainers to include our changes on the official repository, so all the ''nixpkgs'' users can eventually benefit of our improvements.
* We should probably get in touch with the ''nixpkgs'' maintainers at [irc://irc.freenode.net/#nixos #nixos] or [http://lists.science.uu.nl/mailman/listinfo/nix-dev nix-dev] to figure out the appropriate branch (e.g. <tt>master</tt>, <tt>x-updates</tt> or whatever) where the changes would be welcome.
* We should probably get in touch with the ''nixpkgs'' maintainers at [irc://irc.freenode.net/#nixos #nixos] to figure out the appropriate branch (e.g. <tt>master</tt>, <tt>x-updates</tt> or whatever) where the changes would be welcome.
* So now we have to formally ''request'' them to ''pull'' our changes, from our branch to the appropriate branch in ''NixOS/nixpkgs''. Hence the name "''pull request''" which I don't feel so intuitive.
* So now we have to formally ''request'' them to ''pull'' our changes, from our branch to the appropriate branch in ''NixOS/nixpkgs''. Hence the name "''pull request''" which I don't feel so intuitive.
* There is a "Pull Request" button in GitHub. Push it. Go ahead.
* There is a "Pull Request" button in GitHub. Push it. Go ahead.
Line 445: Line 490:


* [[Create and debug nix packages]]
* [[Create and debug nix packages]]
* [[Generic Algorithm on Doing Packaging]]
* [[Contributing#Contributing to the documentation|Contributing to Nix documentation]]
* [[Nixpkgs/Update Scripts]] - automatic updating of packages
[[:Category:Tutorial]]
[[:Category:Nixpkgs]]
[[:Category:Contributing]]

Latest revision as of 10:53, 3 November 2024

Introduction

This is the easiest and more straightforward way I know to modify and expand nixpkgs. I hope this article will help future newbies so the experts can focus on other stuff.

This document is not intended as a formal approach to learn nix, nixpkgs, NixOS or systems administration. It is just a sort of tutorial for newbies to be able to easily fix and contribute trivial improvements. I believe this approach will help newbies such as myself to get (slowly and inefficiently) educated on the nix language, the nix package manager and NixOS.

The official nixpkgs sources are managed on a git repository at GitHub (https://github.com/NixOS/nixpkgs). And GitHub implements most of the tools for the nixpkgs development workflow.

This tutorial was designed for NixOS, and may not always apply on other systems. If you want reproduce the environment used in this tutorial exactly, you can git reset --hard 111c8db50038 in your local copy of Nixpkgs.

Cloning the "NixOS/nixpkgs" repository on GitHub

Retrieving a working copy of your repository

  • Now you can clone your repository somewhere on your system:
mkdir ~/devel/
cd ~/devel/
git clone https://github.com/your_username/nixpkgs.git \
    --depth=1 # Prevents cloning the entire history, which is multiple gigabytes
cd nixpkgs/
git remote add upstream https://github.com/NixOS/nixpkgs.git
  • When the clone is not recent or you suspect there have been changes upstream (in official nixpkgs):
cd ~/devel/nixpkgs/
git fetch upstream
git merge upstream/master
  • You should get a git repository with the whole nixpkgs tree under ~/devel/nixpkgs/
  • Unless explicitly stated, for the rest of the document we are working from this directory: cd ~/devel/nixpkgs/
  • Just to ensure everything is clear, on this directory there should be two main entries: the default.nix file and the pkgs/ directory.
  • From now on, pkgs/ will refer to this nixpkgs/pkgs/ so if you keep the equivalent of ~/devel/nixpkgs/ as your CWD (current working directory), most commands and pathnames can be copied and pasted in real life.
  • So for example, on this document pkgs/top-level/ refers to ~/devel/nixpkgs/pkgs/top-level/
  • This looks perhaps too verbose, but I feel this extra context would have clarified many ambiguous explanations to me.
  • Also, when talking about directories I will try to always include the trailing slash.

Following the white rabbit

  • This section does not apply for packages in the directory pkgs/by-name. For them, see the included pkgs/by-name/README.md.
  • I will center this tutorial on the nix expression that packages the i3 window manager, as this expression is quite simple and it was contributing to this expression that I got to understand this whole workflow I am documenting.
  • In ~/devel/nixpkgs/default.nix there is an import statement that reads import ./pkgs/top-level/all-packages.nix
  • I have the feeling that most references between nix sources (nix expressions) in nixpkgs are stated as relative paths pointing up and down the nixpkgs/ directory hierarchy.
  • Please have a look at this pkgs/top-level/all-packages.nix. There are many more or less complex definitions at the beginning, and then a lot of declarations such as:
i3       = callPackage ../applications/window-managers/i3 { };
i3lock   = callPackage ../applications/window-managers/i3/lock.nix { };
i3status = callPackage ../applications/window-managers/i3/status.nix { };

and pango = callPackage ../development/libraries/pango/1.30.x.nix { };

  • Now let's have a look at the contents of pkgs/applications/window-managers/i3/:
$ ls -1 pkgs/applications/window-managers/i3/
default.nix
lock.nix
status.nix
  • The first assignment i3 = callPackage ... refers implicitly to default.nix on the pkgs/applications/window-managers/i3/ directory.
  • The second and third assignments are explicitly referring to lock.nix and status.nix files on this pkgs/applications/window-managers/i3/ directory.
  • Each of these three files contain somewhat equivalent nix expressions that the nix package manager will consider as packages thanks to the callPackage on pkgs/top-level/all-packages.nix.

The actual package "recipe"

  • We will dive (not too deep, though) on pkgs/applications/window-managers/i3/default.nix which is the nix expression that corresponds to the package for the i3 window manager:
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
  xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
  libXcursor, coreutils, perl, pango }:

stdenv.mkDerivation rec {
  name = "i3-${version}";
  version = "4.5.1";

  src = fetchurl {
    url = "http://i3wm.org/downloads/${name}.tar.bz2";
    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
  };

  buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];

  postPatch = ''
    patchShebangs
  '';

  configurePhase = "makeFlags=PREFIX=$out";

  meta = {
    description = "i3 is a tiling window manager";
    homepage = "http://i3wm.org";
    maintainers = [ stdenv.lib.maintainers.garbas ];
    license = stdenv.lib.licenses.bsd3;
  };
}
  • As an example of the trivial fixes and improvements I was talking about, you can have a look at two changesets (commits in the git terminology) I did:
  • As I anticipated above, I am not going to explain what's the purpose of the list of buildInputs, how a list enclosed by square brackets separated by spaces differs from the list at the beginning or the relevance of the colon that follows its closing curly brace. I just don't know so much yet.

Making changes

  • On your local clone of your forked repository you can edit whatever you want.
  • That is, in our example, under ~/devel/nixpkgs/
  • As you can see on the commits cited above I edited pkgs/applications/window-managers/i3/default.nix. I did several changes at once:
    • Added pango to both dependency lists. This pango refers to pango = callPackage ... in pkgs/top-level/all-packages.nix
    • Changed the version string.
    • Changed the checksum for the tarball on the sha256 string.
    • Removed a line on the postPatch string. This line was a sed command which was erasing some lines from the file common.mk, which is part of the sources in the tarball, before building the application.
  • And in an attempt to extract something positive from the issue, we are going to review an additional unfortunate change I made on pkgs/top-level/all-packages.nix
  • To figure out the checksum for the tarball you have the nix-prefetch-url utility. In addition to returning the sha256 checksum it will also leave the downloaded blob on your nix store, thus avoiding additional downloads when you test the build process. You get the path to the downloaded blob and the sha256sum on the last two lines of the command's output:
$ nix-prefetch-url 'http://i3wm.org/downloads/i3-4.5.1.tar.bz2'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  858k  100  858k    0     0   866k      0 --:--:-- --:--:-- --:--:--  867k
path is "/nix/store/clvs3jvyk24bskkn5pfd7fjpyaa1svsx-i3-4.5.1.tar.bz2"
1wi3p1s4z7r6kvzwxlg59fx6bfiqc2mlybhqmqddf8aaghf5zrds
  • Almost out of scope here, but you can easily review your changes. Go get some git documentation if you need: $ git diff
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
 { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
+  libXcursor, coreutils, perl, pango }:
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
-  version = "4.5";
+  version = "4.5.1";
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
-    sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
 
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
   '';
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0d3e4cc..a1c0d4b 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7224,7 +7224,9 @@ let
 
   hydrogen = callPackage ../applications/audio/hydrogen { };
 
-  i3 = callPackage ../applications/window-managers/i3 { };
+  i3 = callPackage ../applications/window-managers/i3 {
+    cairo = cairo.override { xcbSupport = true; };
+  };
 
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
     cairo = cairo.override { xcbSupport = true; };

Testing the changes

  • The changes made to these nix expressions are affecting the build process and therefore the behavior of the resulting binaries, configuration and other support files (man pages, scripts, etc.). Additionally there might be other packages or system services that depend on the ones we are altering, so these changes should be carefully considered, reviewed and thoroughly tested before they can be submitted and accepted on the official nixpkgs repository.
  • The worst case scenario is that a lot of people will get a broken package (or a broken system, even unable to boot) if the changes have unexpected side effects or if the changes are wrong and are reviewed too lightly. In special this happens more often for architectures other than 32 and 64-bit x86, because these are not so popular and thus less known and tested.
  • If your current directory is, as we said, ~/devel/nixpkgs/, you can request nix to build the modified package (see man nix-build):
nix-build -A i3
  • In this invocation the i3 in -A i3 refers to i3 = callPackage ... in pkgs/top-level/all-packages.nix. In the same way as the pango we talked about above, both names are attributes from the top-level Nix expression being evaluated. In this case ~/devel/nixpkgs/default.nix
  • Here I would have detected the error I introduced on pkgs/top-level/all-packages.nix. Too bad I didn't still know how to test my changes:
$ nix-build -A i3
error: function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
(use `--show-trace' to show detailed location information)
  • Let's follow the advice by adding --show-trace to the nix-build invocation:
$ nix-build --show-trace -A i3 
error: while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:98:35':
while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:59:24':
while evaluating the builtin function `isAttrs':
function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
  • Not that I got any more clues from this extra step. But it turned out that there is no mention to cairo on pkgs/applications/window-managers/i3/default.nix so it was illegal to invoke the cairo.override ... thing inside the i3 = callPackage ... on pkgs/top-level/all-packages.nix:
i3 = callPackage ../applications/window-managers/i3 {
    cairo = cairo.override { xcbSupport = true; }; # WRONG!
  };
  • So I would have reverted my changes to pkgs/top-level/all-packages.nix. Take care with this command as it is quite eager to undo your changes. But that's out of scope here. See some git documentation:
git checkout pkgs/top-level/all-packages.nix
  • Now let's try again:
$ nix-build -A i3
/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1
  • When the build succeeds, nix-build will leave the result linked as ./result on the current working directory:
$ stat -c %N ./result
  File: "./result" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1"
  • If convenient, you can review and fiddle with the resulting stuff:
$ find result/ -type f
result/include/i3/ipc.h
result/bin/i3-input
result/bin/i3-nagbar
result/bin/i3-msg
result/bin/i3-migrate-config-to-v4
result/bin/i3-sensible-terminal
result/bin/i3-sensible-pager
result/bin/i3-sensible-editor
result/bin/i3bar
result/bin/i3
result/bin/i3-config-wizard
result/bin/i3-dmenu-desktop
result/bin/i3-dump-log
result/etc/i3/config.keycodes
result/etc/i3/config
result/share/xsessions/i3.desktop
result/share/applications/i3.desktop
  • And for some real-life testing we would invoke nix-env as explained above:
$ nix-env -f . -iA i3
replacing old `i3-4.5.1'
installing `i3-4.5.1'
building path(s) `/nix/store/49ag28h77sjmg3rygpsm83jxkhn5jzyn-user-environment'
created 2006 symlinks in user environment
  • You might double-check that the binaries on $PATH are indeed the new ones:
$ stat `type -P i3` | head -n 1
  File: "~/.nix-profile/bin/i3" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1/bin/i3"
  • It is quite annoying to test a window manager, i3 in special as it does not implement the --replace feature.
  • And sometimes there are lots of other tricky possibilities. For example i3 invokes other binaries from the same package, such as i3bar. I lost a great deal of time trying to figure out why the i3 status bar was refusing to render pango fonts. That was because I was launching ~/devel/nixpkgs/result/bin/i3 instead of having the whole package replaced on my nix user environment. And the old i3bar from the nix store was on my $PATH. Obviously that was before I learned the nix-env invocation shown above.

Pushing the local changes to GitHub

Committing to the local repository

  • The first step is to commit your local changes to the local repository.
  • A quick overview of the affected stuff:
$ git status
# On branch example
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   pkgs/applications/window-managers/i3/default.nix
#       modified:   pkgs/top-level/all-packages.nix
#
no changes added to commit (use "git add" and/or "git commit -a")
  • As we have just seen, you can review the pending changes with git diff
  • You can limit the scope to review piece by piece: $ git diff pkgs/top-level/all-packages.nix
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0d3e4cc..a1c0d4b 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7224,7 +7224,9 @@ let
 
   hydrogen = callPackage ../applications/audio/hydrogen { };
 
-  i3 = callPackage ../applications/window-managers/i3 { };
+  i3 = callPackage ../applications/window-managers/i3 {
+    cairo = cairo.override { xcbSupport = true; };
+  };
 
   i3lock = callPackage ../applications/window-managers/i3/lock.nix {
     cairo = cairo.override { xcbSupport = true; };
  • This change, at the time, looked good to me. So I went ahead committing it:
$ git commit pkgs/top-level/all-packages.nix -m 'i3 window manager: cairo.override with xcbSupport enabled'
[example 731abb2] i3 window manager: cairo.override with xcbSupport enabled
 1 file changed, 3 insertions(+), 1 deletion(-)
  • Let's review what's left: $ git diff pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
 { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
+  libXcursor, coreutils, perl, pango }:
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
-  version = "4.5";
+  version = "4.5.1";
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
-    sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
 
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
   '';
  • These changes were made with two different purposes. I don't like commits mixing unrelated changes. It turns out that git can help here saving a lot of time and potential human error: git commit --patch pkgs/applications/window-managers/i3/default.nix.
  • git will sequentially show "groups of changes" (it calls them hunks). For each hunk we are asked what we want to do; we can do several things than are displayed by typing a question mark. I will explain with my words the ones we are interested in right now. Obviously the explanation given by git prevails:
    • type y to include the changes on this hunk in the commit
    • type n to leave the changes on this hunk out of this commit
    • type s to split the hunk if it consists of unrelated changes. You are then asked again about the smaller hunks.
  • Note that this command is not destructive. Changes left out of the commit are not discarded.
  • So I will do a first commit with all changes related to enabling pango support for i3. Please note how I explain git to s split the big hunk, y include the first small hunk on the commit, n skip the second hunk, n skip the third hunk, y and y to include the last two hunks:

$ git commit --patch pkgs/applications/window-managers/i3/default.nix -m 'i3 window manager: enable Pango support for anti-aliased fonts'

diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
 { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
+  libXcursor, coreutils, perl, pango }:
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
-  version = "4.5";
+  version = "4.5.1";
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
-    sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
 
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
   '';
 
Stage this hunk [y,n,q,a,d,/,s,e,?]? s
Split into 5 hunks.
@@ -1,6 +1,6 @@
 { fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
   xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
-  libXcursor, coreutils, perl }:
+  libXcursor, coreutils, perl, pango }:
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y
@@ -4,7 +4,7 @@
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
-  version = "4.5";
+  version = "4.5.1";
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
@@ -8,7 +8,7 @@
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
-    sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
@@ -12,6 +12,6 @@
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
-    libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+    libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
 
   postPatch = ''
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? y
@@ -16,6 +16,5 @@
 
   postPatch = ''
-    sed -i -e '/^# Pango/,/^$/d' common.mk
     patchShebangs .
   '';
 
Stage this hunk [y,n,q,a,d,/,K,g,e,?]? y

[example 729796b] i3 window manager: enable Pango support for anti-aliased fonts
 1 file changed, 2 insertions(+), 3 deletions(-)
  • As expected, the skipped hunks are still on the local file, uncommitted:

$ git diff pkgs/applications/window-managers/i3/default.nix

diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index b977a6a..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -4,11 +4,11 @@
 
 stdenv.mkDerivation rec {
   name = "i3-${version}";
-  version = "4.5";
+  version = "4.5.1";
 
   src = fetchurl {
     url = "http://i3wm.org/downloads/${name}.tar.bz2";
-    sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+    sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
   };
 
   buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
  • As these changes are both related, we can issue a simple commit:
$ git commit pkgs/applications/window-managers/i3/default.nix -m "i3 window manager: version bump from 4.5 to 4.5.1"
[example 4c7daed] i3 window manager: version bump from 4.5 to 4.5.1
 1 file changed, 2 insertions(+), 2 deletions(-)
  • Let's check if we're done:
$ git status
# On branch example
# Your branch is ahead of 'origin/example' by 3 commit.
#   (use "git push" to publish your local commits)
#
nothing to commit, working directory clean

Pushing to GitHub

  • Now we want to push the local commits to our clone of NixOS/nixpkgs at GitHub:
$ git push 
Counting objects: 27, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (19/19), 1.78 KiB, done.
Total 19 (delta 14), reused 14 (delta 12)
To https://github.com/your_username/nixpkgs.git
   1c86db4..4c7daed  example -> example

Sending a pull request to the official nixpkgs repository

step by step guide: pull request

  • Now there are 3 commits on the example branch of out nixpkgs repository at GitHub.
  • The idea is to propose the official NixOS/nixpkgs maintainers to include our changes on the official repository, so all the nixpkgs users can eventually benefit of our improvements.
  • We should probably get in touch with the nixpkgs maintainers at #nixos to figure out the appropriate branch (e.g. master, x-updates or whatever) where the changes would be welcome.
  • So now we have to formally request them to pull our changes, from our branch to the appropriate branch in NixOS/nixpkgs. Hence the name "pull request" which I don't feel so intuitive.
  • There is a "Pull Request" button in GitHub. Push it. Go ahead.
  • Now you will be asked to choose the origin repository and branch (branch example from our repository) and the destination repository and branch (e.g. branch master) on NixOS/nixpkgs).
  • You should also enter a subject and a small abstract.
  • And you have a last chance to review the commits that will be included on the pull request and the affected files.
  • If you're ready to go, there is a green button that reads "send pull request".
  • That's it :)

Waiting for your changes to appear on the nix channel

  • If your pull request is accepted, your changes will be part of NixOS/nixpkgs.
  • Or else you will have some kind of feedback, probably via GitHub and the nix-dev mailing list. You can follow up there or drop by #nixos to discuss whatever has to be discussed.
  • Once the changes enter the official repository they are more likely to be seen and probably used by other users.
  • In addition to that, if they enter the master branch, Hydra will eventually build the improved versions or new packages, and then the resulting binaries will be available on the NixOS channel.

See also

Category:Tutorial Category:Nixpkgs Category:Contributing