NixOS Wiki

Packaging/Quirks and Caveats: Difference between revisions

← Older edit
Visual
imported>Mic92
No edit summary
m update url to wayback machine url
 
(20 intermediate revisions by 8 users not shown)
Line 1: Line 1:
This article is about the quirks on how to package software where the source code is available.
A good start for packaging your first piece if software is the [http://nixos.org/nixpkgs/manual/#chap-quick-start Quickstart Chapter in the nixpkgs manual]
A good start for packaging your first piece if software is the [http://nixos.org/nixpkgs/manual/#chap-quick-start Quickstart Chapter in the nixpkgs manual]
Also see the [[Generic_Algorithm_on_Doing_Packaging|Generic Algorithm on doing Packaging]]
Also see the [[Generic_Algorithm_on_Doing_Packaging|Generic Algorithm on doing Packaging]]


For packaging executable without building them from source check out the article [[Packaging Binaries]].


== Build software with Autotools ==
== Build software with Autotools ==


Add <code>autoreconfHook</code> to <code>buildInputs</code> to automatically build software which uses <code>automake</code> and <code>autoconf</code>:
Add <code>autoreconfHook</code> to <code>nativeBuildInputs</code> to automatically build software which uses <code>automake</code> and <code>autoconf</code>:


<pre> buildInputs = [ ...  autoreconfHook ];</pre>
<syntaxhighlight lang="nix">
Examples in nixpkgs: * [https://github.com/NixOS/nixpkgs/blob/f4c253ff2f68fbe3e302f944e8347233d9dc8c9d/pkgs/tools/networking/samplicator/default.nix samplicator]
nativeBuildInputs = [ ...  autoreconfHook ];
</syntaxhighlight>
Examples in nixpkgs:  
* [https://github.com/NixOS/nixpkgs/blob/f4c253ff2f68fbe3e302f944e8347233d9dc8c9d/pkgs/tools/networking/samplicator/default.nix samplicator]


=== Package simple python scripts ===
== Configure Scripts that are using pkg-config ==


For scripts like a single Python file, it is not necessary to specify <code>src</code> in <code>mkDerivation</code>. When you want to use <code>buildPythonPackage</code> the sources need to provide a <code>setup.py</code> file which also is overkill for a lot of projects. The default <code>mkDerivation</code> will attempt to unpack your source code. This can be prevented that by applying <code>unpackPhase = &quot;:&quot;;</code> (<code>:</code> is a no-op in shell scripts).
Some configure scripts are using <code>pkg-config</code> to determine the location of libraries and headers. Nixpkgs supports this by adding <code>pkg-config</code> to <code>nativeBuildInputs</code>  
<syntaxhighlight lang="nix">
nativeBuildInputs = [ ... pkg-config ];
</syntaxhighlight>
Examples in nixpkgs:
* [https://github.com/NixOS/nixpkgs/blob/96d41e393da3ca27fbcc7c82b7221a5c923460c0/pkgs/development/libraries/libmms/default.nix#L13 libmms]


<pre class="nix">myscript-package = pkgs.stdenv.mkDerivation {
One typical error when <code>pkg-config</code> is required but not in the <code>nativeBuildInputs</code> is the following during the configure phase:
   name = &quot;myscript&quot;;
<syntaxhighlight lang="shell">
./configure: line 20832: syntax error near unexpected token `nss,'
./configure: line 20832: `                      PKG_CHECK_MODULES(nss, nss)'
</syntaxhighlight>
== Package simple python scripts ==
 
For scripts like a single Python file, it is not necessary to specify <code>src</code> in <code>mkDerivation</code>. When you want to use <code>buildPythonPackage</code> the sources need to provide a <code>setup.py</code> file which also is overkill for a lot of projects. The default <code>mkDerivation</code> will attempt to unpack your source code. This can be prevented by applying <code>unpackPhase = ":";</code> (<code>:</code> is a no-op in shell scripts).
 
<syntaxhighlight lang="nix">myscript-package = pkgs.stdenv.mkDerivation {
   name = "myscript";
   buildInputs = [
   buildInputs = [
     (pkgs.python36.withPackages (pythonPackages: with pythonPackages; [
     (pkgs.python36.withPackages (pythonPackages: with pythonPackages; [
Line 23: Line 43:
     ]))
     ]))
   ];
   ];
   unpackPhase = &quot;:&quot;;
   unpackPhase = ":";
   installPhase = &quot;install -m755 -D ${./myscript.py} $out/bin/myscript&quot;;
   installPhase = "install -m755 -D ${./myscript.py} $out/bin/myscript";
};</pre>
};</syntaxhighlight>
<code>nix</code> will automatically replace shebangs, for ex. <code>#!/usr/bin/env python3</code> with dependencies given in <code>buildInputs</code>. As the derivation got <code>pkgs.python36.withPackages (...)</code> in <code>buildInputs</code>, it will create a [https://virtualenv.pypa.io/en/stable/ virtualenv]-like python wrapper. The python wrapper will have all specified dependencies and will be used to call the script.
<code>stdenv</code>'s <code>patchShebangs</code> will automatically replace shebangs in the fixup phase, for ex. <code>#!/usr/bin/env python3</code> with dependencies given in <code>buildInputs</code>. As the derivation got <code>pkgs.python36.withPackages (...)</code> in <code>buildInputs</code>, it will create a [https://virtualenv.pypa.io/en/stable/ virtualenv]-like python wrapper. The python wrapper will have all specified dependencies and will be used to call the script.


In NixOS, the package can be put into <code>environment.systemPackages</code>, and <code>myscript</code> will be available as a global command.
In NixOS, the package can be put into <code>environment.systemPackages</code>, and <code>myscript</code> will be available as a global command.
Line 32: Line 52:
Source: [http://stackoverflow.com/questions/43837691/how-to-package-a-single-python-script-with-nix/43837692#43837692 nh2 @ StackOverflow]
Source: [http://stackoverflow.com/questions/43837691/how-to-package-a-single-python-script-with-nix/43837692#43837692 nh2 @ StackOverflow]


A more lightweight alternative is to use <code>nix-shell</code> in the shebang line as described in this [https://gist.github.com/travisbhartwell/f972aab227306edfcfea gist]
A more lightweight alternative is to use <code>nix-shell</code> in the shebang line as described in this [https://web.archive.org/web/20230330010914/http://iam.travishartwell.net/2015/06/17/nix-shell-shebang/ blog post]. This causes the expression to be evaluated and built every time the script is run; this means that the dependencies will always be kept up to date, but since nix-shell only creates a temporary GC root the dependencies may be removed by a garbage collection, so this approach is not advisable for users who don't have an internet connection available all the time.


== Caveats ==
== Caveats ==
Line 38: Line 58:
After packaging software and successfully generating an executable some functions of the package might still not work. This is a collection of error and how to fix them:
After packaging software and successfully generating an executable some functions of the package might still not work. This is a collection of error and how to fix them:


=== GLib-GIO-Message: Using the 'memory' GSettings backend.  Your settings will not be saved or shared with ===
Sometime the error mesage might be also:


=== GLib-GIO-Message: Using the 'memory' GSettings backend.  Your settings will not be saved or shared with ===
<syntaxHighlight lang="nix">
GLib-GIO-ERROR **: No GSettings schemas are installed on the system
</syntaxHighlight>


Fixed by adding <code>wrapGAppsHook</code> to buildInputs:
Fixed by adding <code>wrapGAppsHook</code> to buildInputs:


<syntaxHighlight lang="nix">
<syntaxHighlight lang="nix">
buildInputs = [ ...  wrapGAppsHook ];
nativeBuildInputs = [ ...  wrapGAppsHook ];
</syntaxHighlight>
</syntaxHighlight>
Sample PR in nixpkgs:  
Sample PR in nixpkgs:  
* [https://github.com/NixOS/nixpkgs/pull/24617/files networkmanagerapplet]
* [https://github.com/NixOS/nixpkgs/pull/24617/files networkmanagerapplet]


=== Namespace Gdk not available ===
You will need <code>/nix/store/*-gtk+3-*/lib/girepository-1.0</code> in <code>GI_TYPELIB_PATH</code>.
Similar solution as above, solved by:
<syntaxHighlight lang=nix>
  nativeBuildInputs = [ ...  wrapGAppsHook ];
  buildInputs = [ gtk3 ];
</syntaxHighlight>
=== ImportError: libstdc++.so.6: cannot open shared object file: No such file ===
This can happen when importing python libraries:
Solution: add <code>${stdenv.cc.cc.lib}/lib/libstdc++.so.6</code> to the <code>LD_LIBRARY_PATH</code>.
A sample <code>shell.nix</code>:
<syntaxHighlight lang="nix">
{ pkgs ? (import <nixpkgs> {}).pkgs }:
with pkgs;
mkShell {
  buildInputs = [
    python3Packages.virtualenv # run virtualenv .
    python3Packages.pyqt5 # avoid installing via pip
    python3Packages.pyusb # fixes the pyusb 'No backend available' when installed directly via pip
  ];
  shellHook = ''
    # fixes libstdc++ issues and libgl.so issues
    LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:/run/opengl-driver/lib/
    # fixes xcb issues :
    QT_PLUGIN_PATH=${qt5.qtbase}/${qt5.qtbase.qtPluginPrefix}
  '';
}
</syntaxHighlight>
=== Test cannot access <code>/etc/protocols</code>, <code>/etc/services</code> or expects a special <code>/etc/passwd</code> when building in sandbox ===


For packaged python libs: <code>ImportError: libstdc++.so.6: cannot open shared object file: No such file</code>
Sometimes libraries try to fetch protocol specs via <code>socket.getprotobyname('tcp')</code> which fails in sandboxes because /etc/protocols is unaccessible. Override pre- and postCheck phases with this:
<syntaxHighlight lang="nix">
    preCheck = ''
      export NIX_REDIRECTS=/etc/protocols=${pkgs.iana-etc}/etc/protocols \
        LD_PRELOAD=${pkgs.libredirect}/lib/libredirect.so
    '';
    postCheck = ''
      unset NIX_REDIRECTS LD_PRELOAD
    '';
</syntaxHighlight>


Solution: According to the IRC, add <code>${stdenv.cc.cc.lib}/lib/libstdc++.so.6</code> to the search path.
[[Category:Cookbook]]
Retrieved from "https://wiki.nixos.org/wiki/Packaging/Quirks_and_Caveats"