Packaging/Quirks and Caveats: Difference between revisions

From NixOS Wiki
imported>Makefu
m formatting
m update url to wayback machine url
 
(5 intermediate revisions by 5 users not shown)
Line 16: Line 16:
* [https://github.com/NixOS/nixpkgs/blob/f4c253ff2f68fbe3e302f944e8347233d9dc8c9d/pkgs/tools/networking/samplicator/default.nix samplicator]
* [https://github.com/NixOS/nixpkgs/blob/f4c253ff2f68fbe3e302f944e8347233d9dc8c9d/pkgs/tools/networking/samplicator/default.nix samplicator]


== Configure Scripts that are using pkgconfig ==
== Configure Scripts that are using pkg-config ==


Some configure scripts are using <code>pkgconfig</code> to determine the location of libraries and headers. Nixpkgs supports this by adding <code>pkgconfig</code> to <code>nativeBuildInputs</code>  
Some configure scripts are using <code>pkg-config</code> to determine the location of libraries and headers. Nixpkgs supports this by adding <code>pkg-config</code> to <code>nativeBuildInputs</code>  
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
nativeBuildInputs = [ ...  pkgconfig ];
nativeBuildInputs = [ ...  pkg-config ];
</syntaxhighlight>
</syntaxhighlight>
Examples in nixpkgs:  
Examples in nixpkgs:  
* [https://github.com/NixOS/nixpkgs/blob/96d41e393da3ca27fbcc7c82b7221a5c923460c0/pkgs/development/libraries/libmms/default.nix#L13 libmms]
* [https://github.com/NixOS/nixpkgs/blob/96d41e393da3ca27fbcc7c82b7221a5c923460c0/pkgs/development/libraries/libmms/default.nix#L13 libmms]


One typical error when <code>pkg-config</code> is required but not in the <code>nativeBuildInputs</code> is the following during the configure phase:
<syntaxhighlight lang="shell">
./configure: line 20832: syntax error near unexpected token `nss,'
./configure: line 20832: `                      PKG_CHECK_MODULES(nss, nss)'
</syntaxhighlight>
== Package simple python scripts ==
== Package simple python scripts ==


For scripts like a single Python file, it is not necessary to specify <code>src</code> in <code>mkDerivation</code>. When you want to use <code>buildPythonPackage</code> the sources need to provide a <code>setup.py</code> file which also is overkill for a lot of projects. The default <code>mkDerivation</code> will attempt to unpack your source code. This can be prevented that by applying <code>unpackPhase = ":";</code> (<code>:</code> is a no-op in shell scripts).
For scripts like a single Python file, it is not necessary to specify <code>src</code> in <code>mkDerivation</code>. When you want to use <code>buildPythonPackage</code> the sources need to provide a <code>setup.py</code> file which also is overkill for a lot of projects. The default <code>mkDerivation</code> will attempt to unpack your source code. This can be prevented by applying <code>unpackPhase = ":";</code> (<code>:</code> is a no-op in shell scripts).


<syntaxhighlight lang="nix">myscript-package = pkgs.stdenv.mkDerivation {
<syntaxhighlight lang="nix">myscript-package = pkgs.stdenv.mkDerivation {
Line 47: Line 52:
Source: [http://stackoverflow.com/questions/43837691/how-to-package-a-single-python-script-with-nix/43837692#43837692 nh2 @ StackOverflow]
Source: [http://stackoverflow.com/questions/43837691/how-to-package-a-single-python-script-with-nix/43837692#43837692 nh2 @ StackOverflow]


A more lightweight alternative is to use <code>nix-shell</code> in the shebang line as described in this [http://iam.travishartwell.net/2015/06/17/nix-shell-shebang/ blog post]. This causes the expression to be evaluated and built every time the script is run; this means that the dependencies will always be kept up to date, but since nix-shell only creates a temporary GC root the dependencies may be removed by a garbage collection, so this approach is not advisable for users who don't have an internet connection available all the time.
A more lightweight alternative is to use <code>nix-shell</code> in the shebang line as described in this [https://web.archive.org/web/20230330010914/http://iam.travishartwell.net/2015/06/17/nix-shell-shebang/ blog post]. This causes the expression to be evaluated and built every time the script is run; this means that the dependencies will always be kept up to date, but since nix-shell only creates a temporary GC root the dependencies may be removed by a garbage collection, so this approach is not advisable for users who don't have an internet connection available all the time.


== Caveats ==
== Caveats ==
Line 82: Line 87:
This can happen when importing python libraries:  
This can happen when importing python libraries:  
Solution: add <code>${stdenv.cc.cc.lib}/lib/libstdc++.so.6</code> to the <code>LD_LIBRARY_PATH</code>.
Solution: add <code>${stdenv.cc.cc.lib}/lib/libstdc++.so.6</code> to the <code>LD_LIBRARY_PATH</code>.
A sample <code>shell.nix</code>:
<syntaxHighlight lang="nix">
{ pkgs ? (import <nixpkgs> {}).pkgs }:
with pkgs;
mkShell {
  buildInputs = [
    python3Packages.virtualenv # run virtualenv .
    python3Packages.pyqt5 # avoid installing via pip
    python3Packages.pyusb # fixes the pyusb 'No backend available' when installed directly via pip
  ];
  shellHook = ''
    # fixes libstdc++ issues and libgl.so issues
    LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:/run/opengl-driver/lib/
    # fixes xcb issues :
    QT_PLUGIN_PATH=${qt5.qtbase}/${qt5.qtbase.qtPluginPrefix}
  '';
}
</syntaxHighlight>


=== Test cannot access <code>/etc/protocols</code>, <code>/etc/services</code> or expects a special <code>/etc/passwd</code> when building in sandbox ===
=== Test cannot access <code>/etc/protocols</code>, <code>/etc/services</code> or expects a special <code>/etc/passwd</code> when building in sandbox ===
Line 96: Line 120:
</syntaxHighlight>
</syntaxHighlight>


[[Category:Guide]]
[[Category:Cookbook]]

Latest revision as of 06:19, 14 September 2024

This article is about the quirks on how to package software where the source code is available.

A good start for packaging your first piece if software is the Quickstart Chapter in the nixpkgs manual Also see the Generic Algorithm on doing Packaging

For packaging executable without building them from source check out the article Packaging Binaries.

Build software with Autotools

Add autoreconfHook to nativeBuildInputs to automatically build software which uses automake and autoconf:

nativeBuildInputs = [ ...  autoreconfHook ];

Examples in nixpkgs:

Configure Scripts that are using pkg-config

Some configure scripts are using pkg-config to determine the location of libraries and headers. Nixpkgs supports this by adding pkg-config to nativeBuildInputs

nativeBuildInputs = [ ...  pkg-config ];

Examples in nixpkgs:

One typical error when pkg-config is required but not in the nativeBuildInputs is the following during the configure phase:

./configure: line 20832: syntax error near unexpected token `nss,'
./configure: line 20832: `                      PKG_CHECK_MODULES(nss, nss)'

Package simple python scripts

For scripts like a single Python file, it is not necessary to specify src in mkDerivation. When you want to use buildPythonPackage the sources need to provide a setup.py file which also is overkill for a lot of projects. The default mkDerivation will attempt to unpack your source code. This can be prevented by applying unpackPhase = ":"; (: is a no-op in shell scripts).

myscript-package = pkgs.stdenv.mkDerivation {
  name = "myscript";
  buildInputs = [
    (pkgs.python36.withPackages (pythonPackages: with pythonPackages; [
      consul
      six
      requests
    ]))
  ];
  unpackPhase = ":";
  installPhase = "install -m755 -D ${./myscript.py} $out/bin/myscript";
};

stdenv's patchShebangs will automatically replace shebangs in the fixup phase, for ex. #!/usr/bin/env python3 with dependencies given in buildInputs. As the derivation got pkgs.python36.withPackages (...) in buildInputs, it will create a virtualenv-like python wrapper. The python wrapper will have all specified dependencies and will be used to call the script.

In NixOS, the package can be put into environment.systemPackages, and myscript will be available as a global command.

Source: nh2 @ StackOverflow

A more lightweight alternative is to use nix-shell in the shebang line as described in this blog post. This causes the expression to be evaluated and built every time the script is run; this means that the dependencies will always be kept up to date, but since nix-shell only creates a temporary GC root the dependencies may be removed by a garbage collection, so this approach is not advisable for users who don't have an internet connection available all the time.

Caveats

After packaging software and successfully generating an executable some functions of the package might still not work. This is a collection of error and how to fix them:

GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with

Sometime the error mesage might be also:

GLib-GIO-ERROR **: No GSettings schemas are installed on the system

Fixed by adding wrapGAppsHook to buildInputs:

nativeBuildInputs = [ ...  wrapGAppsHook ];

Sample PR in nixpkgs:

Namespace Gdk not available

You will need /nix/store/*-gtk+3-*/lib/girepository-1.0 in GI_TYPELIB_PATH.

Similar solution as above, solved by:

  nativeBuildInputs = [ ...  wrapGAppsHook ];
  buildInputs = [ gtk3 ];

ImportError: libstdc++.so.6: cannot open shared object file: No such file

This can happen when importing python libraries: Solution: add ${stdenv.cc.cc.lib}/lib/libstdc++.so.6 to the LD_LIBRARY_PATH.

A sample shell.nix:

{ pkgs ? (import <nixpkgs> {}).pkgs }:
with pkgs;
mkShell {
  buildInputs = [
    python3Packages.virtualenv # run virtualenv .
    python3Packages.pyqt5 # avoid installing via pip
    python3Packages.pyusb # fixes the pyusb 'No backend available' when installed directly via pip
  ];
  shellHook = ''
    # fixes libstdc++ issues and libgl.so issues
    LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:/run/opengl-driver/lib/
    # fixes xcb issues :
    QT_PLUGIN_PATH=${qt5.qtbase}/${qt5.qtbase.qtPluginPrefix}
  '';
}

Test cannot access /etc/protocols, /etc/services or expects a special /etc/passwd when building in sandbox

Sometimes libraries try to fetch protocol specs via socket.getprotobyname('tcp') which fails in sandboxes because /etc/protocols is unaccessible. Override pre- and postCheck phases with this:

     preCheck = ''
       export NIX_REDIRECTS=/etc/protocols=${pkgs.iana-etc}/etc/protocols \
         LD_PRELOAD=${pkgs.libredirect}/lib/libredirect.so
     '';
     postCheck = ''
       unset NIX_REDIRECTS LD_PRELOAD
     '';