Packaging/Binaries: Difference between revisions

From NixOS Wiki
imported>Makefu
No edit summary
m update url to wayback machine url
 
(45 intermediate revisions by 18 users not shown)
Line 1: Line 1:
Downloading a binary on NixOS and trying to run it will almost never work. This is due to hard-coded paths in the executable.
Downloading and attempting to run a binary on NixOS will almost never work. This is due to hard-coded paths in the executable. Unfortunately, almost all unfree and proprietary software comes in binary form - the main reason to include binaries is because no source code is available.
This tutorial will guide you through packaging a binary executable.
This tutorial will guide you through packaging a binary executable.
== Starting Point ==
 
If a downloaded binary is not packaged for NixOS then running the binary may fail like the following:
 
<syntaxHighlight lang=bash>
$ ~/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu  --help
bash: /home/gdforj/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu: cannot execute: required file not found
</syntaxHighlight>
 
Which indicates the interpreter, [[#The Dynamic Loader|the dynamic loader]] in this case, could not be located.
 
== Using AutoPatchelfHook ==
<code>autoPatchelfHook</code> is the current (as of 19.09) preferred way to package binaries.
 
<syntaxHighlight lang=nix>
{ stdenv, lib
, fetchurl
, alsaLib
, openssl
, zlib
, pulseaudio
, autoPatchelfHook
}:
 
stdenv.mkDerivation rec {
  pname = "studio-link";
  version = "21.07.0";
 
  src = fetchurl {
    url = "https://download.studio.link/releases/v${version}-stable/linux/studio-link-standalone-v${version}.tar.gz";
    hash = "sha256-4CkijAlenhht8tyk3nBULaBPE0GBf6DVII699/RmmWI=";
  };
 
  nativeBuildInputs = [
    autoPatchelfHook
  ];
 
  buildInputs = [
    alsaLib
    openssl
    zlib
    pulseaudio
  ];
 
  sourceRoot = ".";
 
  installPhase = ''
    runHook preInstall
    install -m755 -D studio-link-standalone-v${version} $out/bin/studio-link
    runHook postInstall
  '';
 
  meta = with lib; {
    homepage = "https://studio-link.com";
    description = "Voip transfer";
    platforms = platforms.linux;
  };
}
</syntaxHighlight>
 
See [https://github.com/NixOS/nixpkgs/commit/ea5787ad5291ee1c131326cb9c9fec03d359edff this commit], or the example in method 5 of [https://unix.stackexchange.com/a/522823 this answer], for more details.
 
=== Additional Library Paths ===
Some packages may put their library files somewhere other than <code>$out/lib/</code> in the install phase, while <code>autoPatchelfHook</code> by default only looks in that one directory for every package in <code>buildInputs</code>.  You can tell <code>autoPatchelfHook</code> to look in additional directories by passing it to the command <code>addAutoPatchelfSearchPath</code> in any phase before <code>autoPatchelfHook</code> is run.
 
For example,
<pre>
  preBuild = ''
    addAutoPatchelfSearchPath ${jre8}/lib/openjdk/jre/lib/
  '';
</pre>
 
== Manual Method ==
This sections describes the manual method of packaging a binary. It serves as a reference on the packaging issue with binaries and how these issues can be solved in nix derivations.
This tutorial is about how to patch the executable with <code>patchelf</code> which is sufficient for most cases. If no source is available for a program patchelf is the preferred way in nixpkgs to add support for the package.
 
However sometimes this is not enough, more hardcoded paths may be scattered all over the place. For this you may need to set up an FHSUserEnv, a Linux Standard Base style directory structure with a final <code>chroot</code> call to fixate all paths. For a tutorial on how to use this technique, check out [https://web.archive.org/web/20240125095859/http://reflexivereflection.com/posts/2015-02-28-deb-installation-nixos.html Anders Papittos blog post on installing debian packages in nixos].
=== Starting Point ===
We want to package a tool called "MasterPDFEditor", the package for debian can be found at [http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb] ([https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb archive.org mirror]).
We want to package a tool called "MasterPDFEditor", the package for debian can be found at [http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb] ([https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb archive.org mirror]).
This tutorial assumes you run nixos-17.09 or later (for <code>nix-index</code> to work).  
This tutorial assumes you run nixos-17.09 or later (for <code>nix-index</code> to work).  
Line 10: Line 86:
   stdenv.cc # stdenv.cc is required for setting $NIX_CC env
   stdenv.cc # stdenv.cc is required for setting $NIX_CC env
$ wget https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb
$ wget https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb
# we extract data.tar.xz from the deb package and untar it
$ # we extract data.tar.xz from the deb package and untar it
$ ar p master-pdf-editor-4.3.10_qt5.amd64.deb data.tar.xz | tar xJ
$ dpkg-deb -x master-pdf-editor-4.3.10_qt5.amd64.deb .
ls opt/master-pdf-editor-4/
$ ls opt/master-pdf-editor-4/
fonts  license.txt      masterpdfeditor4.png  templates
fonts  license.txt      masterpdfeditor4.png  templates
lang  masterpdfeditor4  stamps
lang  masterpdfeditor4  stamps
# running the executable does not `just work`
$ # running the executable does not `just work`
$ opt/master-pdf-editor-4/masterpdfeditor4
$ opt/master-pdf-editor-4/masterpdfeditor4
bash: opt/master-pdf-editor-4/masterpdfeditor4: No such file or directory
bash: opt/master-pdf-editor-4/masterpdfeditor4: No such file or directory
</syntaxHighlight>
</syntaxHighlight>
== The Dynamic Loader ==
== The Dynamic Loader ==


The binary has the dynamic loader ("ELF interpreter") set to a static path which is in general not available under NixOS: <code>/lib64/ld-linux-x86-64.so.2</code>
The binary has the dynamic loader ("ELF interpreter") set to a static path which is in general not available under NixOS: <code>/lib64/ld-linux-x86-64.so.2</code>


We can use patchelf to show set the library path and dynamic linker appropriately:
We can use patchelf to show and set the library path and dynamic linker appropriately:
<syntaxHighlight lang="console">
<syntaxHighlight lang="console">
$ cd opt/master-pdf-editor-4/
$ cd opt/master-pdf-editor-4/
Line 64: Line 141:
libgcc_s.so.1
libgcc_s.so.1
libc.so.6
libc.so.6
# after we've patched the interpreter we can use ldd to find the libraries which are currently not found (check <code>=> not found</code>):
$ # now that the interpreter path is patched we can use ldd to find the libraries which are currently not found (check '=> not found'):
$ ldd masterpdfeditor4 | grep 'not found'
$ ldd masterpdfeditor4 | grep 'not found'
libsane.so.1 => not found
libsane.so.1 => not found
Line 78: Line 155:
All the libraries which are not yet found are the ones we need to add to the runtime search path of the executable (RPATH). Again we can use <code>patchelf</code> to do this. We will be using <code>nix-index</code> for finding the files we are looking for:
All the libraries which are not yet found are the ones we need to add to the runtime search path of the executable (RPATH). Again we can use <code>patchelf</code> to do this. We will be using <code>nix-index</code> for finding the files we are looking for:
<syntaxHighlight lang="console">
<syntaxHighlight lang="console">
# we generate the database index of all files in our channel first
$ # we generate the database index of all files in our channel first
$ nix-index
$ nix-index
+ querying available packages
+ querying available packages
Line 106: Line 183:
</syntaxHighlight>
</syntaxHighlight>
Unfortunately there is no "right" way to choose which package to actually take, you can check out other derivations by grepping in nixpkgs.
Unfortunately there is no "right" way to choose which package to actually take, you can check out other derivations by grepping in nixpkgs.
{{Tip|in case there are too many .so libraries, you may save time by automated approach, executing [https://github.com/lexleogryfon/de-generate nix-de-generate] for target application, which will produce list of nix packages (in nix expression) to satisfy dependencies of target application.}}


The next step is to create a library path for all these packages. We use nix-repl to resolve the paths:
The next step is to create a library path for all these packages. We use nix-repl to resolve the paths:
<syntaxHighlight lang="console">
<syntaxHighlight lang="console">
$ nix-repl
$ nix repl '<nixpkgs>'
# load all variables from nixpkgs
nix-repl> :l <nixpkgs>
# .out can be omitted because this is the default output for all packages
# .out can be omitted because this is the default output for all packages
# makeLibraryPath outputs the correct path for each package to use as rpath
# makeLibraryPath outputs the correct path for each package to use as rpath
Line 125: Line 201:
# SUCCESS!!!
# SUCCESS!!!
</syntaxHighlight>
</syntaxHighlight>
[[File:MasterPDFScreenshot.png|thumb|Screenshot of working masterpdf]]


== Creating the Derivation for upstream Packaging ==
== Creating the Derivation for upstream Packaging ==
Packaging is straight forward. We just have to add all the steps we did into a simple derivation file. We call it <code>default.nix</code> and store it in the checked out nixpkgs repository at <code>pkgs/applications/misc/master-pdf-editor</code>
Packaging is straight forward. We just have to add all the steps we did into a simple derivation file. We call it <code>package.nix</code> and store it in the checked out nixpkgs repository at <code>pkgs/by-name/ma/master-pdf-editor</code>
The content looks like this:
The content looks like this:
<syntaxHighlight lang="nix">
<syntaxHighlight lang="nix">
Line 137: Line 212:


   src = fetchurl {
   src = fetchurl {
      url = "http://get.code-industry.net/public/${name}_qt5.amd64.deb";
    url = "http://get.code-industry.net/public/master-pdf-editor-${version}_qt5.amd64.deb";
      sha256 = "1z26qjhbiyz33rm7mp8ycgl5ka0v3v5lv5i5v0b5mx35arvx2zzy";
    hash = "sha256-1z26qjhbiyz33rm7mp8ycgl5ka0v3v5lv5i5v0b5mx35arvx2zzy";
   };
   };
   sourceRoot = ".";
   sourceRoot = ".";
   unpackCmd = ''
   unpackCmd = "dpkg-deb -x master-pdf-editor-${version}_qt5.amd64.deb .";
      ar p "$src" data.tar.xz | tar xJ
 
   '';
  dontConfigure = true;
   dontBuild = true;


  buildPhase = ":";  # nothing to build
  libPath = lib.makeLibraryPath [
    qt5.qtbase        # libQt5PrintSupport.so.5
    qt5.qtsvg        # libQt5Svg.so.5
    stdenv.cc.cc.lib  # libstdc++.so.6
    saneBackends    # libsane.so.1
  ];
   installPhase = ''
   installPhase = ''
    runHook preInstall
     mkdir -p $out/bin
     mkdir -p $out/bin
     cp -R usr/share opt $out/
     cp -R usr/share opt $out/
     # fix the path in the desktop file
     # fix the path in the desktop file
     substituteInPlace \
     substituteInPlace \
        $out/share/applications/masterpdfeditor4.desktop \
      $out/share/applications/masterpdfeditor4.desktop \
        --replace /opt/ $out/opt/
      --replace /opt/ $out/opt/
     # symlink the binary to bin/
     # symlink the binary to bin/
     ln -s $out/opt/master-pdf-editor-4/masterpdfeditor4 $out/bin/masterpdfeditor4
     ln -s $out/opt/master-pdf-editor-4/masterpdfeditor4 $out/bin/masterpdfeditor4
    runHook postInstall
   '';
   '';
   postFixup = ''
   preFixup = let
    # we prepare our library path in the let clause to avoid it become part of the input of mkDerivation
    libPath = lib.makeLibraryPath [
      qt5.qtbase        # libQt5PrintSupport.so.5
      qt5.qtsvg        # libQt5Svg.so.5
      stdenv.cc.cc.lib  # libstdc++.so.6
      saneBackends      # libsane.so.1
    ];
  in ''
     patchelf \
     patchelf \
        --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
      --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
        --set-rpath "${libPath}" \
      --set-rpath "${libPath}" \
        $out/opt/master-pdf-editor-4/masterpdfeditor4
      $out/opt/master-pdf-editor-4/masterpdfeditor4
   '';
   '';


   meta = with stdenv.lib; {
   meta = with lib; {
     homepage = https://code-industry.net/masterpdfeditor/;
     homepage = https://code-industry.net/masterpdfeditor/;
     description = "a multifunctional PDF Editor";
     description = "a multifunctional PDF Editor";
     license = licenses.proprietary;
     license = licenses.unfree;
     platforms = platforms.linux;
     platforms = platforms.linux;
     maintainers = [ your_name ];
     maintainers = [ your_name ];
Line 180: Line 261:


Because we created a derivation which is meant to be called by callPackage we can build the package now only via:
Because we created a derivation which is meant to be called by callPackage we can build the package now only via:
<code>nix-build -E '((import <nixpkgs> {}).callPackage (import ./default.nix) { })' --keep-failed --no-out-link</code>
<code>nix-build -E '((import <nixpkgs> {}).callPackage (import ./package.nix) { })' --keep-failed --no-out-link</code>
 
== Creating a Pull Request ==


With this new package you can [[Pull request|create a pull request for nixpkgs]]. Be aware that binary distributions are frowned upon if the source is available.


== Add the package to nixpkgs ==
In order to add this new package to nixpkgs and be able to install it via <code>nix-build -A pkgs.master-pdf-editor</code> you need to add it to <code>pkgs/top-level/all-packages.nix</code>:
<syntaxHighlight lang=nix>
  ...
  masscan = callPackage ../tools/security/masscan { };


   master-pdf-editor = callPackage ../applications/misc/master-pdf-editor {};
== autoPatchelfHook ==
 
<code>autoPatchelfHook</code> can make the manual written patchelf invocations unnecessary. [[Category:Tutorial]]
[[Category:nixpkgs]]
 
== Wrong file paths ==
 
Some programs will try to access hard-coded FHS file paths like <code>/usr/lib</code> or <code>/opt</code>. Mostly though, this will produce silent <code>No such file</code> errors, which can break the program.
 
To make these errors visible, we can use <code>strace</code>
 
<pre>
strace --trace=file,process --follow-forks --string-limit=200 \
  ./result/bin/some-program 2>strace.log
 
cat strace.log | grep -e 'No such file' -e 'execve("' \
   | grep -v -E -e '(open|stat|access)(at)?\(.*"/nix/store/' \
    -e resumed -e '/etc/ld-nix.so.preload'
</pre>
 
=== Example ===
 
<pre>
[pid 357679] openat(AT_FDCWD, "/opt/brother/Printers/hll3210cw/inf/lut/0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 357679] openat(AT_FDCWD, "0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)
</pre>
 
This means: process <code>357679</code> is trying to open file <code>0600-k_cache17.bin</code> either from the hard-coded path in <code>/opt/brother/Printers/hll3210cw/inf/lut</code> or from the current workdir.
So, as a quick fix, we could change the working directory of the wrapped binary with [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/setup-hooks/make-wrapper.sh makeWrapper]:
 
<pre>
makeWrapper <executable> <out_path> --run 'cd /nix/store/path/to/workdir'
</pre>
 
=== Finding the failing process ===
How can we find which process is throwing the <code>No such file</code> error? Let's search for the process ID and the <code>exec</code> syscall:


  meson = ../development/tools/build-managers/meson { };
<pre>
  ...
cat strace.log | grep 357679 | grep exec
</syntaxHighlight>


== Creating a Pull Request ==
[pid 357679] execve("/nix/store/ybl1pacslmhci9zy5qv95hshdgz6ihjl-brother-hll3210cw-1.0.2-0/opt/brother/Printers/hll3210cw/lpd/brhll3210cwfilter", ["/nix/store/ybl1pacslmhci9zy5qv95"..., "-pi", "/nix/store/ybl1pacslmhci9zy5qv95"..., "-rc", "/nix/store/ybl1pacslmhci9zy5qv95"...], 0x8ba010 /* 132 vars */ <unfinished ...>
</pre>


With this new package you can [[Pull request|create a pull request for nixpkgs]]. Be aware that binary distributions are frowned upon if the source is available.
== See also ==


[[Category:Tutorial]]
* [[Steam#steam-run|steam-run]] as a quick way to run binaries
[[Category:nixpkgs]]

Latest revision as of 19:24, 14 September 2024

Downloading and attempting to run a binary on NixOS will almost never work. This is due to hard-coded paths in the executable. Unfortunately, almost all unfree and proprietary software comes in binary form - the main reason to include binaries is because no source code is available. This tutorial will guide you through packaging a binary executable.

If a downloaded binary is not packaged for NixOS then running the binary may fail like the following:

$ ~/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu  --help
bash: /home/gdforj/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu: cannot execute: required file not found

Which indicates the interpreter, the dynamic loader in this case, could not be located.

Using AutoPatchelfHook

autoPatchelfHook is the current (as of 19.09) preferred way to package binaries.

 { stdenv, lib
, fetchurl
, alsaLib
, openssl
, zlib
, pulseaudio
, autoPatchelfHook
}:

stdenv.mkDerivation rec {
  pname = "studio-link";
  version = "21.07.0";

  src = fetchurl {
    url = "https://download.studio.link/releases/v${version}-stable/linux/studio-link-standalone-v${version}.tar.gz";
    hash = "sha256-4CkijAlenhht8tyk3nBULaBPE0GBf6DVII699/RmmWI=";
  };

  nativeBuildInputs = [
    autoPatchelfHook
  ];

  buildInputs = [
    alsaLib
    openssl
    zlib
    pulseaudio
  ];

  sourceRoot = ".";

  installPhase = ''
    runHook preInstall
    install -m755 -D studio-link-standalone-v${version} $out/bin/studio-link
    runHook postInstall
  '';

  meta = with lib; {
    homepage = "https://studio-link.com";
    description = "Voip transfer";
    platforms = platforms.linux;
  };
}

See this commit, or the example in method 5 of this answer, for more details.

Additional Library Paths

Some packages may put their library files somewhere other than $out/lib/ in the install phase, while autoPatchelfHook by default only looks in that one directory for every package in buildInputs. You can tell autoPatchelfHook to look in additional directories by passing it to the command addAutoPatchelfSearchPath in any phase before autoPatchelfHook is run.

For example,

  preBuild = ''
    addAutoPatchelfSearchPath ${jre8}/lib/openjdk/jre/lib/
  '';

Manual Method

This sections describes the manual method of packaging a binary. It serves as a reference on the packaging issue with binaries and how these issues can be solved in nix derivations. This tutorial is about how to patch the executable with patchelf which is sufficient for most cases. If no source is available for a program patchelf is the preferred way in nixpkgs to add support for the package.

However sometimes this is not enough, more hardcoded paths may be scattered all over the place. For this you may need to set up an FHSUserEnv, a Linux Standard Base style directory structure with a final chroot call to fixate all paths. For a tutorial on how to use this technique, check out Anders Papittos blog post on installing debian packages in nixos.

Starting Point

We want to package a tool called "MasterPDFEditor", the package for debian can be found at [1] (archive.org mirror). This tutorial assumes you run nixos-17.09 or later (for nix-index to work).

Let's download the archive and unpack the archive in a nix shell for testing:

$ nix-shell -p binutils stdenv wget dpkg nix-index \
   stdenv.cc # stdenv.cc is required for setting $NIX_CC env
$ wget https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb
$ # we extract data.tar.xz from the deb package and untar it
$ dpkg-deb -x master-pdf-editor-4.3.10_qt5.amd64.deb .
$ ls opt/master-pdf-editor-4/
fonts  license.txt       masterpdfeditor4.png  templates
lang   masterpdfeditor4  stamps
$ # running the executable does not `just work`
$ opt/master-pdf-editor-4/masterpdfeditor4
bash: opt/master-pdf-editor-4/masterpdfeditor4: No such file or directory

The Dynamic Loader

The binary has the dynamic loader ("ELF interpreter") set to a static path which is in general not available under NixOS: /lib64/ld-linux-x86-64.so.2

We can use patchelf to show and set the library path and dynamic linker appropriately:

$ cd opt/master-pdf-editor-4/
# the current interpreter
$ patchelf --print-interpreter masterpdfeditor4
# all the needed libraries
/lib64/ld-linux-x86-64.so.2

We start by patching the interpreter and see if the executable already starts:

$ patchelf \
      --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
      masterpdfeditor4
$ ./masterpdfeditor4
./masterpdfeditor4: error while loading shared libraries: libsane.so.1: cannot open shared object file: No such file or directory

We went beyond the No such file or directory error, means that setting the interpreter worked! Unfortunately we have new errors which show that some shared objects cannot be found.

Extra Dynamic Libraries

Just like before, the executable simply expects libraries to be available in the Linux Standard Base(LSB) directories. Because Nix tries to avoid impurity and global directories we also have to explicitly set the libraries.

Patchelf can show the required libraries but we still have to set them manually.

$ patchelf --print-needed masterpdfeditor4
libdl.so.2
librt.so.1
libsane.so.1
libQt5Svg.so.5
libQt5PrintSupport.so.5
libQt5Widgets.so.5
libQt5Gui.so.5
libQt5Network.so.5
libQt5Core.so.5
libGL.so.1
libpthread.so.0
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
$ # now that the interpreter path is patched we can use ldd to find the libraries which are currently not found (check '=> not found'):
$ ldd masterpdfeditor4 | grep 'not found'
	libsane.so.1 => not found
	libQt5Svg.so.5 => not found
	libQt5PrintSupport.so.5 => not found
	libQt5Widgets.so.5 => not found
	libQt5Gui.so.5 => not found
	libQt5Network.so.5 => not found
	libQt5Core.so.5 => not found
	libstdc++.so.6 => not found

All the libraries which are not yet found are the ones we need to add to the runtime search path of the executable (RPATH). Again we can use patchelf to do this. We will be using nix-index for finding the files we are looking for:

$ # we generate the database index of all files in our channel first
$ nix-index
+ querying available packages
+ generating index: 41977 paths found :: 15957 paths not in binary cache :: 00000 paths in queue 
+ wrote index of 21,621,061 bytes
# we use the power of nix-locate to find the packages which contain the file:
$ nix-locate -1 -w lib/libsane.so.1
(wineStaging.out)
saneBackends.out
saneBackendsGit.out
$ nix-locate -1 -w lib/libQt5Svg.so.5
libsForQt56.qtinstaller.out
qt56.qtsvg.out
qt5.qtsvg.out
qt56.full.out
libsForQt5.qtinstaller.out
$ nix-locate -1 -w lib/libQt5PrintSupport.so.5
robomongo.out
libsForQt56.qtinstaller.out
qt5.qtbase.out
qt56.qtbase.out
qt56.full.out
libsForQt5.qtinstaller.out
$ nix-locate -1 -w lib/libstdc++.so.6
# ...
# libsdtc++.so.6 is `special`, it resides in stdenv.cc.cc.lib (see other packages)

Unfortunately there is no "right" way to choose which package to actually take, you can check out other derivations by grepping in nixpkgs.

The next step is to create a library path for all these packages. We use nix-repl to resolve the paths:

$ nix repl '<nixpkgs>'
# .out can be omitted because this is the default output for all packages
# makeLibraryPath outputs the correct path for each package to use as rpath
nix-repl> with pkgs; lib.makeLibraryPath [ saneBackends qt5.qtbase qt5.qtsvg stdenv.cc.cc.lib ]
"/nix/store/7lbi3gn351j4hix3dqhis58adxbmvbxa-sane-backends-1.0.25/lib:/nix/store/0990ls1p2nnxq6605mr9lxpps8p7qvw7-qtbase-5.9.1/lib:/nix/store/qzhn2svk71886fz3a79vklps781ah0lb-qtsvg-5.9.1/lib:/nix/store/snc31f0alikhh3a835riyqhbsjm29vki-gcc-6.4.0-lib/lib"


Let's try out the path we generated:

$ patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" --set-rpath /nix/store/7lbi3gn351j4hix3dqhis58adxbmvbxa-sane-backends-1.0.25/lib:/nix/store/0990ls1p2nnxq6605mr9lxpps8p7qvw7-qtbase-5.9.1/lib:/nix/store/qzhn2svk71886fz3a79vklps781ah0lb-qtsvg-5.9.1/lib:/nix/store/snc31f0alikhh3a835riyqhbsjm29vki-gcc-6.4.0-lib/lib   masterpdfeditor4
$ ./masterpdfeditor4
# SUCCESS!!!

Creating the Derivation for upstream Packaging

Packaging is straight forward. We just have to add all the steps we did into a simple derivation file. We call it package.nix and store it in the checked out nixpkgs repository at pkgs/by-name/ma/master-pdf-editor The content looks like this:

{ stdenv, lib, qt5, saneBackends, makeWrapper, fetchurl }:
stdenv.mkDerivation rec {
  name = "master-pdf-editor-${version}";
  version = "4.3.10";

  src = fetchurl {
    url = "http://get.code-industry.net/public/master-pdf-editor-${version}_qt5.amd64.deb";
    hash = "sha256-1z26qjhbiyz33rm7mp8ycgl5ka0v3v5lv5i5v0b5mx35arvx2zzy";
  };
  sourceRoot = ".";
  unpackCmd = "dpkg-deb -x master-pdf-editor-${version}_qt5.amd64.deb .";

  dontConfigure = true;
  dontBuild = true;

  installPhase = ''
    runHook preInstall

    mkdir -p $out/bin
    cp -R usr/share opt $out/
    # fix the path in the desktop file
    substituteInPlace \
      $out/share/applications/masterpdfeditor4.desktop \
      --replace /opt/ $out/opt/
    # symlink the binary to bin/
    ln -s $out/opt/master-pdf-editor-4/masterpdfeditor4 $out/bin/masterpdfeditor4

    runHook postInstall
  '';
  preFixup = let
    # we prepare our library path in the let clause to avoid it become part of the input of mkDerivation
    libPath = lib.makeLibraryPath [
      qt5.qtbase        # libQt5PrintSupport.so.5
      qt5.qtsvg         # libQt5Svg.so.5
      stdenv.cc.cc.lib  # libstdc++.so.6
      saneBackends      # libsane.so.1
    ];
  in ''
    patchelf \
      --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
      --set-rpath "${libPath}" \
      $out/opt/master-pdf-editor-4/masterpdfeditor4
  '';

  meta = with lib; {
    homepage = https://code-industry.net/masterpdfeditor/;
    description = "a multifunctional PDF Editor";
    license = licenses.unfree;
    platforms = platforms.linux;
    maintainers = [ your_name ];
  };
}

Because we created a derivation which is meant to be called by callPackage we can build the package now only via: nix-build -E '((import <nixpkgs> {}).callPackage (import ./package.nix) { })' --keep-failed --no-out-link

Creating a Pull Request

With this new package you can create a pull request for nixpkgs. Be aware that binary distributions are frowned upon if the source is available.


autoPatchelfHook

autoPatchelfHook can make the manual written patchelf invocations unnecessary.

Wrong file paths

Some programs will try to access hard-coded FHS file paths like /usr/lib or /opt. Mostly though, this will produce silent No such file errors, which can break the program.

To make these errors visible, we can use strace

strace --trace=file,process --follow-forks --string-limit=200 \
  ./result/bin/some-program 2>strace.log

cat strace.log | grep -e 'No such file' -e 'execve("' \
  | grep -v -E -e '(open|stat|access)(at)?\(.*"/nix/store/' \
    -e resumed -e '/etc/ld-nix.so.preload'

Example

[pid 357679] openat(AT_FDCWD, "/opt/brother/Printers/hll3210cw/inf/lut/0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 357679] openat(AT_FDCWD, "0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)

This means: process 357679 is trying to open file 0600-k_cache17.bin either from the hard-coded path in /opt/brother/Printers/hll3210cw/inf/lut or from the current workdir. So, as a quick fix, we could change the working directory of the wrapped binary with makeWrapper:

makeWrapper <executable> <out_path> --run 'cd /nix/store/path/to/workdir'

Finding the failing process

How can we find which process is throwing the No such file error? Let's search for the process ID and the exec syscall:

cat strace.log | grep 357679 | grep exec

[pid 357679] execve("/nix/store/ybl1pacslmhci9zy5qv95hshdgz6ihjl-brother-hll3210cw-1.0.2-0/opt/brother/Printers/hll3210cw/lpd/brhll3210cwfilter", ["/nix/store/ybl1pacslmhci9zy5qv95"..., "-pi", "/nix/store/ybl1pacslmhci9zy5qv95"..., "-rc", "/nix/store/ybl1pacslmhci9zy5qv95"...], 0x8ba010 /* 132 vars */ <unfinished ...>

See also