NixOS Wiki

Packaging/Binaries: Difference between revisions

← Older edit
Visual
imported>Milahu
add section: Wrong file paths
m update url to wayback machine url
 
(10 intermediate revisions by 8 users not shown)
Line 1: Line 1:
Downloading and attempting to run a binary on NixOS will almost never work. This is due to hard-coded paths in the executable. Unfortunately, almost all unfree and proprietary software comes in binary form - the main reason to include binaries is because no source code is available.
Downloading and attempting to run a binary on NixOS will almost never work. This is due to hard-coded paths in the executable. Unfortunately, almost all unfree and proprietary software comes in binary form - the main reason to include binaries is because no source code is available.
This tutorial will guide you through packaging a binary executable.
This tutorial will guide you through packaging a binary executable.
If a downloaded binary is not packaged for NixOS then running the binary may fail like the following:
<syntaxHighlight lang=bash>
$ ~/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu  --help
bash: /home/gdforj/.local/share/nvim/mason/packages/rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu: cannot execute: required file not found
</syntaxHighlight>
Which indicates the interpreter, [[#The Dynamic Loader|the dynamic loader]] in this case, could not be located.


== Using AutoPatchelfHook ==
== Using AutoPatchelfHook ==
Line 16: Line 25:


stdenv.mkDerivation rec {
stdenv.mkDerivation rec {
   name = "studio-link-${version}";
   pname = "studio-link";
 
   version = "21.07.0";
   version = "21.07.0";


   src = fetchurl {
   src = fetchurl {
     url = "https://download.studio.link/releases/v${version}-stable/linux/studio-link-standalone-v${version}.tar.gz";
     url = "https://download.studio.link/releases/v${version}-stable/linux/studio-link-standalone-v${version}.tar.gz";
     sha256 = "sha256-4CkijAlenhht8tyk3nBULaBPE0GBf6DVII699/RmmWI=";
     hash = "sha256-4CkijAlenhht8tyk3nBULaBPE0GBf6DVII699/RmmWI=";
   };
   };


Line 39: Line 47:


   installPhase = ''
   installPhase = ''
    runHook preInstall
     install -m755 -D studio-link-standalone-v${version} $out/bin/studio-link
     install -m755 -D studio-link-standalone-v${version} $out/bin/studio-link
    runHook postInstall
   '';
   '';


Line 52: Line 62:
See [https://github.com/NixOS/nixpkgs/commit/ea5787ad5291ee1c131326cb9c9fec03d359edff this commit], or the example in method 5 of [https://unix.stackexchange.com/a/522823 this answer], for more details.
See [https://github.com/NixOS/nixpkgs/commit/ea5787ad5291ee1c131326cb9c9fec03d359edff this commit], or the example in method 5 of [https://unix.stackexchange.com/a/522823 this answer], for more details.


=== architecture differs from target ===
=== Additional Library Paths ===
 
Some packages may put their library files somewhere other than <code>$out/lib/</code> in the install phase, while <code>autoPatchelfHook</code> by default only looks in that one directory for every package in <code>buildInputs</code>.  You can tell <code>autoPatchelfHook</code> to look in additional directories by passing it to the command <code>addAutoPatchelfSearchPath</code> in any phase before <code>autoPatchelfHook</code> is run.
autoPatchelfHook error:


For example,
<pre>
<pre>
skipping <file> because its architecture (i386) differs from target (i386:x86-64)
  preBuild = ''
</pre>
    addAutoPatchelfSearchPath ${jre8}/lib/openjdk/jre/lib/
 
  '';
fix:
 
<pre>
  meta = {
-    architectures = [ "amd64" ];
+    architectures = [ "x86" ];
  };
</pre>
</pre>


Line 73: Line 76:
This tutorial is about how to patch the executable with <code>patchelf</code> which is sufficient for most cases. If no source is available for a program patchelf is the preferred way in nixpkgs to add support for the package.
This tutorial is about how to patch the executable with <code>patchelf</code> which is sufficient for most cases. If no source is available for a program patchelf is the preferred way in nixpkgs to add support for the package.


However sometimes this is not enough, more hardcoded paths may be scattered all over the place. For this you may need to set up an FHSUserEnv, a Linux Standard Base style directory structure with a final <code>chroot</code> call to fixate all paths. For a tutorial on how to use this technique, check out [https://reflexivereflection.com/posts/2015-02-28-deb-installation-nixos.html Anders Papittos blog post on installing debian packages in nixos].
However sometimes this is not enough, more hardcoded paths may be scattered all over the place. For this you may need to set up an FHSUserEnv, a Linux Standard Base style directory structure with a final <code>chroot</code> call to fixate all paths. For a tutorial on how to use this technique, check out [https://web.archive.org/web/20240125095859/http://reflexivereflection.com/posts/2015-02-28-deb-installation-nixos.html Anders Papittos blog post on installing debian packages in nixos].
=== Starting Point ===
=== Starting Point ===
We want to package a tool called "MasterPDFEditor", the package for debian can be found at [http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb] ([https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb archive.org mirror]).
We want to package a tool called "MasterPDFEditor", the package for debian can be found at [http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb] ([https://web.archive.org/web/20170914112947/http://get.code-industry.net/public/master-pdf-editor-4.3.10_qt5.amd64.deb archive.org mirror]).
Line 200: Line 203:


== Creating the Derivation for upstream Packaging ==
== Creating the Derivation for upstream Packaging ==
Packaging is straight forward. We just have to add all the steps we did into a simple derivation file. We call it <code>default.nix</code> and store it in the checked out nixpkgs repository at <code>pkgs/applications/office/master-pdf-editor</code>
Packaging is straight forward. We just have to add all the steps we did into a simple derivation file. We call it <code>package.nix</code> and store it in the checked out nixpkgs repository at <code>pkgs/by-name/ma/master-pdf-editor</code>
The content looks like this:
The content looks like this:
<syntaxHighlight lang="nix">
<syntaxHighlight lang="nix">
Line 210: Line 213:
   src = fetchurl {
   src = fetchurl {
     url = "http://get.code-industry.net/public/master-pdf-editor-${version}_qt5.amd64.deb";
     url = "http://get.code-industry.net/public/master-pdf-editor-${version}_qt5.amd64.deb";
     sha256 = "1z26qjhbiyz33rm7mp8ycgl5ka0v3v5lv5i5v0b5mx35arvx2zzy";
     hash = "sha256-1z26qjhbiyz33rm7mp8ycgl5ka0v3v5lv5i5v0b5mx35arvx2zzy";
   };
   };
   sourceRoot = ".";
   sourceRoot = ".";
Line 219: Line 222:


   installPhase = ''
   installPhase = ''
    runHook preInstall
     mkdir -p $out/bin
     mkdir -p $out/bin
     cp -R usr/share opt $out/
     cp -R usr/share opt $out/
Line 227: Line 232:
     # symlink the binary to bin/
     # symlink the binary to bin/
     ln -s $out/opt/master-pdf-editor-4/masterpdfeditor4 $out/bin/masterpdfeditor4
     ln -s $out/opt/master-pdf-editor-4/masterpdfeditor4 $out/bin/masterpdfeditor4
    runHook postInstall
   '';
   '';
   preFixup = let
   preFixup = let
Line 254: Line 261:


Because we created a derivation which is meant to be called by callPackage we can build the package now only via:
Because we created a derivation which is meant to be called by callPackage we can build the package now only via:
<code>nix-build -E '((import <nixpkgs> {}).callPackage (import ./default.nix) { })' --keep-failed --no-out-link</code>
<code>nix-build -E '((import <nixpkgs> {}).callPackage (import ./package.nix) { })' --keep-failed --no-out-link</code>
 
== Add the package to nixpkgs ==
In order to add this new package to nixpkgs and be able to install it via <code>nix-build -A pkgs.master-pdf-editor</code> you need to add it to <code>pkgs/top-level/all-packages.nix</code>:
<syntaxHighlight lang=nix>
  ...
  masscan = callPackage ../tools/security/masscan { };
 
  master-pdf-editor = callPackage ../applications/office/master-pdf-editor {};
 
  meson = ../development/tools/build-managers/meson { };
  ...
</syntaxHighlight>


== Creating a Pull Request ==
== Creating a Pull Request ==
Line 280: Line 275:
== Wrong file paths ==
== Wrong file paths ==


Some programs will try to access hard-coded FHS file paths like <code>/usr/lib</code>, but mostly, this will result in <code>No such file</code> errors, which can break the program
Some programs will try to access hard-coded FHS file paths like <code>/usr/lib</code> or <code>/opt</code>. Mostly though, this will produce silent <code>No such file</code> errors, which can break the program.
 
To make these errors visible, we can use <code>strace</code>
 
<pre>
strace --trace=file,process --follow-forks --string-limit=200 \
  ./result/bin/some-program 2>strace.log
 
cat strace.log | grep -e 'No such file' -e 'execve("' \
  | grep -v -E -e '(open|stat|access)(at)?\(.*"/nix/store/' \
    -e resumed -e '/etc/ld-nix.so.preload'
</pre>
 
=== Example ===
 
<pre>
[pid 357679] openat(AT_FDCWD, "/opt/brother/Printers/hll3210cw/inf/lut/0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 357679] openat(AT_FDCWD, "0600-k_cache17.bin", O_RDONLY) = -1 ENOENT (No such file or directory)
</pre>
 
This means: process <code>357679</code> is trying to open file <code>0600-k_cache17.bin</code> either from the hard-coded path in <code>/opt/brother/Printers/hll3210cw/inf/lut</code> or from the current workdir.
So, as a quick fix, we could change the working directory of the wrapped binary with [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/setup-hooks/make-wrapper.sh makeWrapper]:
 
<pre>
makeWrapper <executable> <out_path> --run 'cd /nix/store/path/to/workdir'
</pre>
 
=== Finding the failing process ===
How can we find which process is throwing the <code>No such file</code> error? Let's search for the process ID and the <code>exec</code> syscall:


<pre>
<pre>
strace --trace=file --follow-forks ./result/bin/some-program 2>strace.log
cat strace.log | grep 357679 | grep exec


cat strace.log | grep 'No such file' | grep -v -e '/nix/store/' -e '/etc/ld-nix.so.preload' | grep '"/' | less -S
[pid 357679] execve("/nix/store/ybl1pacslmhci9zy5qv95hshdgz6ihjl-brother-hll3210cw-1.0.2-0/opt/brother/Printers/hll3210cw/lpd/brhll3210cwfilter", ["/nix/store/ybl1pacslmhci9zy5qv95"..., "-pi", "/nix/store/ybl1pacslmhci9zy5qv95"..., "-rc", "/nix/store/ybl1pacslmhci9zy5qv95"...], 0x8ba010 /* 132 vars */ <unfinished ...>
</pre>
</pre>


Retrieved from "https://wiki.nixos.org/wiki/Packaging/Binaries"