Cross Compiling: Difference between revisions

From NixOS Wiki
imported>Mic92
Klinger (talk | contribs)
(14 intermediate revisions by 7 users not shown)
Line 4: Line 4:
Quick example to cross compile a package: [[Cheatsheet#Cross-compile_packages]].
Quick example to cross compile a package: [[Cheatsheet#Cross-compile_packages]].


== Cross-Compiling in nixpkgs unstable (pre-18.09) ==
== Cross-Compiling a package in nixpkgs ==
A lot has happened in 2018 and now cross-compilation is well supported within the unstable branch of nixpkgs.
Cross-compilation is well supported in nixpkgs since 18.09.
The basic idea is to use <code>pkgsCross.platform</code> instead of <code>pkgs</code>:
The basic idea is to use <code>pkgsCross.platform</code> instead of <code>pkgs</code>:
<syntaxHighlight lang=bash>
<syntaxHighlight lang=bash>
nix build -f channel:nixos-unstable pkgsCross.raspberryPi.emacs
nix-build '<nixpkgs>' -A pkgsCross.raspberryPi.openssl
</syntaxHighlight>
</syntaxHighlight>
== How to obtain a shell with a cross compiler ==
Create a file <code>crossShell.nix</code> as follows:
<syntaxhighlight lang="nix">
with import <nixpkgs> {
  crossSystem = {
    config = "aarch64-unknown-linux-gnu";
  };
};
mkShell {
  packages = [ zlib ]; # your dependencies here
}
</syntaxhighlight>
and then use it to obtain a shell:
{{commands|nix-shell crossShell.nix}}
The resulting shell contains a cross toolchain and zlib in this example. Note that contrary to native shells, the compiler and some other tools are prefixed: there is no <code>gcc</code> but a <code>aarch64-unknown-linux-gnu-gcc</code>. Some convenience environment variables expand to the prefixed version of tools: <code>$CC</code>, <code>$LD</code>...
Examples of how to specify your target system can be found in [https://github.com/NixOS/nixpkgs/blob/master/lib/systems/examples.nix lib/systems/examples.nix]. If the exact system you are targeting is available in this file then you can use the existing definition as in the following example:
<syntaxhighlight lang="nix">
let pkgs = import <nixpkgs> {
    crossSystem = (import <nixpkgs/lib>).systems.examples.armv7l-hf-multiplatform;
};
in
mkShell {}
</syntaxhighlight>
Even shorter:
<syntaxhighlight lang="nix">
let pkgs = import <nixpkgs> {}; in
pkgs.pkgsCross.armv7l-hf-multiplatform.mkShell {}
</syntaxhighlight>
The examples above do not work as is with build dependencies (<code>nativeBuildInputs</code>). A solution is to use <code>callPackage</code> to enable splicing:
<syntaxhighlight lang="nix">
let pkgs = import <nixpkgs> {
  crossSystem = {
    config = "aarch64-unknown-linux-gnu";
  };
};
in
  pkgs.callPackage (
    {mkShell, pkg-config, zlib}:
    mkShell {
      nativeBuildInputs = [ pkg-config ]; # you build dependencies here
      buildInputs = [ zlib ]; # your dependencies here
    }
  ) {}
</syntaxhighlight>
See also {{issue|49526}}.
== Lazy cross-compiling ==
If you target "aarch64-unknown-linux-gnu", there is a nice way to reduce amount of cross-compiling and side-step journey to fix cross errors. The idea is to fetch non-essential dependencies from binary cache of regular aarch64 binaries.
Say we are building SDL2.
<syntaxhighlight lang="nix">
let
    # this will use aarch64 binaries from binary cache, so no need to build those
    pkgsArm = import <nixpkgs> {
        config = {};
        overlays = [];
        system = "aarch64-linux";
    };
    # these will be your cross packages
    pkgsCross = import <nixpkgs> {
      overlays = [(self: super: {
        # we want to hack on SDL, don't want to hack on those. Some even don't cross-compile
        inherit (pkgsArm)
          xorg libpulseaudio libGL guile systemd libxkbcommon
        ;
      })];
      crossSystem = {
        config = "aarch64-unknown-linux-gnu";
      };
    };
in pkgsCross.SDL2.override {
      # those shouldn't be neither pkgsCross, nor pkgsArm
      # because those trigger
      #    cannot execute binary file: Exec format error
      # in this case it was enough to just use buildPackages variants
      # but in general there may be problems
      inherit (pkgsCross.buildPackages)
        wayland wayland-protocols
      ;
  }
</syntaxhighlight>




Line 17: Line 110:


* If it is used at build-time it's <code>depsBuildXXX</code>
* If it is used at build-time it's <code>depsBuildXXX</code>
** compiler producing native binaries go to <code>depsBuildBuild</code>
** compiler producing native binaries go to <code>depsBuildBuild</code>;
** compiler producing cross binaries, all setup hooks and programs executed by the builder go to <code>depsBuildHost</code>
** compiler producing cross binaries, all setup hooks and programs executed by the builder go to <code>depsBuildHost</code>:
*** common examples: <code>pkgconfig, autoreconfHook, makeWrapper, intltool, bison, flex</code>
*** common examples: <code>pkg-config, autoreconfHook, makeWrapper, intltool, bison, flex</code>.
* If it is used at run-time it's <code>depsHostXXX</code>. [Stack linking doesn't effect this, even if it allows us to forget where things came from.]
* If it is used at run-time it's <code>depsHostXXX</code>. [Static linking doesn't effect this, even if it allows us to forget where things came from.]
** if it’s an interpreter that will be needed by an installed script, it should go in <code>depsHostTarget</code>.  
** if it’s an interpreter that will be needed by an installed script, it should go in <code>depsHostTarget</code>.  
** otherwise it is probably only needed at build time and can go in <code>depsBuildHost</code>
** otherwise it is probably only needed at build time and can go in <code>depsBuildHost</code>
* If it is a tool and "acts" (e.g. helps build) on build-time stuff, then it's <code>depsXXXBuild</code>
* If it is a tool and "acts" (e.g. helps build) on build-time stuff, then it's <code>depsXXXBuild</code>.
* If it is a tool and "acts" on run-time stuff, then it's <code>depsXXXHost</code>
* If it is a tool and "acts" on run-time stuff, then it's <code>depsXXXHost</code>.
* if it is not a tool, it's <code>depsXXX(XXX+1)</code>(build + 1 == host, host +1 == target) for backwards compatibility, use <code>nativeBuildInputs</code> instead of <code>depsBuildHost</code> and <code>buildInputs</code> instead of <code>depsHostTarget</code>.
* If it is not a tool, it's <code>depsXXX(XXX+1)</code>(build + 1 == host, host +1 == target). For backwards compatibility use <code>nativeBuildInputs</code> instead of <code>depsBuildHost</code> and <code>buildInputs</code> instead of <code>depsHostTarget</code>.


Source: https://github.com/NixOS/nixpkgs/pull/50881#issuecomment-440772499
Source: https://github.com/NixOS/nixpkgs/pull/50881#issuecomment-440772499
Line 33: Line 126:


* Nixpkgs manual on [https://nixos.org/nixpkgs/manual/#chap-cross cross compiling]
* Nixpkgs manual on [https://nixos.org/nixpkgs/manual/#chap-cross cross compiling]
* Nixpkgs manual on [https://nixos.org/nixpkgs/manual/#ssec-stdenv Specifying dependencies]
* Nixpkgs manual on [https://nixos.org/nixpkgs/manual/#ssec-stdenv-dependencies Specifying dependencies]


* [https://matthewbauer.us/blog/beginners-guide-to-cross.html Introduction to Cross Compilation with nix by Matthew Bauer]
* [https://matthewbauer.us/blog/beginners-guide-to-cross.html Introduction to Cross Compilation with nix by Matthew Bauer]
Line 39: Line 132:
== Additional resources ==
== Additional resources ==


* [https://docs.google.com/presentation/d/11B3Igxj0KmsxgT_UQvKufd5El_oH-T__Sl3JNCZeOnY/edit?usp=sharing Slides from the cross-compilation workshop on 35c3]
* [https://logs.nix.samueldr.com/nixos/2018-08-03#1533327247-1533327971; 2018-08-03 discussion on #nixos] ([https://matrix.to/#/!AinLFXQRxTuqNpXyXk:matrix.org/$15333274371713496LOAor:matrix.org Mirror of chat on Matrix.org])
* [https://logs.nix.samueldr.com/nixos/2018-08-03#1533327247-1533327971; 2018-08-03 discussion on #nixos] ([https://matrix.to/#/!AinLFXQRxTuqNpXyXk:matrix.org/$15333274371713496LOAor:matrix.org Mirror of chat on Matrix.org])
[[Category:nix]]
[[Category:Development]]

Revision as of 21:07, 24 April 2024

For building arm software check out the Article NixOS on ARM
If you are looking for building 32bit software, check out Packaging/32bit Applications
Quick example to cross compile a package: Cheatsheet#Cross-compile_packages.

Cross-Compiling a package in nixpkgs

Cross-compilation is well supported in nixpkgs since 18.09. The basic idea is to use pkgsCross.platform instead of pkgs:

nix-build '<nixpkgs>' -A pkgsCross.raspberryPi.openssl

How to obtain a shell with a cross compiler

Create a file crossShell.nix as follows:

with import <nixpkgs> {
  crossSystem = {
    config = "aarch64-unknown-linux-gnu";
  };
};

mkShell {
  packages = [ zlib ]; # your dependencies here
}

and then use it to obtain a shell:

nix-shell crossShell.nix

The resulting shell contains a cross toolchain and zlib in this example. Note that contrary to native shells, the compiler and some other tools are prefixed: there is no gcc but a aarch64-unknown-linux-gnu-gcc. Some convenience environment variables expand to the prefixed version of tools: $CC, $LD...

Examples of how to specify your target system can be found in lib/systems/examples.nix. If the exact system you are targeting is available in this file then you can use the existing definition as in the following example:

let pkgs = import <nixpkgs> {
    crossSystem = (import <nixpkgs/lib>).systems.examples.armv7l-hf-multiplatform;
};
in
 mkShell {}

Even shorter:

let pkgs = import <nixpkgs> {}; in
pkgs.pkgsCross.armv7l-hf-multiplatform.mkShell {}

The examples above do not work as is with build dependencies (nativeBuildInputs). A solution is to use callPackage to enable splicing:

let pkgs = import <nixpkgs> {
  crossSystem = {
    config = "aarch64-unknown-linux-gnu";
  };
};
in
  pkgs.callPackage (
    {mkShell, pkg-config, zlib}:
    mkShell {
      nativeBuildInputs = [ pkg-config ]; # you build dependencies here
      buildInputs = [ zlib ]; # your dependencies here
    }
  ) {}

See also #49526.

Lazy cross-compiling

If you target "aarch64-unknown-linux-gnu", there is a nice way to reduce amount of cross-compiling and side-step journey to fix cross errors. The idea is to fetch non-essential dependencies from binary cache of regular aarch64 binaries.

Say we are building SDL2.

let
    # this will use aarch64 binaries from binary cache, so no need to build those
    pkgsArm = import <nixpkgs> {
        config = {};
        overlays = [];
        system = "aarch64-linux";
    };

    # these will be your cross packages
    pkgsCross = import <nixpkgs> {

       overlays = [(self: super: {

         # we want to hack on SDL, don't want to hack on those. Some even don't cross-compile
         inherit (pkgsArm)
           xorg libpulseaudio libGL guile systemd libxkbcommon
         ;
 
       })];
       crossSystem = {
         config = "aarch64-unknown-linux-gnu";
       };
     };

in pkgsCross.SDL2.override { 
      # those shouldn't be neither pkgsCross, nor pkgsArm
      # because those trigger
      #     cannot execute binary file: Exec format error
      # in this case it was enough to just use buildPackages variants
      # but in general there may be problems
      inherit (pkgsCross.buildPackages) 
         wayland wayland-protocols
      ;
   }


How to specify dependencies

Depending in which if packages are required at build time or at runtime they need to go to different inputs the derivation.

  • If it is used at build-time it's depsBuildXXX
    • compiler producing native binaries go to depsBuildBuild;
    • compiler producing cross binaries, all setup hooks and programs executed by the builder go to depsBuildHost:
      • common examples: pkg-config, autoreconfHook, makeWrapper, intltool, bison, flex.
  • If it is used at run-time it's depsHostXXX. [Static linking doesn't effect this, even if it allows us to forget where things came from.]
    • if it’s an interpreter that will be needed by an installed script, it should go in depsHostTarget.
    • otherwise it is probably only needed at build time and can go in depsBuildHost
  • If it is a tool and "acts" (e.g. helps build) on build-time stuff, then it's depsXXXBuild.
  • If it is a tool and "acts" on run-time stuff, then it's depsXXXHost.
  • If it is not a tool, it's depsXXX(XXX+1)(build + 1 == host, host +1 == target). For backwards compatibility use nativeBuildInputs instead of depsBuildHost and buildInputs instead of depsHostTarget.

Source: https://github.com/NixOS/nixpkgs/pull/50881#issuecomment-440772499


References

Additional resources