NixOS on ARM/Raspberry Pi 5: Difference between revisions

Add remote builder section
m Add remote builder steps
Line 170: Line 170:
* '''emulated native compile using QEMU''' virtualization by enabling the [https://docs.kernel.org/admin-guide/binfmt-misc.html binfmt] kernel feature on NixOS configuration setting <code>boot.binfmt.emulatedSystems = [ "aarch64-linux" ];</code>). This can be fast if everything is downloaded pre-compiled from the cache.nixos.org cache and only few packages really need local compilation. In reality it can be extremely slow, e.g. compiling a Linux kernel alone can take days.
* '''emulated native compile using QEMU''' virtualization by enabling the [https://docs.kernel.org/admin-guide/binfmt-misc.html binfmt] kernel feature on NixOS configuration setting <code>boot.binfmt.emulatedSystems = [ "aarch64-linux" ];</code>). This can be fast if everything is downloaded pre-compiled from the cache.nixos.org cache and only few packages really need local compilation. In reality it can be extremely slow, e.g. compiling a Linux kernel alone can take days.
* '''cross-compile''' to ARM using as to happen natively, but nothing will be cached from cache.nixos.org as this is not pre-build. So the compile itself is fast but there will be a lot more to compile locally. In practice it's quite fragile, because you may encounter packages that don't really support cross-compilation get stuck.
* '''cross-compile''' to ARM using as to happen natively, but nothing will be cached from cache.nixos.org as this is not pre-build. So the compile itself is fast but there will be a lot more to compile locally. In practice it's quite fragile, because you may encounter packages that don't really support cross-compilation get stuck.
* '''native compile on an remote builder''' like the Pi 5 itself running its custom Debian Linux at the beginning or later NixOS. This is reasonably fast as most packages are pre-build and cached on cache.nixos.org, and building a remaining Linux kernel only takes 2-3h on the Pi 5.
* '''native compile on an remote builder''' like the Pi 5 itself running its custom Debian Linux at the beginning or later NixOS. This is quite simple to setup and reasonably fast as most packages are pre-build and cached on cache.nixos.org, and building a remaining Linux kernel only takes 2-3h on the Pi 5.
'''Setting up the Pi 5 as a remote native builder''' can be done following the steps at [[Distributed build|https://wiki.nixos.org/wiki/Distributed_build]]. The rough steps are as follows:
 
# '''Install the Nix package manager''' on Pi 5 Debian OS the normal, multi-user way with <code>sh <(curl -L <nowiki>https://nixos.org/nix/install</nowiki>) --daemon</code>. If you already have NixOS running on the Pi 5, then you can skip this step.
# '''Setup a <code>ssh</code> connection''' from your local machine to the Pi, especially adding <code>SetEnv PATH=/nix/var/nix/profiles/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin</code> to the Pi's <code>/etc/ssh/sshd_config</code> file. If you already have NixOS running on the Pi 5, then you can skip this step.
# '''Make the remote Pi known to you local computer''' by adding it as a <code>nix.buildMachines</code> entry to your  <code>/etc/nix/configuration.nix</code> file and use connection protocol <code>ssh-ng</code>(!).
# You can then '''build, e.g. an NixOS sd card image''' with a call similar to  <code>nix build .\#nixosConfigurations.pi5.config.system.build.sdImage</code>
# '''flash that resulting image onto an SD card''' or NVMe SSD using a call similar to <code>zstdcat result/sd-image/nixos-sd-image-23.11.20230703.ea4c80b-aarch64-linux.img.zst | sudo dd of=/dev/mmcblk0 bs=100M status=progress</code> and place that card into the Pi 5.  


Missing:
Missing:


# How to do cross-compilation.
# How to do cross-compilation.
# How to create a convenient SD card image to not install anything manually.
# How to install U-Boot instead of EDK2 and systemd-boot.