NixOS on ARM/Raspberry Pi 5
The Raspberry Pi family of devices is a series of single-board computers made by the Raspberry Pi Foundation. They are all based on Broadcom System-on-a-chip (SoCs).
| Raspberry Pi 5 Family | |
|---|---|
| |
| Manufacturer | Raspberry Pi Foundation |
| Architecture | AArch64 |
| Bootloader | Custom or UEFI |
| Boot order | Configurable; SD, USB, Netboot |
| Maintainer | leo60228 |
| Raspberry Pi 5B | |
| SoC | BCM2712 |
Status
Support for the Raspberry Pi 5 is currently downstream and experimental. It will not be fully supported until mature support exists in the upstream Linux kernel and the U-Boot boot loader, but using an UEFI bootloader already provides a workable solution.
The Raspberry Pi 5's boot process follows the typical boot stages on embedded devices, some of which have to be adapted to work with NixOS. For that it's helpful to understand more about the stages:
1. ROM boot loader: The first-stage boot loader comes "burned in" on the Pi in a tiny One-Time-Programmable memory (OTP) so it cannot be changed anymore. It's only able to load the next second-stage boot loader below, and reset it in case you have messed up.
See the official documentation.
Nothing to adapt here.
2. EEPROM boot loader: The second-stage boot loader comes built-in on the Pi in a bit larger, rewriteable
EEPROM memory
This loader is also very limited is only able to search for and start yet another, third-stage boot loader from other
storage hardware like an SD card, an NVME SSD, an USB disk, or from the network.
This loader (as many other second-stage boot loaders of other devices) is so size-constraint that it only contains
the bare mimum code to be able to read from an FAT formatted partition. That's why you see and want a separate small
/boot partition on your SD card or SSD that is formatted "FAT" or "VFAT", while your main data is stored on a second
"rootfs" or / partition with fancy, newer partition types like "ext4", "ZFS" or "btrfs".
See the official documentation.
See EEPROM image releases for improved and wider hardware support.
This boot loader can be updated via the rpi-eeprom-update terminal tool
(also available in Nixpkgs)
and loads the binary images (the firmware-2712/pieeprom-*.bin files) from the
rpi-eeprom Github project.
Nothing to adapt here yet. However, there's a [https://github.com/raspberrypi/firmware/issues/1857 feature request to support smaller third-stage boot loaders in this second-stage].
3. Firmware boot loader: The third-stage boot loader is loaded from the first partition (usually called /boot)
of an SD card, NVME SSD or other storage hardware described above.
Because size is usually not an issue here anymore you can have large, fully-fledged boot loaders like
systemd-boot (default with NixOS; requires UEFI), or full
U-Boot (popular with embedded devices like the Pi) or
GRUB (generally popular with Linux Distros).
However, the standard Pi 5 setup has no third-stage boot loader.
The second stage EEPROM boot loader loads the firmware (code to control other hardware on the Pi 5; [https://en.wikipedia.org/wiki/Devicetree device tree
files] in compact binary format *.dtb), some settings (cmdline.txt on which partition to find the rest of the system;
config.txt for general boot settings),
and directly the Linux kernel from a /boot/firmware/ folder. On the Pi 5's default Debian image this is
the kernel2712.img (specialized, more-performant kernel
named after the Pi 5's Brodcom BCM2712 ARMv8 SoC chip) or as a fallback the kernel8.img (generic, slower ARMv8 kernel for the Pi 4 that
also works for Pi 5) that you find on the Pi Firmware Github project.
See the official documentation.
Adaptions for booting NixOS:
In order to boot NixOS with it's default boot loader systemd-boot we need UEFI support, so replace the
1. ROM -> 2. EEPROM -> Kernel -> NixOS
workflow with third-stage and fourth-stage boot loaders (both as firmware on an SD card or NVME SSD etc.)
1. ROM -> 2. EEPROM -> 3. EDK2 (UEFI boot loader) -> 4. systemd-boot -> Kernel -> NixOS
as follows:
1. Install EDK2:
We need the first partition of the SD card (or NVME SSD, etc.) again to be formatted as FAT
but labelled ESP (EFI System Partition) to conform to (U)EFI standards.
Into this partition we need to place the EDK2 boot loader file RPI_EFI.fd
and a config.txt file with a line armstub=RPI_EFI.fd
which instructs the EEPROM boot loader to load our EDK2 boot loader instead.
See the EDK2 for Pi 5 Github project; the releases already contain both of these files.
See a guide on how to setup partitions and these files.
2. Install systemd-boot, kernel and NixOS:
The rest is usual NixOS installation on a second partition with the caveat to select a Linux kernel that supports the Pi 5; a suitable kernel can be created using leo60228's flake until that Pi 5 compatible Linux kernel is available in Nixos-hardware.
Follow this guide to build
a NixOS system closure that you can install manually onto the Pi with a nixos-install call.
That install tool will install the systemd-boot loader at /boot/EFI/systemd/systemd-bootaa64.efi
and the kernel files at /boot/EFI/nixos/*.efi onto your first ESP partition
and the rest of the NixOS system into your second partition.
Note that building that NixOS system closure has to happen natively, e.g. on a similar aarch64/Arm64 system like newer Macs (fast), or with
the binfmt kernel feature
(enabled with setting boot.binfmt.emulatedSystems = [ "aarch64-linux" ];) using QEMU virtualization
on any NixOS computer(veeeeerry slow)
Missing:
- How to do faster cross-compilation without binfmt.
- How to create a convenient SD card image to not install anything manually.
- How to install U-Boot instead of EDK2 and systemd-boot.
GPU
For the GPU drivers to work, dtoverlay=vc4-kms-v3d-pi5 must be added to /boot/config.txt, and the vendor kernel must currently be used. Only Wayland-based compositors are supported. Note that Xwayland applications may produce broken graphics on KDE; the root cause of this issue has not yet been evaluated.
Board-specific installation notes
First, install EDK2, following the instructions from the port README. With EDK2 installed as the Platform Firmware, you can follow the standard instructions for UEFI on ARM.
EDK2 enables booting a mainline kernel, but hardware support will be very limited. Notably, you'll need to perform the installation using Wi-Fi, as Ethernet is unsupported. Once the system is installed, you can switch to the vendor's modified kernel. This is not (yet?) available in Nixpkgs, so you'll need to get it from a flake. If you're not using flakes, you can simply add this to your configuration:
/etc/nixos/configuration.nix
{
boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5;
}
For the vendor kernel to boot properly, you must switch from ACPI to Device Tree in the UEFI settings (at Device Manager -> Raspberry Pi Configuration -> ACPI / Device Tree -> System Table Mode). When using the vendor kernel (which provides full power management support), you may additionally wish to remove force_turbo=1 from /boot/config.txt.
If you are using nixos-unstable, then you can also use the rpi4 kernel (this is supposed to be a generic aarch64 kernel for rpi 3,4,5). Although, due to a smaller page size, this will have slightly worse performance:
/etc/nixos/configuration.nix
{
boot.kernelPackages = pkgs.linuxPackages_rpi4;
}
Bluetooth
If your bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your nixos configuration:
/etc/nixos/configuration.nix
{
boot.kernelParams = [ "8250.nr_uarts=11" "console=ttyAMA10,9600" "console=tty0"];
}