Official NixOS Wiki

NixOS on ARM/Raspberry Pi 5: Difference between revisions

← Older edit
JamieMagee (talk | contribs)
5 edits
VisualWikitext
Pigs (talk | contribs)
275 edits
m Add category
Document nixos-unstable image support for the Pi 5 family, NVMe boot limits, serial setup, power and cooling, and the archived UEFI port
 
(21 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{ARM/breadcrumb}}
{{ARM/breadcrumb}}
<div class="infobox">
<div class="infobox">
{|class="table"
{| class="table"
!colspan="2" class="title"|Raspberry Pi 5 Family
! colspan="2" class="title" | Raspberry Pi 5 family
|-
|-
|colspan="2"|[[File:Raspberry_Pi_5,_8_GB_RAM.jpg|frameless|256px|A Raspberry Pi 5.]]
| colspan="2" | [[File:Raspberry_Pi_5,_8_GB_RAM.jpg|frameless|256px|A Raspberry Pi 5.]]
|-
|-
!Manufacturer
! Manufacturer
|Raspberry Pi Foundation
| Raspberry Pi Ltd
|-
|-
!Architecture
! Architecture
|AArch64
| AArch64
|-
|-
!Bootloader
! Stock boot method
|Custom, UEFI or u-boot
| Raspberry Pi EEPROM and firmware, U-Boot, and extlinux from microSD
|-
|-
!Boot order
! SoC
|Configurable; SD, USB, Netboot
| BCM2712
|-
|-
!Maintainer
! Variants
|leo60228
| Pi 5, Pi 500, Pi 500+, CM5, and CM5 Lite
|-
!colspan="2" class="title"|Raspberry Pi 5B
|-
!SoC
|BCM2712
|}
|}
</div>
</div>
The Raspberry Pi family of devices is a series of single-board computers made by the Raspberry Pi Foundation. They are all based on Broadcom System-on-a-chip (SoCs).
== Status ==
'''NixOS works on Raspberry Pi 5 but is currently experimental'''.
NixOS doesn't run out-of-box, but relies on several tweaks on the boot process that are maintained by different individuals and spread over multiple repositories.
The Raspberry Pi 5's boot process follows the [https://youtu.be/UvFG76qM6co?t=308 typical boot stages on embedded devices], and has the following boot loader steps by default:
  1. ROM -> 2. EEPROM -> Pi-custom Firmware/Kernel -> PiOS (custom Debian)
'''The pain points for NixOS support''' are the Pi's [https://github.com/raspberrypi/rpi-eeprom custom EEPROM boot bootloader], its  [https://github.com/raspberrypi/firmware proprietary, closed-source firmware] (code to use to hardware components) and [https://github.com/raspberrypi/linux its separately maintained Linux kernel], all of which we would need to '''update, build and test constantly and separately''' from the other NixOS Linux kernel variations, '''which is a large, unmaintainable burden for the NixOS community''' when the Pi 5 is not the only supported SoC; so we don't want that.
Instead our goal is to '''migrate towards the standard, generic boot process''' with UEFI:
  1. ROM -> 2. EEPROM -> 3. UEFI boot loader -> 4. systemd-boot boot loader -> Generic Kernel -> NixOS
We can't affect the ROM and EEPROM boot loaders as they come built-in into the hardware. However, [https://de.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface UEFI] is also used for booting normal Intel/AMD computers, and the [https://www.freedesktop.org/software/systemd/man/latest/systemd-boot.html systemd-boot] boot loader is THE software that allows us to have and select from multiple NixOS generations on boot (and perform rollbacks if we messed up). '''The currently required manual steps are roughly listed further down below''' and require lots of Linux and NixOS understanding.
Alternatively, '''if you want to get up-and-running quickly in the mean-time''' there's a setup that '''reuses lots of the pi-custom software''' (non-desired approach but works)
  1. ROM -> 2. EEPROM -> Pi-custom Firmware/Kernel -> NixOS
and creates a hard disk image that you can flash onto an SD card or NVMe SSD. The heavy lifting has been done and is maintained at https://github.com/tstat/raspberry-pi-nix (just follow the example and hints at https://github.com/tstat/raspberry-pi-nix/issues/13) and has the best out-of-the-box experience. You have to remote-build many Nix packages, probably the kernel as well, yourself (e.g. using the [https://wiki.nixos.org/wiki/Distributed_build Pi without NixOS as an intermediate remote builder]) and that can take several hours though.
It will also likely get [https://github.com/u-boot/u-boot U-Boot] support soon (likely only interesting for people that need more exotic hardware support like hats):
  1. ROM -> 2. EEPROM -> Pi-custom Firmware/Kernel -> U-Boot boot loader -> NixOS
== Raspberry Pi Boot stages ==
To understand the adaptions for NixOS better, it's helpful to understand more about the stages:
1. '''ROM boot loader''': The first-stage boot loader comes "burned in" on the Pi in a tiny
[[wikipedia:Programmable_ROM#One_time_programmable_memory|One-Time-Programmable memory (OTP)]]
so it cannot be changed anymore. It's '''only able to load the next second-stage boot loader below''', and reset it in case you have messed up.
See the [https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#first-stage-bootloader official documentation].
Nothing to adapt here.
2. '''EEPROM boot loader''': The second-stage boot loader comes built-in on the Pi in a larger, rewriteable
[[wikipedia:EEPROM|EEPROM]].
'''This loader is also very limited and is only able to search for and start yet another, third-stage boot loader from other '''
storage hardware like an SD card, an NVMe SSD, a USB disk, or from the network.
This loader (like many second-stage boot loaders of other devices) is so size-constrained that it only contains
the bare minimum code to be able to read from an FAT formatted partition. That's why you see and want a separate small
<code>/boot</code> partition on your SD card or SSD that is formatted "FAT" or "VFAT", while your main data is stored on a second
"rootfs" or <code>/</code> partition with fancy, newer partition types like "ext4", "ZFS" or "btrfs".


See the [https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#second-stage-bootloader official documentation].
The generic AArch64 SD image on <code>nixos-unstable</code> supports the '''Raspberry Pi 5''', Raspberry Pi 500, Raspberry Pi 500+, Compute Module 5, and Compute Module 5 Lite. NixOS 26.05 images do not include the required Pi 5-family boot files, so use an image from <code>nixos-unstable</code>.<ref name="pi5-image">[https://github.com/NixOS/nixpkgs/pull/537862 nixpkgs PR #537862: sd-image-aarch64: support rpi5]</ref> The stock image boots from microSD through Raspberry Pi firmware, U-Boot, and extlinux.


See [https://github.com/raspberrypi/rpi-eeprom/releases EEPROM image releases] for improved and wider hardware support.
== Installation ==
This boot loader can be updated via the <code>rpi-eeprom-update</code> terminal tool
(also [https://search.nixos.org/packages?channel=unstable&type=packages&query=raspberrypi-eeprom available in Nixpkgs])
and loads the binary images (the <code>firmware-2712/pieeprom-*.bin</code> files) from the
[https://github.com/raspberrypi/rpi-eeprom/tree/master/firmware-2712 rpi-eeprom GitHub project].


Nothing to adapt here yet. However, there's a  
Use the unstable generic AArch64 SD image described on the [[NixOS on ARM/Raspberry Pi#Installation|family page]]. The image contains the device trees used by Raspberry Pi 5, Raspberry Pi 500, Raspberry Pi 500+, Compute Module 5, and Compute Module 5 Lite, together with a single 64-bit U-Boot binary for Raspberry Pi boards.<ref name="generic-image">[https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/sd-card/sd-image-aarch64.nix Nixpkgs generic AArch64 SD image module]</ref> Raspberry Pi 500 and Raspberry Pi 500+ share board type <code>0x19</code>, for which U-Boot selects <code>bcm2712-rpi-500.dtb</code>.<ref name="revision-codes">[https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#new-style-revision-codes Raspberry Pi revision codes]</ref><ref name="uboot-models">[https://github.com/u-boot/u-boot/blob/v2026.07/board/raspberrypi/rpi/rpi.c U-Boot v2026.07 Raspberry Pi model table]</ref> In a flake, import <code>nixos-hardware.nixosModules.raspberry-pi-5</code>. With channels, import <code>&lt;nixos-hardware/raspberry-pi/5&gt;</code>. The family page explains that a profile does not create an image or stage U-Boot.
[https://github.com/raspberrypi/firmware/issues/1857 feature request to support smaller third-stage boot loaders in this second-stage].


3. '''Firmware boot loader''': The third-stage boot loader is loaded from the first partition (usually called <code>/boot</code>)
== Kernel and hardware defaults ==
of an SD card, NVME SSD or other storage hardware described above.
Because '''size is usually not an issue here, you can have large, fully-fledged boot loaders''' like
[https://www.freedesktop.org/software/systemd/man/latest/systemd-boot.html systemd-boot] (default with NixOS; requires UEFI), or full
[https://github.com/u-boot/u-boot U-Boot] (popular with embedded devices like the Pi) or
[https://www.gnu.org/software/grub/ GRUB] (generally popular among Linux distros).


However, '''the standard Pi 5 setup has no third-stage boot loader'''.
The profile selects a pinned kernel from Raspberry Pi's downstream Linux tree and limits device trees to BCM2712 Raspberry Pi boards. Its initrd includes NVMe, BCM2712 PCIe, RP1 clock, and support for the RP1 multifunction device. It disables GRUB and enables extlinux generation.<ref name="profile">[https://github.com/NixOS/nixos-hardware/blob/master/raspberry-pi/5/default.nix nixos-hardware Raspberry Pi 5 profile]</ref>
The second stage EEPROM boot loader loads the firmware (code to control other hardware on the Pi 5; [https://en.wikipedia.org/wiki/Devicetree device tree files]
in compact binary format <code>*.dtb</code>), some settings (<code>cmdline.txt</code> for kernel settings,
<code>config.txt</code> for firmware settings),
and the Linux kernel itself from a <code>/boot/firmware/</code> folder. On the Pi 5's default Debian image this is
the <code>kernel2712.img</code> (specialized, more-performant kernel
named after the Pi 5's Brodcom BCM2712 ARMv8 SoC chip) or as a fallback the <code>kernel8.img</code> (generic, slower ARMv8 kernel for the Pi 4 that
also works for Pi 5) that you find on the [https://github.com/raspberrypi/firmware/tree/master/boot Pi firmware GitHub project].


See the
See [[NixOS on ARM/Raspberry Pi#Board profiles|Board profiles]] for Wi-Fi firmware requirements on custom systems.
[https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#differences-on-raspberry-pi-5 official documentation].


== Setting up a generic UEFI NixOS ==
=== Graphics ===
The task to get a generic NixOS setup requires a


# UEFI boot loader for Pi 5: There exists a WIP [https://github.com/worproject/rpi5-uefi EDK2 for Pi 5 Github project] but with a few limitations see the project for details.
The shared profile defaults request full VC4 KMS. For Xorg, the Pi 5 profile marks the VC4 modesetting device as the primary GPU. This configuration has not been tested with RP1-connected DPI, composite, or MIPI DSI displays.<ref name="profile" /> HDMI and other VC4 display configurations use <code>vc4-kms-v3d</code>. They do not require <code>vc4-kms-v3d-pi5</code>.
# systemd-boot boot loader: Works
# generic Linux kernel that works for the Pi 5's ARM v8 processor and hardware: An almost generic Pi 5 compatible kernel exists [https://github.com/NixOS/nixos-hardware/blob/master/raspberry-pi/5/default.nix at the NixOS-hardware repository]; it's an adaption from a kernel for the Pi 4)


  1. ROM -> 2. EEPROM -> 3. UEFI boot loader (EDK2) -> 4. systemd-boot boot loader -> Pi 4/5-adapted NixOS Kernel -> NixOS
== Storage ==
1. '''Install EDK2''' (UEFI firmware implementation):


We need the first partition of the SD card (or NVMe SSD, etc.) again to be formatted as FAT
The Pi 5 profile prepares Linux and the initrd to use PCIe and NVMe. This allows a system to load U-Boot, the kernel, and the initrd from microSD, then mount its root filesystem from NVMe.
but marked as an ESP (EFI System Partition) to conform to (U)EFI standards.
In this partition we need to place the EDK2 firmware file <code>RPI_EFI.fd</code>
and a <code>config.txt</code> file with a line <code>armstub=RPI_EFI.fd</code>
which instructs the EEPROM boot loader to load EDK2 instead of a Linux loader stub.


See the [https://github.com/worproject/rpi5-uefi EDK2 for Pi 5 Github project]; the releases already contain both of these files.
This Linux support does not mean that released U-Boot can load the NixOS boot files from NVMe. In U-Boot v2026.07, <code>rpi_arm64_defconfig</code> enables PCIe enumeration but does not set <code>CONFIG_NVME_PCI</code> or run <code>nvme scan</code>.<ref name="uboot-config">[https://github.com/u-boot/u-boot/blob/v2026.07/configs/rpi_arm64_defconfig U-Boot v2026.07 rpi_arm64_defconfig]</ref> Its NVMe driver also lacks the PCIe inbound DMA address translation required on Pi 5. A July 2026 patch series proposes those changes, but they are not part of v2026.07.<ref name="nvme-series">[https://lists.denx.de/pipermail/u-boot/2026-July/624063.html U-Boot patch series: Fix NVMe, not only on Raspberry Pi 5]</ref>


See a [https://github.com/NixOS/nixpkgs/issues/260754#issuecomment-1908664693 guide on how to setup partitions and these files].
The Pi EEPROM can discover NVMe media and may load <code>u-boot.bin</code> from it. Released U-Boot cannot then continue the normal extlinux flow from that device. Updating only the EEPROM does not remove this limitation. The stock U-Boot v2026.07 chain therefore does not support complete NVMe boot.


2. '''Install systemd-boot, kernel and NixOS''':
== Serial console ==


The rest is a usual NixOS installation on a second partition with the caveat to '''select a Linux kernel that supports the Pi 5''' like the [https://github.com/NixOS/nixos-hardware/pull/927 Pi 5 compatible Linux kernel is available in nixos-hardware].
Pi 5 exposes its primary UART through the dedicated debug connector. Linux names the hardware device <code>/dev/ttyAMA10</code>, and <code>/dev/serial0</code> points to it.<ref name="uart">[https://www.raspberrypi.com/documentation/computers/configuration.html#configure-uarts Raspberry Pi UART documentation]</ref>


Follow [https://github.com/NixOS/nixpkgs/issues/260754#issuecomment-1936211154 this guide] to build
The profile sets <code>enable_uart=0</code> for Pi 5 to prevent ghost UART input from interrupting U-Boot on affected boards.<ref name="config-defaults">[https://github.com/NixOS/nixos-hardware/blob/master/raspberry-pi/common/config-txt-defaults.nix nixos-hardware Raspberry Pi config.txt defaults]</ref> This example enables a serial console:
a NixOS system closure that you can install manually onto the Pi with a <code>nixos-install</code> call.
That install tool will install the systemd-boot loader at <code>/boot/EFI/systemd/systemd-bootaa64.efi</code>
and the kernel files at <code>/boot/EFI/nixos/*.efi</code> onto your first ESP partition
and the rest of the NixOS system into your second partition.


== Alternative board-specific installation notes ==
<syntaxhighlight lang="nix">
First, install EDK2, following the [https://github.com/worproject/rpi5-uefi#getting-started instructions from the port README]. With EDK2 installed as the Platform Firmware, you can follow the [[NixOS_on_ARM/UEFI|standard instructions for UEFI on ARM]].
{ lib, ... }:
 
EDK2 enables booting a mainline kernel, but hardware support will be very limited. Notably, you'll need to perform the installation using Wi-Fi, as Ethernet is unsupported. Once the system is installed, you can switch to the vendor's modified kernel. This is not (yet?) available in Nixpkgs, so you'll need to get it from [https://gitlab.com/vriska/nix-rpi5 a flake]. If you're not using flakes, you can simply add this to your configuration:
 
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{
  boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5;
}
</nowiki>}}
 
For the vendor kernel to boot properly, you must switch from ACPI to Device Tree in the UEFI settings (at Device Manager → Raspberry Pi Configuration → ACPI / Device Tree → System Table Mode). When using the vendor kernel (which provides full power management support), you may additionally wish to remove <code>force_turbo=1</code> from <code>/boot/config.txt</code>.
 
If you are using nixos-unstable, then you can also use the rpi4 kernel (which is a generic aarch64 kernel for Pi 3 and later models). Although, due to a smaller page size, this will have slightly worse performance:
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{
{
   boot.kernelPackages = pkgs.linuxPackages_rpi4;
  hardware.raspberry-pi.configtxt.settings.pi5.enable_uart = lib.mkForce true;
   boot.kernelParams = [ "console=ttyAMA10,115200n8" ];
}
}
</nowiki>}}
</syntaxhighlight>
 
== Troubleshooting ==
 
=== GPU ===
For the GPU drivers to work, <code>dtoverlay=vc4-kms-v3d-pi5</code> must be added to <code>/boot/config.txt</code>, and the vendor kernel must currently be used. Only Wayland-based compositors are supported without additional configuration (see the nixos-hardware PR linked previously). Note that Xwayland applications may produce broken graphics on KDE; the root cause of this issue has not yet been evaluated.
 
The rpi5-uefi download does not include overlays. You can get them by copying the <code>boot/overlays</code> folder from [https://github.com/raspberrypi/firmware the firmware repository] to <code>/boot</code> (so that <code>/boot/overlays/vc4-kms-v3d-pi5.dtbo</code> is available).
 
=== Bluetooth ===
If your Bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your NixOS configuration:{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{
  boot.kernelParams = [ "8250.nr_uarts=11" "console=ttyAMA10,9600" "console=tty0"];
}
</nowiki>}}
 
== Using the Pi 5 as a remote builder to build native ARM packages for the Pi 5 ==
Building an NixOS system image that can be flashed to an SD card or NVMe SSD requires to '''build ARM binaries''', more specifically for the <code>"aarch64-linux"</code>platform. From a typical Intel/AMD computer we can either
 
* '''emulated native compile using QEMU''' virtualization by enabling the [https://docs.kernel.org/admin-guide/binfmt-misc.html binfmt] kernel feature on NixOS configuration setting <code>boot.binfmt.emulatedSystems = [ "aarch64-linux" ];</code>). This can be fast if everything is downloaded pre-compiled from the cache.nixos.org cache and only few packages really need local compilation. In reality it can be extremely slow, e.g. compiling a Linux kernel alone can take days.
* '''cross-compile''' to ARM using as to happen natively, but nothing will be cached from cache.nixos.org as this is not pre-build. So the compile itself is fast but there will be a lot more to compile locally. In practice it's quite fragile, because you may encounter packages that don't really support cross-compilation get stuck.
* '''native compile on an remote builder''' like the Pi 5 itself running its custom Debian Linux at the beginning or later NixOS. This is quite simple to setup and reasonably fast as most packages are pre-build and cached on cache.nixos.org, and building a remaining Linux kernel only takes 2-3h on the Pi 5.
'''Setting up the Pi 5 as a remote native builder''' can be done following the steps at [[Distributed build|https://wiki.nixos.org/wiki/Distributed_build]]. The rough steps are as follows:
 
# '''Install the Nix package manager''' on Pi 5 Debian OS the normal, multi-user way with <code>sh <(curl -L <nowiki>https://nixos.org/nix/install</nowiki>) --daemon</code>. If you already have NixOS running on the Pi 5, then you can skip this step.
# '''Setup a <code>ssh</code> connection''' from your local machine to the Pi, especially adding <code>SetEnv PATH=/nix/var/nix/profiles/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin</code> to the Pi's <code>/etc/ssh/sshd_config</code> file. If you already have NixOS running on the Pi 5, then you can skip this step.
# '''Make the remote Pi known to you local computer''' by adding it as a <code>nix.buildMachines</code> entry to your  <code>/etc/nix/configuration.nix</code> file and use connection protocol <code>ssh-ng</code>(!).
# You can then '''build, e.g. an NixOS sd card image''' with a call similar to  <code>nix build .\#nixosConfigurations.pi5.config.system.build.sdImage</code>
# '''flash that resulting image onto an SD card''' or NVMe SSD using a call similar to <code>zstdcat result/sd-image/nixos-sd-image-23.11.20230703.ea4c80b-aarch64-linux.img.zst | sudo dd of=/dev/mmcblk0 bs=100M status=progress</code> and place that card into the Pi 5.
 
Missing:


# How to do cross-compilation. ( and/or explain why the above emulation is required at all)
Test U-Boot interaction after enabling the UART, and use a 3.3 V adapter designed for the Pi 5 debug connector.


== Deploy and Update the Pi 5 NixOS system once it's running NixOS ==
== Power and cooling ==
Once the Pi 5 is running NixOS, you can update it with newer NixOS system configurations using e.g. the usual  <code>nix-rebuild</code> 


tool with a call similar to
Raspberry Pi recommends its 27 W USB-C supply for Pi 5. A 5 V, 3 A supply can boot the board, but it reduces the current available to USB devices and fan peripherals. Without a heatsink or fan, the Pi 5 may thermally throttle under sustained load.<ref name="power">[https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#power-supply Raspberry Pi power-supply documentation]</ref>


<code>nixos-rebuild --flake .#pi5 --build-host piuser@pi5 --target-host piuser@pi5 --use-remote-sudo switch</code>
== Alternative implementation ==


that uses the SSH connection from the remote builder section.
The [[NixOS on ARM/Raspberry Pi#Alternative implementations|family page]] describes the [https://github.com/nvmd/nixos-raspberrypi nvmd/nixos-raspberrypi] flake. Its module names and <code>boot.loader.raspberry-pi</code> options are not Nixpkgs or <code>nixos-hardware</code> APIs.


See [https://nixcademy.com/2023/08/10/nixos-rebuild-remote-deployment/ this guide] for a good explanation of this terminal call.
== UEFI ==


== Build NixOS to run on Raspberry Pi 5 ==
The [https://github.com/worproject/rpi5-uefi worproject/rpi5-uefi] EDK2 port is an advanced alternative to U-Boot. The project was archived in February 2025 and is no longer maintained. Its support notice says that the final release was tested on early BCM2712C1 boards and reports problems with D0 boards and newer EEPROM firmware. It is therefore not the default Pi 5 installation path. For experiments, follow the archived project's limitations and the [[NixOS on ARM/UEFI|general ARM UEFI instructions]].
Above here, may be out of date.


The best resource found as of 2025 Jan 25th is this issue:
== See also ==


https://github.com/NixOS/nixpkgs/issues/260754
* [[NixOS on ARM/Raspberry Pi]]
* [[NixOS on ARM/Installation]]
* [https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#nvme-ssd-boot Raspberry Pi NVMe documentation]


Specifically, if you follow the instructions in this comment, it works.
== References ==


https://github.com/NixOS/nixpkgs/issues/260754#issuecomment-2501839916
<references />


[[Category:NixOS on ARM]]
[[Category:NixOS on ARM]]
Retrieved from "https://wiki.nixos.org/wiki/NixOS_on_ARM/Raspberry_Pi_5"