NixOS on ARM/Raspberry Pi 5: Difference between revisions
WIP, sorry for the temporary formatting mess, I need to go back to the visual editor, but can't without losing changes :( |
Add current working maintained solution, reformat the page into alternative options |
||
| Line 38: | Line 38: | ||
=== Generic UEFI boot support === | === Generic UEFI boot support === | ||
NixOS doesn't run out-of-box, but relies on several tweaks on the boot process that are maintained by different individuals and spread over multiple repositories. The Raspberry Pi 5's boot process follows the [https://youtu.be/UvFG76qM6co?t=308 typical boot stages on embedded devices], and has the following boot loader steps by default: | NixOS doesn't run out-of-box, but relies on several tweaks on the boot process that are maintained by different individuals and spread over multiple repositories. The Raspberry Pi 5's boot process follows the [https://youtu.be/UvFG76qM6co?t=308 typical boot stages on embedded devices], and has the following boot loader steps by default: | ||
[[File:Raspberry Pi 5 Boot Process.png|alt=A flowchart showing 4 boxes with text in each, one leading to another. In the first box, ROM. In the second, EEPROM. In the third, Firmware / Kernel. In the fourth, Raspberry Pi OS.|center|thumb|500x500px|The normal boot process of a Raspberry Pi 5]] | |||
<!-- I'm including here the Mermaid code to generate this diagram. It would be amazing if we could do --> | <!-- I'm including here the Mermaid code to generate this diagram. It would be amazing if we could do --> | ||
<!-- this via a template, but until then, here it is: --> | <!-- this via a template, but until then, here it is: --> | ||
| Line 46: | Line 46: | ||
<!-- FK -\-> OS[Raspberry Pi OS] --> | <!-- FK -\-> OS[Raspberry Pi OS] --> | ||
The pain points for NixOS support are the Pi's [https://github.com/raspberrypi/rpi-eeprom custom EEPROM boot bootloader], its [https://github.com/raspberrypi/firmware proprietary, closed-source firmware] (code to use to hardware components) and [https://github.com/raspberrypi/linux its separately maintained Linux kernel], all of which we would need to update, build and test constantly and separately from the other NixOS Linux kernel variations, which is a large, unmaintainable burden for the NixOS community when the Pi 5 is not the only supported SoC. | |||
The more sustainable goal would be to move towards UEFI support, which would mimic how desktop computers boot into the NixOS operating system, thus minimising the amount of bespoke maintenance needed for the development board. This is what this process would look like: | |||
[[File:Raspberry Pi 5 Ideal Boot Process.png|alt=A flowchart containing 5 boxes with text. In this order, each box contains the following: ROM, EEPROM, UEFI Bootloader, Systemd Bootloader, Generic Linux Kernel, and NixOS.|center|thumb|500x500px|The ideal Raspberry Pi 5 boot process]] | |||
The ROM and EEPROM can't be modified, as they are built into the hardware on the development board. However, [https://de.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface UEFI] is also used for booting normal Intel/AMD computers, and the [https://www.freedesktop.org/software/systemd/man/latest/systemd-boot.html systemd-boot] boot loader is THE software that allows us to have and select from multiple NixOS generations on boot (and perform rollbacks if we messed up). Further details about the steps involved in achieving this can be found in the ''Other Solutions'' section, under ''Generic UEFI Boot.'' | |||
Finally, there should also be [https://github.com/u-boot/u-boot U-Boot] support soon, as most development boards are widely supported by the project. In that case, the boot process would look like: | |||
[[File:Raspberry Pi 5 U-Boot Boot Process.png|alt=A flowchart showing 5 boxes containing text. In this order, each box contains the following text: ROM, EEPROM, Firmware or Kernel, U-Boot, and NixOS.|center|thumb|500x500px|The U-Boot boot process]] | |||
== Proposed Solution == | |||
There is a repository aiming to develop a declarative way of defining Raspberry Pi setups, including, but not limited to, the Raspberry Pi 5. The project is called [https://github.com/nvmd/nixos-raspberrypi?tab=readme-ov-file nixos-raspberrypi], and so far seems to be successful at achieving functioning system builds. It also provides a binary cache, speeding up iteration and deployment. | |||
== Other Solutions == | |||
In general, as show in in the first section, the closed source and proprietary software powering part of the Raspberry Pi development boards is a big source of contention. Therefore, there are largely two kinds of solutions going forward: '''moving towards a generic boot process''', and '''using proprietary software''', each with advantages and disadvantages. | |||
=== Using proprietary software === | |||
Besides the current ''Proposed Solution'', there has been one more attempt at https://github.com/tstat/raspberry-pi-nix (just follow the example and hints at https://github.com/tstat/raspberry-pi-nix/issues/13) and has the best out-of-the-box experience. You have to remote-build many Nix packages, probably the kernel as well, yourself (e.g. using the [https://wiki.nixos.org/wiki/Distributed_build Pi without NixOS as an intermediate remote builder]) and that can take several hours though. | |||
=== Generic UEFI Boot === | |||
This solution is technically more complex than the others, and the overview is separated into sections. | |||
== Raspberry Pi Boot stages == | ==== Raspberry Pi Boot stages ==== | ||
To understand the adaptions for NixOS better, it's helpful to understand more about the stages: | To understand the adaptions for NixOS better, it's helpful to understand more about the stages: | ||
| Line 112: | Line 121: | ||
[https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#differences-on-raspberry-pi-5 official documentation]. | [https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#differences-on-raspberry-pi-5 official documentation]. | ||
== Setting up a generic UEFI NixOS == | ==== Setting up a generic UEFI NixOS ==== | ||
The task to get a generic NixOS setup requires a | The task to get a generic NixOS setup requires a | ||
| Line 142: | Line 151: | ||
and the rest of the NixOS system into your second partition. | and the rest of the NixOS system into your second partition. | ||
== Alternative board-specific installation notes == | ==== Alternative board-specific installation notes ==== | ||
First, install EDK2, following the [https://github.com/worproject/rpi5-uefi#getting-started instructions from the port README]. With EDK2 installed as the Platform Firmware, you can follow the [[NixOS_on_ARM/UEFI|standard instructions for UEFI on ARM]]. | First, install EDK2, following the [https://github.com/worproject/rpi5-uefi#getting-started instructions from the port README]. With EDK2 installed as the Platform Firmware, you can follow the [[NixOS_on_ARM/UEFI|standard instructions for UEFI on ARM]]. | ||
| Line 162: | Line 171: | ||
</nowiki>}} | </nowiki>}} | ||
== Troubleshooting == | ==== Troubleshooting ==== | ||
=== GPU === | ===== GPU ===== | ||
For the GPU drivers to work, <code>dtoverlay=vc4-kms-v3d-pi5</code> must be added to <code>/boot/config.txt</code>, and the vendor kernel must currently be used. Only Wayland-based compositors are supported without additional configuration (see the nixos-hardware PR linked previously). Note that Xwayland applications may produce broken graphics on KDE; the root cause of this issue has not yet been evaluated. | For the GPU drivers to work, <code>dtoverlay=vc4-kms-v3d-pi5</code> must be added to <code>/boot/config.txt</code>, and the vendor kernel must currently be used. Only Wayland-based compositors are supported without additional configuration (see the nixos-hardware PR linked previously). Note that Xwayland applications may produce broken graphics on KDE; the root cause of this issue has not yet been evaluated. | ||
The rpi5-uefi download does not include overlays. You can get them by copying the <code>boot/overlays</code> folder from [https://github.com/raspberrypi/firmware the firmware repository] to <code>/boot</code> (so that <code>/boot/overlays/vc4-kms-v3d-pi5.dtbo</code> is available). | The rpi5-uefi download does not include overlays. You can get them by copying the <code>boot/overlays</code> folder from [https://github.com/raspberrypi/firmware the firmware repository] to <code>/boot</code> (so that <code>/boot/overlays/vc4-kms-v3d-pi5.dtbo</code> is available). | ||
=== Bluetooth === | ===== Bluetooth ===== | ||
If your Bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your NixOS configuration:{{file|/etc/nixos/configuration.nix|nix|<nowiki> | If your Bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your NixOS configuration:{{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
{ | { | ||
| Line 176: | Line 186: | ||
</nowiki>}} | </nowiki>}} | ||
== Using the Pi 5 as a remote builder to build native ARM packages for the Pi 5 == | ==== Using the Pi 5 as a remote builder to build native ARM packages for the Pi 5 ==== | ||
Building an NixOS system image that can be flashed to an SD card or NVMe SSD requires to '''build ARM binaries''', more specifically for the <code>"aarch64-linux"</code>platform. From a typical Intel/AMD computer we can either | Building an NixOS system image that can be flashed to an SD card or NVMe SSD requires to '''build ARM binaries''', more specifically for the <code>"aarch64-linux"</code>platform. From a typical Intel/AMD computer we can either | ||
| Line 194: | Line 204: | ||
# How to do cross-compilation. ( and/or explain why the above emulation is required at all) | # How to do cross-compilation. ( and/or explain why the above emulation is required at all) | ||
== Deploy and Update the Pi 5 NixOS system once it's running NixOS == | ==== Deploy and Update the Pi 5 NixOS system once it's running NixOS ==== | ||
Once the Pi 5 is running NixOS, you can update it with newer NixOS system configurations using e.g. the usual <code>nix-rebuild</code> | Once the Pi 5 is running NixOS, you can update it with newer NixOS system configurations using e.g. the usual <code>nix-rebuild</code> | ||
| Line 204: | Line 214: | ||
See [https://nixcademy.com/2023/08/10/nixos-rebuild-remote-deployment/ this guide] for a good explanation of this terminal call. | See [https://nixcademy.com/2023/08/10/nixos-rebuild-remote-deployment/ this guide] for a good explanation of this terminal call. | ||
[[Category:NixOS on ARM]] | [[Category:NixOS on ARM]] | ||