NixOS on ARM/Raspberry Pi 5: Difference between revisions
TobiasBora (talk | contribs) No edit summary |
|||
| (8 intermediate revisions by 3 users not shown) | |||
| Line 37: | Line 37: | ||
=== Generic UEFI boot support === | === Generic UEFI boot support === | ||
NixOS doesn't run out-of-box, but relies on several tweaks on the boot process that are maintained by different individuals and spread over multiple repositories. The Raspberry Pi 5's boot process follows the [https://youtu.be/UvFG76qM6co?t=308 typical boot stages on embedded devices], and has the following boot loader steps by default: | NixOS doesn't run out-of-box, but relies on several tweaks on the boot process that are maintained by different individuals and spread over multiple repositories. The Raspberry Pi 5's boot process follows the [https://youtu.be/UvFG76qM6co?t=308 typical boot stages on embedded devices], and has the following boot loader steps by default:{{caption|align=center|{{mermaid|flowchart LR| | ||
[[ | ROM --> EEPROM | ||
EEPROM --> FK[Firmware / Kernel] | |||
FK --> OS[Raspberry Pi OS] | |||
}}|The normal boot process of a Raspberry Pi 5|alt=A flowchart showing 4 boxes with text in each, one leading to another. In the first box, ROM. In the second, EEPROM. In the third, Firmware / Kernel. In the fourth, Raspberry Pi OS.}} | |||
The pain points for NixOS support are the Pi's [https://github.com/raspberrypi/rpi-eeprom custom EEPROM boot bootloader], its [https://github.com/raspberrypi/firmware proprietary, closed-source firmware] (code to use to hardware components) and [https://github.com/raspberrypi/linux its separately maintained Linux kernel], all of which we would need to update, build and test constantly and separately from the other NixOS Linux kernel variations, which is a large, unmaintainable burden for the NixOS community when the Pi 5 is not the only supported SoC. | The pain points for NixOS support are the Pi's [https://github.com/raspberrypi/rpi-eeprom custom EEPROM boot bootloader], its [https://github.com/raspberrypi/firmware proprietary, closed-source firmware] (code to use to hardware components) and [https://github.com/raspberrypi/linux its separately maintained Linux kernel], all of which we would need to update, build and test constantly and separately from the other NixOS Linux kernel variations, which is a large, unmaintainable burden for the NixOS community when the Pi 5 is not the only supported SoC. | ||
The more sustainable goal would be to move towards UEFI support, which would mimic how desktop computers boot into the NixOS operating system, thus minimising the amount of bespoke maintenance needed for the development board. This is what this process would look like: | The more sustainable goal would be to move towards UEFI support, which would mimic how desktop computers boot into the NixOS operating system, thus minimising the amount of bespoke maintenance needed for the development board. This is what this process would look like: | ||
[[ | {{caption|align=center|{{mermaid|flowchart LR| | ||
ROM --> EEPROM | |||
EEPROM --> UEFI[UEFI Bootloader] | |||
UEFI --> SD[Systemd Bootloader] | |||
SD --> LK[Generic Linux Kernel] | |||
LK --> OS[Raspberry Pi OS] | |||
}}|The ideal Raspberry Pi 5 boot process|alt=A flowchart containing 5 boxes with text. In this order, each box contains the following: ROM, EEPROM, UEFI Bootloader, Systemd Bootloader, Generic Linux Kernel, and NixOS.}} | |||
The ROM and EEPROM can't be modified, as they are built into the hardware on the development board. However, [https://de.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface UEFI] is also used for booting normal Intel/AMD computers, and the [https://www.freedesktop.org/software/systemd/man/latest/systemd-boot.html systemd-boot] boot loader is THE software that allows us to have and select from multiple NixOS generations on boot (and perform rollbacks if we messed up). Further details about the steps involved in achieving this can be found in the ''Other Solutions'' section, under ''Generic UEFI Boot.'' | The ROM and EEPROM can't be modified, as they are built into the hardware on the development board. However, [https://de.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface UEFI] is also used for booting normal Intel/AMD computers, and the [https://www.freedesktop.org/software/systemd/man/latest/systemd-boot.html systemd-boot] boot loader is THE software that allows us to have and select from multiple NixOS generations on boot (and perform rollbacks if we messed up). Further details about the steps involved in achieving this can be found in the ''Other Solutions'' section, under ''Generic UEFI Boot.'' | ||
Finally, there should also be [https://github.com/u-boot/u-boot U-Boot] support soon, as most development boards are widely supported by the project. In that case, the boot process would look like: | Finally, there should also be [https://github.com/u-boot/u-boot U-Boot] support soon, as most development boards are widely supported by the project. In that case, the boot process would look like: | ||
[[ | {{caption|align=center|{{mermaid|flowchart LR| | ||
ROM --> EEPROM | |||
EEPROM --> FK[Firmware / Kernel] | |||
FK --> U[U-Boot Bootloader] | |||
U --> OS[Raspberry Pi OS] | |||
}}|Theoretical Raspberry Pi 5 U-Boot boot process|alt=A flowchart showing 5 boxes containing text. In this order, each box contains the following text: ROM, EEPROM, Firmware or Kernel, U-Boot, and NixOS.}} | |||
== Proposed Solution == | == Proposed Solution == | ||
| Line 70: | Line 78: | ||
=== Step 1: Building the SD Image === | === Step 1: Building the SD Image === | ||
{{Note|It might be possible to do the following steps via the official SD installer as well. If you manage to do it, please mark this section as optional and remove this notice.}}The project provides build images as well, which are modified versions of the nix-hardware ones. Before getting into building one, in order to make use of the community binary caches, and potentially avoid rebuilding the image, make sure you are added to the trusted users in your own nix configuration: | {{Note|It might be possible to do the following steps via the official SD installer as well. If you manage to do it, please mark this section as optional and remove this notice.}}The project provides build images as well, which are modified versions of the nix-hardware ones. Before getting into building one, in order to make use of the community binary caches, and potentially avoid rebuilding the image, make sure you are added to the trusted users in your own nix configuration: | ||
{{File| | {{File|||3=trusted-users = username|name=nix.conf|lang=conf}} | ||
Or, alternatively, in your nixos configuration: | Or, alternatively, in your nixos configuration: | ||
{{File| | {{File|||3={ | ||
nix.settings.trusted-users = [ "your-username" ]; | nix.settings.trusted-users = [ "your-username" ]; | ||
<nowiki>}</nowiki>}} | <nowiki>}</nowiki>|name=configuration.nix|lang=nix}} | ||
Afterwards, you can build one of the SD images:<syntaxhighlight lang=" | Afterwards, you can build one of the SD images:<syntaxhighlight lang="console"> | ||
$ nix build github: | $ nix build github:nvmd/nixos-raspberrypi#installerImages.rpi5 | ||
</syntaxhighlight>You can either [[Cross Compiling|cross-compile]] the image on your system, or alternatively, you can install the "normal" Raspberry Pi OS on your Raspberry Pi 5, then install Nix standalone (multi-user), and set it up as a [[distributed build]] machine. | </syntaxhighlight>You can either [[Cross Compiling|cross-compile]] the image on your system, or alternatively, you can install the "normal" Raspberry Pi OS on your Raspberry Pi 5, then install Nix standalone (multi-user), and set it up as a [[distributed build]] machine. | ||
Finally, simply copy the image to the current directory from the <code>result</code> directory, extract it and write it to the SD card. You can find more information on the [[NixOS on ARM/Installation]] page. | Finally, simply copy the image to the current directory from the <code>result</code> directory, extract it and write it to a USB drive, and boot to this drive on the raspberry pi (by default the raspberry pi will first try to boot the SD card, then the USB drive, so just don't put any SD card when booting and plug it later. Note that since the installer will format the SD card, you cannot put the installer on the SD card itself). You can find more information on the [[NixOS on ARM/Installation]] page. | ||
=== Step 2: Installing NixOS === | === Step 2: Installing NixOS === | ||
Once you're running an installer image, you can create a flake referencing the repository. For example:{{File| | Once you're running an installer image, you can create a flake referencing the repository. For example:{{File|3={ | ||
description = "Example Raspberry Pi 5 configuration flake"; | description = "Example Raspberry Pi 5 configuration flake"; | ||
inputs = { | inputs = { | ||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05"; | nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05"; | ||
nixos-raspberrypi.url = "github:nvmd/nixos-raspberrypi"; | nixos-raspberrypi.url = "github:nvmd/nixos-raspberrypi/main"; | ||
}; | }; | ||
| Line 147: | Line 155: | ||
}; | }; | ||
}; | }; | ||
}}}Finally, simply <code>nixos-rebuild switch --flake .#yourHostname</code> the flake. | <nowiki>}</nowiki>|name=flake.nix|lang=nix}}Finally, simply <code>nixos-rebuild switch --flake .#yourHostname</code> the flake. | ||
== Other Solutions == | == Other Solutions == | ||
| Line 245: | Line 253: | ||
EDK2 enables booting a mainline kernel, but hardware support will be very limited. Notably, you'll need to perform the installation using Wi-Fi, as Ethernet is unsupported. Once the system is installed, you can switch to the vendor's modified kernel. This is not (yet?) available in Nixpkgs, so you'll need to get it from [https://gitlab.com/vriska/nix-rpi5 a flake]. If you're not using flakes, you can simply add this to your configuration: | EDK2 enables booting a mainline kernel, but hardware support will be very limited. Notably, you'll need to perform the installation using Wi-Fi, as Ethernet is unsupported. Once the system is installed, you can switch to the vendor's modified kernel. This is not (yet?) available in Nixpkgs, so you'll need to get it from [https://gitlab.com/vriska/nix-rpi5 a flake]. If you're not using flakes, you can simply add this to your configuration: | ||
{{file| | {{file|3=<nowiki> | ||
{ | { | ||
boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5; | boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5; | ||
} | } | ||
</nowiki>}} | </nowiki>|name=configuration.nix|lang=nix}} | ||
For the vendor kernel to boot properly, you must switch from ACPI to Device Tree in the UEFI settings (at Device Manager → Raspberry Pi Configuration → ACPI / Device Tree → System Table Mode). When using the vendor kernel (which provides full power management support), you may additionally wish to remove <code>force_turbo=1</code> from <code>/boot/config.txt</code>. | For the vendor kernel to boot properly, you must switch from ACPI to Device Tree in the UEFI settings (at Device Manager → Raspberry Pi Configuration → ACPI / Device Tree → System Table Mode). When using the vendor kernel (which provides full power management support), you may additionally wish to remove <code>force_turbo=1</code> from <code>/boot/config.txt</code>. | ||
If you are using nixos-unstable, then you can also use the rpi4 kernel (which is a generic aarch64 kernel for Pi 3 and later models). Although, due to a smaller page size, this will have slightly worse performance: | If you are using nixos-unstable, then you can also use the rpi4 kernel (which is a generic aarch64 kernel for Pi 3 and later models). Although, due to a smaller page size, this will have slightly worse performance: | ||
{{file| | {{file|||<nowiki> | ||
{ | { | ||
boot.kernelPackages = pkgs.linuxPackages_rpi4; | boot.kernelPackages = pkgs.linuxPackages_rpi4; | ||
} | } | ||
</nowiki>}} | </nowiki>|name=configuration.nix|lang=nix}} | ||
==== Troubleshooting ==== | ==== Troubleshooting ==== | ||
| Line 269: | Line 277: | ||
===== Bluetooth ===== | ===== Bluetooth ===== | ||
If your Bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your NixOS configuration:{{file| | If your Bluetooth doesn't show up, and you are getting errors in dmesg regarding the serial port at 107050c00, add the following to your NixOS configuration:{{file|||<nowiki> | ||
{ | { | ||
boot.kernelParams = [ "8250.nr_uarts=11" "console=ttyAMA10,9600" "console=tty0"]; | boot.kernelParams = [ "8250.nr_uarts=11" "console=ttyAMA10,9600" "console=tty0"]; | ||
} | } | ||
</nowiki>}} | </nowiki>|name=configuration.nix|lang=nix}} | ||
==== Using the Pi 5 as a remote builder to build native ARM packages for the Pi 5 ==== | ==== Using the Pi 5 as a remote builder to build native ARM packages for the Pi 5 ==== | ||