NixOS on ARM: Difference between revisions

Latest revision as of 20:56, 25 June 2024

NixOS supports 64-bit ARM.

The support level for ARM overall varies depending on the architecture and the specific ecosystems and boards.

The way the ARM integration is built into NixOS is by making generic builds the first-class citizens. This means that as soon as there is upstream support for the board in the kernel and platform firmware, NixOS should work once those are updated.

It is still possible, when needed, to build and use a customized platform firmware and kernel for specific boards^{[reference needed]}.

At this moment in time (early 2024) only AArch64 has full support upstream. With that said, neither armv6l or armv7l are being ignored, fixes are worked on and approved as needed. What's missing is support and builds being maintained in binary form. At the time of writing, no publicly available caches for armv6l or armv7l are available.

For images links, including UEFI install, skip to the Installation page.

Supported devices

Table legend:

Upstream (NixOS) supported devices

NixOS has support for these boards using AArch64 architecture on the nixpkgs-unstable and stable channel.

Support for those board assumes as much is supported as Mainline Linux supports.

Manufacturer	Board	SoC	ISA	CPU	RAM	Storage
Raspberry Pi Foundation	Raspberry Pi 3	Broadcom BCM2837	AArch64 / ARMv7	4× Cortex-A53 @ 1.2 - 1.4 GHz	1 GB	SD/microSD
Raspberry Pi Foundation	Raspberry Pi 4	Broadcom BCM2711	AArch64 / ARMv7	4× Cortex-A72 @ 1.5 - 1.8 GHz	1-8 GB	microSD, eMMC

Community supported devices

These boards are not routinely verified as working.

The baseline support level expected is “Just as much as mainline Linux and U-Boot supports them”, except if specified otherwise.

Manufacturer	Board	SoC	ISA	CPU	RAM	Storage
Apple	Apple Silicon Macs	M1/M1 Pro/M1 Max	AArch64	—	—	NVMe
ASUS	Tinker Board	Rockchip RK3288	ARMv7	4× Cortex-A17	2 GB	microSD
Banana Pi	Banana Pi	Allwinner A20	ARMv7	2× Cortex-A7	1 GB	SD, SATA
Banana Pi M64	Banana Pi M64	Allwinner A64	AArch64	4× Cortex-A53	2 GB	microSD, 8GB eMMc
Banana Pi BPI-M5	Banana Pi BPI-M5	Amlogic S905X3	AArch64	4× Cortex-A55	4 GB LPDDR4	microSD, 16G eMMC
BeagleBoard.org	BeagleBone Black	TI AM335x (src)	ARMv7	1× Cortex-A8 @ 1 GHz	512 MB	4 GB eMMC, microSD
Firefly	AIO-3399C	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	2/4 GB	8/16 GB eMMC, microSD
FriendlyElec	NanoPC-T4	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	4 GB	16 GB eMMC, microSD, NVMe
FriendlyElec	NanoPi-M4	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	4 GB	optional eMMC, microSD
FriendlyElec	NanoPi-R6C	Rockchip RK3588S	AArch64	4× ARM Cortex-A76 @ 2.4 GHz, 4× Cortex-A55 @ 1.8 Ghz	4 GB / 8 GB	optional eMMC, microSD, NVMe
Hardkernel	ODROID-HC1 & ODROID-HC2	Samsung Exynos 5422	ARMv7	4× Cortex-A15 @ 2GHz, 4× Cortex-A7 @ 1.4GHz	2 GB	microSD
Hardkernel	ODROID-C2	Amlogic S905	AArch64	4× Cortex-A53 @ 1.5GHz	2 GB	eMMC, microSD
Hardkernel	ODROID-HC4	Amlogic S905X3	AArch64	4× Cortex-A55 @ 1.8GHz	4 GB	microSD, SATA
Kosagi	Kosagi Novena	i.MX6	ARMv7	4× Cortex-A9 @ 1.2 GHz	4 GB	microSD, SATA
Libre Computer	ROC-RK3399-PC	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	4 GB	eMMC, microSD, NVMe
Libre Computer	ROC-RK3328-CC	Rockchip RK3328	AArch64	4× Cortex-A53 @ 1.4GHz	4 GB	eMMC, microSD
Libre Computer	AML-S905X-CC-V2	Amlogic S905X	AArch64	4× Cortex-A53 @ 1.512 GHz	1/2GB	eMMC, microSD
Linksprite	pcDuino3 Nano	Allwinner A20	ARMv7	2× Cortex-A7 @ 1 GHz	1 GB	4 GB NAND, microSD, SATA
NVIDIA	Jetson TK1	Tegra K1/T124	ARMv7	4× Cortex-A15 @ 2.3 GHz	2 GB	16 GB eMMC, SD, SATA
NXP	i.MX 8M Plus EVK	i.MX 8M Plus	AArch64	4× Cortex-A53 @ 1.8 Ghz	6 GB	32 GB eMMC, microSD
NXP	i.MX 8M Quad EVK	i.MX 8M Quad	AArch64	4× Cortex-A53 @ 1.5 Ghz + 1× Cortex-M4	3 GB	16 GB eMMC, microSD
OLIMEX	Teres-A64	AllWinner A64	AArch64	4× Cortex-A53 @ 1.1 GHz	2GB	16 GB eMMC, microSD
Orange Pi	Orange Pi One	Allwinner H3	ARMv7	4× Cortex-A7 @ 1.2 GHz	512 MB	microSD
Orange Pi	Orange Pi PC	Allwinner H3	ARMv7	4× Cortex-A7 @ 1.6 GHz	1 GB	SD/microSD
Orange Pi	Orange Pi Zero Plus2 (H5)	Allwinner H5	AArch64	4× Cortex-A53 @ 1.2 GHz	1 GB	SD/microSD + 8GB eMMC
Orange Pi	Orange Pi Zero2 (H616)	Allwinner H616	AArch64	4× Cortex-A53 @ 1.2 GHz	1 GB	SD/microSD + 2MB SPI Flash
Orange Pi	Orange Pi R1 Plus LTS	Rockchip RK3328	AArch64	4× Cortex-A53 @ 1.5 GHz	1 GB	microSD
Orange Pi	Orange Pi 5	Rockchip RK3588s	AArch64	4× Cortex-A76 @ 2.4GHz, 4×Cortex-A55 @ 1.8 GHz	4/8/16 GB	microSD, NVMe
Orange Pi	Orange Pi 5 Plus	Rockchip RK3588	AArch64	4× Cortex-A76 @ 2.4GHz, 4×Cortex-A55 @ 1.8 GHz	4/8/16 GB	eMMC, microSD, NVMe
PINE64	PINE A64-LTS	Allwinner R18	AArch64	4× Cortex-A53 @ ? GHz	2 GB	microSD & eMMC
PINE64	Pinebook	Allwinner A64	AArch64	4× Cortex-A53 @ ? Ghz	2 GB	microSD & eMMC
PINE64	Pinebook Pro	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	4 GB	microSD & eMMC
PINE64	ROCK64	Rockchip RK3328	AArch64	4× Cortex-A53 @ 1.5 GHz	1/2/4 GB	microSD/eMMC
PINE64	ROCKPro64	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	2/4 GB	microSD/eMMC
Clockworkpi	uConsole A06	Rockchip RK3399	AArch64	2× Cortex-A72 @ 2.0 GHz, 4× Cortex-A53 @ 1.5 Ghz	4 GB	microSD
Radxa	ROCK5 Model B	Rockchip RK3588	AArch64	4× Cortex-A76 @ 2.4GHz, 4×Cortex-A55 @ 1.8 GHz	4/8/16 GB	eMMC, microSD, NVMe
Radxa	ROCK5 Model A	Rockchip RK3588s	AArch64	4× Cortex-A76 @ 2.4GHz, 4×Cortex-A55 @ 1.8 GHz	4/8/16 GB	eMMC, microSD, NVMe
Raspberry Pi Foundation	Raspberry Pi	Broadcom BCM2835	ARMv6	1 × ARM1176 @ 700 MHz	256 MB / 512 MB	SD/microSD
Raspberry Pi Foundation	Raspberry Pi 2	Broadcom BCM2836	ARMv7	4× Cortex-A7 @ 900 MHz	1 GB	SD/microSD
Raspberry Pi Foundation	Raspberry Pi 3	Broadcom BCM2837	AArch64 / ARMv7	4× Cortex-A53 @ 1.2 GHz	1 GB	SD/microSD
Raspberry Pi Foundation	Raspberry Pi 4	Broadcom BCM2711	AArch64 / ARMv7	4× Cortex-A53 @ 1.5 GHz	1-8 GB	microSD
Raspberry Pi Foundation	Raspberry Pi 5	Broadcom BCM2712	AArch64	4× Cortex-A76 @ 2.4 GHz	4-8 GB	microSD
Toshiba	AC100 (mini laptop)	Tegra 2 250 (T20)	ARMv7	2× Cortex-A9 @ 1 GHz	512 MB	8–32 GB eMMC, SD
Wandboard	Wandboard Solo/Dual/Quad	Freescale i.MX6	ARMv7	1×/2×/4× Cortex-A9 @ 1000 MHz	512 MB / 1 GB / 2 GB	microSD, SATA

Special Devices

It is possible to emulate an ARM platform with QEMU.

Manufacturer	Board	SoC	ISA	CPU	RAM	Storage
QEMU	QEMU	—	ARMv7	up to 8	up to 2 GB	Anything QEMU supports

Installation

Main article: NixOS_on_ARM/Installation

Initial configuration

Main article: NixOS_on_ARM/Initial_Configuration

Troubleshooting

Hanging at `Starting kernel ...`

When booting a NixOS system, it may look like it is hung at Starting kernel ....

It is unlikely to be hung at this part. This is the last message printed by U-Boot. U-Boot prints to both the display and the console at once.

To view the kernel messages and boot output, the “stdout” of the kernel needs to be configured appropriately.

The kernel's stdout will differ depending on semantics.

When there are no valid console= kernel command-line arguments, it will default to the /chosen/stdout-path device tree property. (Generally a serial console.)
When valid console= parameters are present, the kernel picks the leftmost valid one as stdout.

In other words, the solution may be to enable the appropriate console= parameters according to your hardware configuration and system.

For the display, add console=tty0.
For a serial console, refer to the target device configuration.

Details about the boot process

NixOS can also be booted through UEFI on ARM. The semantics are generally the same as on other architectures. Do note that the common use of Device Tree instead of ACPI in consumer-class hardware may make this a bit more awkward.

Otherwise, in SBC-class hardware, it is common that boards are generally expected to use U-Boot as the platform firmware and bootloader. See the section about using NixOS with U-Boot.

Binary caches

AArch64

The official NixOS Hydra instance builds a full set of binaries (available on https://cache.nixos.org) for the AArch64 architecture on the nixpkgs-unstable and stable channels.

armv6l and armv7l

Some users have provided best effort caches in the past for 32 bit ARM, but none are currently available.

Getting Support

There is a dedicated room for the upstream NixOS effort on Matrix, #nixos-on-arm:nixos.org.

Don't hesitate to ask questions. Note that reply times may vary greatly, depending on the provided information.

Resources

Subpages

The following is a list of all sub-pages of the NixOS on ARM topic.

@@ Line 9: / Line 9: @@
 At this moment in time (early 2024) '''only AArch64''' has full support upstream. With that said, neither armv6l or armv7l are being ignored, fixes are worked on and approved as needed. What's missing is support and builds being maintained in binary form. At the time of writing, no publicly available caches for armv6l or armv7l are available.<!-- please get in touch with samueldr on the NixOS on ARM channel if you want to share your own cache, before editing. We need to review the wording to make it crystal clear it's a user-provided cache. -->
-'''For images links, including UEFI install''', skip to the [[NixOS on ARM#Installation|Installation]] section.
+'''For images links, including UEFI install''', skip to the [[NixOS_on_ARM/Installation|Installation page]].
 == Supported devices ==
@@ Line 65: / Line 65: @@
 -->
 <div class="table">
-{|class="table"
+{| class="sortable table"
 !width="2%"| Manufacturer
 !width="2%"| Board
@@ Line 83: / Line 83: @@
 |-
 | ASUS
-| [[NixOS_on_ARM/Tinker Board|Tinker Board]]
+| [[NixOS_on_ARM/ASUS Tinker Board|Tinker Board]]
 | Rockchip RK3288
 | ARMv7
@@ Line 101: / Line 101: @@
 | [[NixOS_on_ARM/Banana Pi M64|Banana Pi M64]]
 | Allwinner A64
-| ARMv8
+| AArch64
 | 4× Cortex-A53
 | 2 GB
@@ Line 109: / Line 109: @@
 | [[NixOS_on_ARM/Banana Pi BPI-M5|Banana Pi BPI-M5]]
 | Amlogic S905X3
-| ARMv8.2
+| AArch64
 | 4× Cortex-A55
 | 4 GB LPDDR4
@@ Line 456: / Line 456: @@
 {{main|NixOS_on_ARM/Installation}}
-== Binary caches ==
+== Initial configuration ==
-=== AArch64 ===
-The [https://hydra.nixos.org/ official NixOS Hydra] instance builds a full set of binaries (available on https://cache.nixos.org) for the AArch64 architecture on the nixpkgs-unstable and stable channels.
-=== armv6l and armv7l ===
-Some '''''users''''' have provided best effort caches for 32 bit ARM, but none are currently available.
-== Enable UART ==
-If you try to use UART to log on NixOS, you might hang on the line "Starting kernel ...". To enable UART, you will need to add at the end of the line that contains <code>loglevel4</code> in the file <code>/extlinux/extlinux.conf</code> the text:
-{{file|/extlinux/extlinux.conf||<nowiki>
-    console=ttyAMA0,115200n8
-</nowiki>}}
-{{file|/extlinux/extlinux.conf||<nowiki>
-    console=ttyS0,115200n8
-</nowiki>}}
-The actual device (<code>ttyAMA0</code>, <code>ttyS0</code>, <code>ttyS1</code>) will depend on the hardware.
-== NixOS installation & configuration ==
 {{main|NixOS_on_ARM/Initial_Configuration}}
-== Details about the boot process ==
+== Troubleshooting ==
-NixOS can be booted through [[NixOS on ARM/UEFI|UEFI]] on ARM too. The semantics are generally the same as on other architectures. Do note that the common use of Device Tree instead of ACPI in consumer-class hardware ''may'' make this a bit more awkward.
+=== Hanging at <code>Starting kernel ...</code> ===
+When booting a NixOS system, it may look like it is hung at <code>Starting kernel ...</code>.
-Otherwise, in SBC-class hardware, it is common that boards are generally expected to use U-Boot as the platform firmware and bootloader. See the section about [[U-Boot#Using NixOS with U-Boot|using NixOS with U-Boot]].
+It is unlikely to be hung at this part. This is the last message printed by U-Boot. U-Boot prints to both the display and the console at once.
-== Porting NixOS to new boards ==
+To view the kernel messages and boot output, the “<code>stdout</code>” of the kernel needs to be configured appropriately.
-=== The easiest way ===
+The kernel's <code>stdout</code> will differ depending on semantics.
-Assuming upstream U-Boot supports the board through a defconfig, it is possible possible to build U-Boot using the cross-compiling architecture from an x86_64 host. Here's a sample use.
+* When there are no valid <code>console=</code> kernel command-line arguments, it will default to the <code>/chosen/stdout-path</code> device tree property. (Generally a serial console.)
+* When valid <code>console=</code> parameters are present, the kernel picks the leftmost valid one as <code>stdout</code>.
-<syntaxhighlight lang="shell-session">
+In other words, the solution may be to enable the appropriate <code>console=</code> parameters according to your hardware configuration and system.
-# Assuming you're in a recent nixpkgs checkout
-$ nix-shell \
-    -I "nixpkgs=$PWD" \
-    -p 'let plat = pkgsCross.aarch64-multiplatform; in plat.buildUBoot{defconfig = "orangepi_zero_plus2_defconfig"; extraMeta.platforms = ["aarch64-linux"]; BL31 = "${plat.armTrustedFirmwareAllwinner}/bl31.bin"; filesToInstall = ["u-boot-sunxi-with-spl.bin"];}'
-</syntaxhighlight>
-For armv7 and armv6 <code>pkgsCross.arm-embedded</code> should work, this is available in the unstable channel (19.03 and following) by setting <code>-I "nixpkgs=/path/to/new-nixpkgs-checkout</code>.
+* For the display, add <code>console=tty0</code>.
+* For a serial console, refer to the target device configuration.
-This should build whatever is needed for, and then build U-Boot for the desired defconfig, then open a shell with the build in <code>$buildInputs</code>. Do note that this particular invocation may need more changes than only the defconfig if built for other than allwinner boards.
+=== Details about the boot process ===
-Here's an example command, for allwinner boards, on how to write to an SD card.
+NixOS can also be booted through [[NixOS on ARM/UEFI|UEFI]] on ARM. The semantics are generally the same as on other architectures. Do note that the common use of Device Tree instead of ACPI in consumer-class hardware [[NixOS_on_ARM/UEFI#Device_Trees|''may'' make this a bit more awkward]].
-<syntaxhighlight lang="shell-session">
+Otherwise, in SBC-class hardware, it is common that boards are generally expected to use U-Boot as the platform firmware and bootloader. See the section about [[U-Boot#Using NixOS with U-Boot|using NixOS with U-Boot]].
-$ sudo dd if=$buildInputs/u-boot-sunxi-with-spl.bin of=/dev/sdX bs=1024 seek=8
-</syntaxhighlight>
-=== The easy way ===
+== Binary caches ==
-''(if you're lucky)''
+=== AArch64 ===
-If your board is an ARMv7 board supported by multi_v7_defconfig and you have access to U-Boot on the board, getting <code>sd-image-armv7l-linux.img</code> to boot is the easiest option:
+The [https://hydra.nixos.org/ official NixOS Hydra] instance builds a full set of binaries (available on https://cache.nixos.org) for the AArch64 architecture on the nixpkgs-unstable and stable channels.
-* If you're lucky and your U-Boot build comes with the extlinux.conf support built in, the image boots out-of-the-box. This is the case for all (upstream) Allwinner and Tegra U-Boots, for instance.
+=== armv6l and armv7l ===
-* Otherwise, you can get the boot information (path to kernel zImage, initrd, DTB, command line arguments) by extracting <code>extlinux.conf</code> from the boot partition of the image, and then attempt to boot it via the U-Boot shell, or some other mechanism that your board's distro uses (e.g. <code>uEnv.txt</code>).
-==== Building U-Boot from your NixOS PC ====
-Assuming
-* Your board is supported upstream by U-Boot or there is a recent enough fork with <code>extlinux.conf</code> support.
-* You do not have nix setup on an ARM device
-* Your nix isn't setup for cross-compilation
-It is still possible to build U-Boot using tools provided by NixOS.
-In the following terminal session, replace <code>orangepi_pc_defconfig</code> with the appropriate board [http://git.denx.de/?p=u-boot.git;a=tree;f=configs;hb=HEAD from the configs folder] of U-Boot.
-{{Commands|<nowiki>
-$ nix-shell -E 'with import <nixpkgs> {}; stdenv.mkDerivation { name = "arm-shell"; buildInputs = [git gnumake gcc gcc-arm-embedded dtc]; }'
-$ git clone git://git.denx.de/u-boot.git
-$ cd u-boot
-# We're checking out a version from before the use of `binman`.
-# The dtc package is 1.4.2, which does not include `pylibftd`.
-# Furthermore, I do not know how to package the library so it would be
-# available in the python interpreter, making binman happy.
-$ git checkout v2017.03
-$ make -j4 CROSS_COMPILE=arm-none-eabi- orangepi_pc_defconfig
-$ make -j4 CROSS_COMPILE=arm-none-eabi-
-</nowiki>}}
-The name of the final file will change depending on the board. For this specific build, and most Allwinner builds, the file will be named <code>u-boot-sunxi-with-spl.bin</code>.
-You can flash this file to boot device with
-{{Commands|<nowiki>
- dd if=u-boot-sunxi-with-spl.bin of=/dev/sdX bs=1024 seek=8
-</nowiki>}}
-Note: This mailing list contains a patch which may help some builds: https://lists.denx.de/pipermail/u-boot/2016-December/275664.html
-=== The hard way ===
-Alternatively/if all else fails, you can do it the hard way and bootstrap NixOS from an existing ARM Linux installation.
-=== Contributing new boards to nixpkgs ===
-* Add a new derivation for your board's U-Boot configuration, see for example ubootPine64LTS in {{Nixpkgs Link|short=all-packages.nix|pkgs/top-level/all-packages.nix}}.
+Some '''''users''''' have provided best effort caches in the past for 32 bit ARM, but none are currently available.
-* If your board's U-Boot configuration doesn't use the <code>extlinux.conf</code> format by default, create a patch to enable it. Some C hacking skills & U-Boot knowledge might be required. For some pointers, see this patch to enable it on the Versatile Express.
-* Make a pull request, also containing the board-specific instructions.
 == Getting Support ==