NixOS on ARM
Installation images and miscellaneous boot files for ARM devices are built & hosted by @dezgeg at http://nixos-arm.dezgeg.me/installer.
A binary cache, containing a subset of the unstable channel, is hosted at http://nixos-arm.dezgeg.me/channel (signed with key nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%).
Additionally, thanks to Graham Christensen and Packet the official NixOS Hydra builds a full set of binaries (available on https://cache.nixos.org) for the AArch64 architecture on the nixpkgs-unstable channel.
Supported devices
The installation images should work on the following devices:
| Manufacturer | Board | SoC | ISA | CPU | RAM | Storage |
|---|---|---|---|---|---|---|
| BeagleBoard.org | Beaglebone Black | TI AM335x | ARMv7 | 1x Cortex-A8 @ 1 GHz | 512 MB | 4 GB eMMC, microSD |
| Linksprite | pcDuino3 Nano | Allwinner A20 | ARMv7 | 2x Cortex-A7 @ 1 GHz | 1 GB | 4 GB NAND, microSD, SATA |
| NVIDIA | Jetson TK1 | Tegra K1/T124 | ARMv7 | 4x Cortex-A15 @ 2.3 GHz | 2 GB | 16 GB eMMC, SD, SATA |
| Raspberry Pi Foundation | Raspberry Pi | Broadcom BCM2835 | ARMv6 | 1x ARM1176 @ 700 MHz | 256 MB / 512 MB | SD/microSD |
| Raspberry Pi Foundation | Raspberry Pi 2 | Broadcom BCM2836 | ARMv7 | 4x Cortex-A7 @ 900 MHz | 1 GB | SD/microSD |
| Raspberry Pi Foundation | Raspberry Pi 3 | Broadcom BCM2837 | AArch64 / ARMv7 | 4x Cortex-A53 @ 1.2 GHz | 1 GB | SD/microSD |
| Toshiba AC100 (mini laptop) | Compal PAZ00 | Tegra 2 250 (T20) | ARMv7 | 2x Cortex-A9 @ 1 GHz | 512 MB | 8-32 GB eMMC, SD |
| Wandboard | Wandboard Solo/Dual/Quad | Freescale i.MX6 | ARMv7 | 1x/2x/4x Cortex-A9 @ 1000 MHz | 512 MB / 1 GB / 2 GB | microSD, SATA |
QEMU emulation is not supported as of right nowTalk.
Installation
The installation images come in two flavors: sd-image-armv6l-linux.img is built for the ARMv6 architecture and it comes with the Raspberry Pi kernel. sd-image-armv7l-linux.img is built for the ARMv7 architecture and comes with the mainline multiplatform ARMv7 kernel (multi_v7_defconfig). Make sure you download the correct image for your board!
The .img files can be directly written to a microSD/SD card (minimal recommended size: 4 GB) using dd.
sudo dd if=sd-image-armv7l-linux.img of=/dev/sdX
Replace /dev/sdX with the path to your SD card device.
The base images are configured to boot up with a serial TTY ( RX/TX UART ) @ 115200 Baud. That way you not necessarily have to have a HDMI Display and keyboard.
Board-specific installation notes
Depending on the board, some additional preparation steps might be needed to make the SD card bootable on your device.
Beaglebone Black
U-Boot and its SPL (called the MLO) need to be copied to specific sectors on the microSD card with dd. Download U-Boot & MLO for the board (uboot-am335x_boneblack_defconfig-2017.03_u-boot.img, uboot-am335x_boneblack_defconfig-2017.03_MLO), and copy them to the correct location with (again, replace /dev/sdX with the correct path to the SD card device):
sudo dd if=uboot-am335x_boneblack_defconfig-2017.03_MLO of=/dev/sdX count=1 seek=1 bs=128k
sudo dd if=uboot-am335x_boneblack_defconfig-2017.03_u-boot.img of=/dev/sdX count=2 seek=1 bs=384k
In case there is a valid boot loader on the eMMC of the board and the NixOS U-Boot isn't getting launched, try holding the button labeled 'S2' when resetting the board. Also the boot ROM might have a size limitation on the microSD cards used for booting; the author wasn't able to get a 64 GB SDXC card working but a 8 GB SDHC card worked.
Jetson TK1
The proprietary NVIDIA bootloader can only boot NVIDIA's L4T kernel, so it needs to be replaced by flashing U-Boot on the board's eMMC via the recovery USB port. The easiest way to do that is to use tegra-uboot-flasher-scripts, though unfortunately that's currently not packaged in NixOS. Once U-Boot is flashed, the ARMv7 image will boot out-of-the-box.
pcDuino 3 Nano
U-Boot needs to be copied to specific sectors on the microSD card with dd. Download U-Boot for the board (uboot-Linksprite_pcDuino3_Nano_defconfig-2015.07_u-boot-sunxi-with-spl.bin), and copy it to the correct location with (again, replace /dev/sdX with the correct path to the SD card device):
sudo dd if=uboot-Linksprite_pcDuino3_Nano_defconfig-2015.07_u-boot-sunxi-with-spl.bin of=/dev/sdX bs=1024 seek=8
Raspberry Pi (1)
The ARMv6 image boots out-of-the-box.
Raspberry Pi 2
The ARMv7 image should boot out-of-the-box, though the author hasn't personally tested this.
Raspberry Pi 3
Both the AArch64 and ARMv7 images boot out-of-the-box. Using the 64-bit AArch64 image is highly recommended, as the availability of binaries is much better and allows the use of the 64-bit instruction set.
Use the following GPIO Pins with an USB-TTL connector:
GND - 3rd in top row, black cable
GPIO 14 TXD - 4th in top row, white cable
GPIO 15 RXD - 5th in top row, green cableUse nix-shell -p screen --run "screen /dev/ttyUSB0 115200" to connect to the console.
{
...
hardware.enableRedistributableFirmware = true;
hardware.firmware = [
(pkgs.stdenv.mkDerivation {
name = "broadcom-rpi3-extra";
src = pkgs.fetchurl {
url = "https://raw.githubusercontent.com/RPi-Distro/firmware-nonfree/54bab3d/brcm80211/brcm/brcmfmac43430-sdio.txt";
sha256 = "19bmdd7w0xzybfassn7x4rb30l70vynnw3c80nlapna2k57xwbw7";
};
phases = [ "installPhase" ];
installPhase = ''
mkdir -p $out/lib/firmware/brcm
cp $src $out/lib/firmware/brcm/brcmfmac43430-sdio.txt
'';
})
];
networking.wireless.enable = true;
}
Wandboard
U-Boot and its SPL need to be copied to specific sectors on the microSD card with dd. Download U-Boot & SPL for the board (uboot-wandboard_defconfig-2017.03_u-boot.img, uboot-wandboard_defconfig-2017.03_SPL), and copy them to the correct location with (again, replace /dev/sdX with the correct path to the SD card device):
sudo dd if=uboot-wandboard_defconfig-2017.03_SPL of=/dev/sdX seek=1 bs=1k
sudo dd if=uboot-wandboard_defconfig-2017.03_u-boot.img of=/dev/sdX seek=69 bs=1k
Only the Quad model has been tested, but the others should work as well with the same U-Boot binary.
Toshiba AC100
The Toshiba AC100 used to ship with Android 2, the fastboot bootloader and a custom partition table format. Nowadays, you can install U-boot on it. It is advisable to make a back-up (documented elsewhere; you'll need nvflash as documented below).
In order to modify the operating system, you need to connect the AC100 with a mini-usb cable to an Intel system we'll call the host. On the host, build the nvflash tool:
pkgs.stdenv.mkDerivation {
name = "tegra-driver-package";
version = "r16.5.0";
src = pkgs.fetchurl {
url = http://developer.download.nvidia.com/mobile/tegra/l4t/r16.5.0/ventana_release_armhf/Tegra20_Linux_R16.5_armhf.tbz2;
sha256 = "11qmqj0yillwfapzcfjr4px4z8l7zkh0v3xrfpr9riff54xsidq2";
};
configurePhase = "";
buildPhase = "";
checkPhase = "";
installPhase = ''
mkdir $out
cp -r * $out/
'';
}
Build the nvflash tool and follow the instructions for installing U-boot starting with the getbct.sh step at https://paz00.ru/index.php/Migrate_to_U-Boot#With_prebuilt_sosuboot-tegra_archive_or_sdcard_image_.28PC_not_required.29
When U-boot is installed, proceed with the sd-image-armv7l-linux.img image from https://www.cs.helsinki.fi/u/tmtynkky/nixos-arm/installer/ and follow the general NixOS installation instructions. If the first installation attempt fails, you will need to use the U-boot command line to select an alternative installation medium, using something like (please correct):
setenv boot_targets=usb0
run distro_bootEnable UART
If you try to use UART to log on NixOs, you will hang on the line "Starting kernel ...". To enable UART, you will need to add at the end of the line that contains loglevel4 in the file /extlinux/extlinux.conf the text
/etc/nixos/configuration.nix
console=ttyAMA0,115200n8
NixOS installation & configuration
The installation image is actually a MBR partition table plus two partitions; a FAT32 /boot and a ext4 root filesystem. The image is designed such that it's possible to directly reuse the SD image's partition layout and "install" NixOS on the very same SD card by simply replacing the default configuration.nix and running nixos-rebuild. Using this installation method is strongly recommended, though if you know exactly what you're doing and how U-Boot on your board works, you can use nixos-install as usual. To help with the SD card installation method, the boot scripts on the image automatically resize the rootfs partition to fit the SD card on the first boot.
Use this as a template:
/etc/nixos/configuration.nix
{ config, pkgs, lib, ... }:
{
# NixOS wants to enable GRUB by default
boot.loader.grub.enable = false;
# Enables the generation of /boot/extlinux/extlinux.conf
boot.loader.generic-extlinux-compatible.enable = true;
# !!! If your board is a Raspberry Pi 1, select this:
boot.kernelPackages = pkgs.linuxPackages_rpi;
# !!! Otherwise (even if you have a Raspberry Pi 2 or 3), pick this:
boot.kernelPackages = pkgs.linuxPackages_latest;
# !!! This is only for ARMv6 / ARMv7. Don't enable this on AArch64, cache.nixos.org works there.
nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];
# !!! Needed for the virtual console to work on the RPi 3, as the default of 16M doesn't seem to be enough.
boot.kernelParams = ["cma=32M"];
# File systems configuration for using the installer's partition layout
fileSystems = {
"/boot" = {
device = "/dev/disk/by-label/NIXOS_BOOT";
fsType = "vfat";
};
"/" = {
device = "/dev/disk/by-label/NIXOS_SD";
fsType = "ext4";
};
};
# !!! Adding a swap file is optional, but strongly recommended!
# swapDevices = [ { device = "/swapfile"; size = 1024; } ];
}
Note: the default configuration.nix will contain something like imports = [ <nixos/modules/installer/cd-dvd/sd-image-armv7l-multiplatform.nix> ]; do not include that in your final installation or you will experience interesting problems. It is only for building the installation image!
To make the ARM experience slightly less painful, the config template adds nixos-arm.dezgeg.me as a binary cache, which contains a small subset of packages on the unstable channel (though a caution for US users: the server hosting them is physically located in Finland). Note that the binary cache isn't enabled on the prebuilt images, so enable it via the command line when building for the first time:
nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%
Resizing the boot partition
It is possible that you run out of disk space on the boot partition after some system upgrades. To resize the boot partition:
- (If not already done, boot once to trigger the initial partition resizing)
- Backup the files currently stored in the boot partition
- Repartition and make sure to delete and then recreate the boot partition (fat32, primary, label:
NIXOS_BOOT) - Copy the files from the backup back to the new boot partition
Details about the boot process
On NixOS, all ARM boards use the popular U-Boot as the bootloader and U-Boot's Generic Distro Configuration Concept as the mechanism to communicate boot information (such as path to kernel zImage, initrd, DTB, command line arguments). For a quick TL;DR about the generic distro configuration support: U-Boot is scripted to scan all attached storage devices & partitions and look for a file named /extlinux/extlinux.conf or /boot/extlinux/extlinux.conf (which will be generated by NixOS, just like /boot/grub/grub.cfg is generated on PCs).
U-Boot also provides an interactive shell and the generation selection menu (just like GRUB). However, support for input or display devices varies greatly, depending on the board:
Beaglebone Black
Only serial console (via the 6-pin FTDI pin header) is supported.
Jetson TK1
Only serial console (via the DB-9 connector) is supported.
pcDuino 3 Nano
USB keyboards and HDMI displays work perfectly. Also a 3.3v serial port via the 3-pin header works.
Raspberry Pi (all versions)
USB keyboards and HDMI displays work perfectly. Using the 3.3v serial port via the pin headers (exact location depends on hardware version) should also work.
Wandboard
Only serial console (via the DB-9 connector) is supported.
Porting NixOS to new boards
The easy way
(if you're lucky)
If your board is an ARMv7 board supported by multi_v7_defconfig and you have access to U-Boot on the board, getting sd-image-armv7l-linux.img to boot is the easiest option:
- If you're lucky and your U-Boot build comes with the extlinux.conf support built in, the image boots out-of-the-box. This is the case for all (upstream) Allwinner and Tegra U-Boots, for instance.
- Otherwise, you can get the boot information (path to kernel zImage, initrd, DTB, command line arguments) by extracting
extlinux.conffrom the boot partition of the image, and then attempt to boot it via the U-Boot shell, or some other mechanism that your board's distro uses (e.g.uEnv.txt).
Building u-boot from your NixOS PC
Assuming
- Your board is supported upstream by u-boot or there is a recent enough fork with
extlinux.confsupport. - You do not have nix setup on an ARM device
- Your nix isn't setup for cross-compilation
It is still possible to build u-boot using tools provided by NixOS.
In the following terminal session, replace orangepi_pc_defconfig with the appropriate board from the configs folder of u-boot.
$ nix-shell -E 'with import <nixpkgs> {}; stdenv.mkDerivation { name = "arm-shell"; buildInputs = [git gnumake gcc gcc-arm-embedded dtc]; }'
$ git clone git://git.denx.de/u-boot.git
$ cd u-boot
# We're checking out a version from before the use of `binman`.
# The dtc package is 1.4.2, which does not include `pylibftd`.
# Furthermore, I do not know how to package the library so it would be
# available in the python interpreter, making binman happy.
$ git checkout v2017.03
$ make -j4 ARCH=arm CROSS_COMPILE=arm-none-eabi- orangepi_pc_defconfig
$ make -j4 ARCH=arm CROSS_COMPILE=arm-none-eabi-
The name of the final file will change depending on the board. For this specific build, and most Allwinner builds, the file will be named u-boot-sunxi-with-spl.bin.
The hard way
Alternatively/if all else fails, you can do it the hard way and bootstrap NixOS from an existing ARM Linux installation.
Contributing new boards to nixpkgs
- Add a new derivation for your board's U-Boot configuration, see for example ubootJetsonTK1 in
all-packages.nix - If your board's U-Boot configuration doesn't use the
extlinux.confformat by default, create a patch to enable it. Some C hacking skills & U-Boot knowledge might be required. For some pointers, see this patch to enable it on the Versatile Express. - Make a pull request, also containing the board-specific instructions. Ping @dezgeg for review and for building & hosting the U-Boots at http://nixos-arm.dezgeg.me/installer.