Install NixOS on Hetzner Online

Note: This article is about installing NixOS on Hetzner Online, which provides dedicated bare-metal servers. It is not to be confused with Hetzner Cloud, which provides VPS cloud servers.

Installation

There are three ways at the time to install NixOS on Hetzner dedicated:

1. From Hetzner's rescue image, one can boot into the nixos installer using a custom kexec image that is configured with the fixed IPv6 provided by Hetzner and also contain your ssh key. Tip: The kexec tarball as generated by nixos-generators can remain put into the /boot partition for future use.
2. Hetzner also provides an interface to upload your own ISO-images. Also, here you may want to build your own iso-image, which has openssh with ssh keys due the lack of a remote console.
3. An easier method to install NixOS on Hetzner, is to use the existing integration into NixOps.
4. An example to install NixOS in the Hetzner rescue mode, including full RAID partitioning, is available here.

Network configuration

Hetzner Online offers both IPv4 (usually in a shared /26 or /27 subnet) and IPv6 (/64 subnet) connectivity to each machine. The assigned addresses can be looked up on the Hetzner Robot on the IPs tab of a machine. The public IPv4 address of the server can automatically be obtained via DHCP. For IPv6 you have to statically configure both address and gateway.

{
  systemd.network = {
    enable = true;
    networks.default = {
      name = "enp1s0"; # The name of the interface
      DHCP = "ipv4";
      addresses = [ 
        {
          # Replace the address with the one assigned to your machine
          Address = "2a01:4f8:AAAA:BBBB::1/64"; 
        }
      ];
      gateway = [ "fe80::1" ];
      linkConfig.RequiredForOnline = "routable";
    };
  };
}

Static IPv4 configuration

Since the IPv4 network configuration is known, it can also be configured statically, preventing reliance on the DHCP service. The gateway and subnet information is visible when hovering the IPv4 address. The subnet size is usually a /26 (255.255.255.224) or a /27 (255.255.255.192).

{
  systemd.network = {
    enable = true;
    networks."30-wan" = {
      name = "enp1s0"; # The predictable name of the network interface
      DHCP = "no";
      addresses = [
        # Replace the addresses with the ones assigned to your machine
        {
          Address = "A.B.C.D/26";
        }
        {
          Address = "2a01:4f8:AAAA:BBBB::1/64";
        }
      ];
      gateway = [
        # Replace the gateway address with the one in your subnet
        "A.B.C.E"
        "fe80::1"
      ];
      linkConfig.RequiredForOnline = "routable";
    };
  };
}

Bootstrap from the Rescue System

Here are some quick notes on how to bootstrap.

The nixos-install-scripts repo may also be a valuable resource:

https://github.com/nix-community/nixos-install-scripts/tree/master/hosters/hetzner-dedicated

Otherwise, inspiration for the kexec approach below comes from https://github.com/ofborg/infrastructure/commit/0712a5cf871b7a6d2fbbd2df539d3cd90ab8fa1f and https://github.com/andir/infra/tree/master/bootstrap

The main principle is that we will go from: Rescue system, install Nix, kexec into a NixOS system, finally install the system.

First, reboot the machine in Rescue mode. Note that just enabling Rescue mode from the dashboard doesn't immediately reboot so make sure to power cycle the server. The Rescue mode runs from a RAM disk, so make also sure that you have enough RAM. Temporarily rescaling to 32 GiB of RAM (the RAM disk will be half of the available RAM) during the bootstrapping process helps. Make sure to select your SSH public key. SSH into the machine:

You can skip the entire next part by using https://github.com/nix-community/nixos-images#kexec-tarballs

# The installer needs sudo
apt install -y sudo

# Let root run the nix installer
mkdir -p /etc/nix
echo "build-users-group =" > /etc/nix/nix.conf

# Install Nix in single-user mode
curl -L https://nixos.org/nix/install | sh
. $HOME/.nix-profile/etc/profile.d/nix.sh

# Install nixos-generators
# This might take a while, so the verbose flag `-v` is included to monitor progress
nix-env -f https://github.com/nix-community/nixos-generators/archive/1.7.0.tar.gz -i -v

# Create a initial config, just to kexec into
cat <<EOF > /root/config.nix
{
  services.openssh.enable = true;
  users.users.root.openssh.authorizedKeys.keys = [
    # Replace with your public key
    "ssh-rsa AAAA..."
  ];
}
EOF

# Generate the kexec script
nixos-generate -o /root/result  -f kexec-bundle -c /root/config.nix

# Switch to the new system
/root/result

At this point, the shell should stop responding. Kill the shell and ssh back into the machine. The server public key will have changed.

format() {
  parted -s "$1" -- mklabel msdos
  parted -s "$1" -- mkpart primary 1MiB 512MiB
  parted -s "$1" -- set 1 boot on
  parted -s "$1" -- mkpart primary 512MiB 100%
  parted -s "$1" -- print
}

# In this particular machine we have two NVMe disks
# If your machine has > 2TB drives, open a ticket and ask for UEFI boot, it will save you a lot of hassle 
format /dev/nvme0n1
format /dev/nvme1n1

# Here we create a single btrfs volume using both disks. Change as needed

# TODO: Use boot.loader.grub.mirroredBoots
mkfs.ext2 /dev/nvme0n1p1
mkfs.btrfs -d raid0 -m raid1 -L nixos /dev/nvme0n1p2 /dev/nvme1n1p2

# Mount the disks
mount /dev/disk/by-label/nixos /mnt
mkdir /mnt/boot
mount /dev/nvme0n1p1 /mnt/boot

# Generate the NixOS configuration.
nixos-generate-config --root /mnt

At this point, edit the /mnt/etc/nixos/configuration.nix and tune as needed. I just added the following lines:

boot.loader.grub.device = "/dev/nvme0n1";
services.openssh.enable = true;
users.users.root.openssh.authorizedKeys.keys = [
  "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGB1Pog97SWdV2UEA40V+3bML+lSZXEd48zCRlS/eGbY3rsXfgUXb5FIBulN9cET9g0OOAKeCZBR1Y2xXofiHDYkhk298rHDuir6cINuoMGUO7VsygUfKguBy63QMPHYnJBE1h+6sQGu/3X9G2o/0Ys2J+lZv4+N7Hqolhbg/Cu6/LUCsJM/udqTVwJGEqszDWPtuuTAIS6utB1QdL9EZT5WBb1nsNyHnIlCnoDKZvrrO9kM0FGKhjJG2skd3+NqmLhYIDhRhZvRnL9c8U8uozjbtj/N8L/2VCRzgzKmvu0Y1cZMWeAAdyqG6LoyE7xGO+SF4Vz1x6JjS9VxnZipIB zimbatm@nixos"
];

Finally run nixos-install, and then reboot the machine.

Voila! (after 1000 steps)