Install NixOS on Hetzner Online

Note: This article is about installing NixOS on Hetzner Online, which provides dedicated bare-metal servers. It is not to be confused with Hetzner Cloud, which provides VPS cloud servers.

Installation

There are three ways at the time to install NixOS on Hetzner dedicated:

1. From Hetzner's rescue image one can boot into the nixos installer using a custom kexec image that is configured with the fixed IPv6 provided by Hetzner and also contain your ssh key. Tip: The kexec tarball as generated by nixos-generators can remain put into the /boot partition for future use.
2. Hetzner also provides an interface to upload your own ISO-images. Also here you may want to build your own iso-image, which has openssh with ssh keys due the lack of a remote console.
3. An easier method to install NixOS on Hetzner, is to use the existing integration into NixOps.
4. An example to install NixOS in the Hetzner rescue mode, including full RAID partitioning, is available here.

Network configuration

From Hetzner's web interface, one can obtain both ipv4/ipv6 addresses and gateways. Hetzner does announce ipv6 addresses servers, so you need to assign those statically. In this example we use networkd to configure the interface. The same configuration can be used for both the kexec installation image and the final server configuration.

{ ... }: {
  # This make sure that our interface is named `eth0`.
  # This should be ok as long as you don't have multiple physical network cards
  # For multiple cards one could add a netdev unit to rename the interface based on the mac address
  networking.usePredictableInterfaceNames = false;
  systemd.network = {
    enable = true;
    networks."eth0".extraConfig = ''
      [Match]
      Name = eth0
      [Network]
      # Add your own assigned ipv6 subnet here here!
      Address = 2a01:4f9:ffff::1/64
      Gateway = fe80::1
      # optionally you can do the same for ipv4 and disable DHCP (networking.dhcpcd.enable = false;)
      # Address =  144.x.x.x/26
      # Gateway = 144.x.x.1
    '';
  };
}

Another possibility is to use networking.interfaces:

let
  external-mac = "00:11:22:33:44:55";
  ext-if = "et0";
  external-ip = "144.x.x.x";
  external-gw = "144.x.x.255";
  external-ip6 = "2a01:XXXX:XXXX::1";
  external-gw6 = "fe80::1";
  external-netmask = 27;
  external-netmask6 = 64;
in {
  # rename the external interface based on the MAC of the interface
  services.udev.extraRules = ''SUBSYSTEM=="net", ATTR{address}=="${external-mac}", NAME="${ext-if}"'';
  networking = {
    interfaces."${ext-if}" = {
      ipv4.addresses = [{
        address = external-ip;
        prefixLength = external-netmask;
      }];
      ipv6.addresses = [{
        address = external-ip6;
        prefixLength = external-netmask6;
      }];
    };
    defaultGateway6 = {
      address = external-gw6;
      interface = ext-if;
    };
    defaultGateway = external-gw;
  };
}

Bootstrap from the Rescue System

Here are some quick notes on how to bootstrap.

The nixos-install-scripts repo may also be a valuable resource:

https://github.com/nix-community/nixos-install-scripts/tree/master/hosters/hetzner-dedicated

Otherwise, inspiration for the kexec approach below comes from https://github.com/ofborg/infrastructure/commit/0712a5cf871b7a6d2fbbd2df539d3cd90ab8fa1f and https://github.com/andir/infra/tree/master/bootstrap

The main principle is that we will go from: Rescue system, install Nix, kexec into a NixOS system, finally install the system.

First, reboot the machine in Rescue mode. Note that just enabling Rescue mode from the dashboard doesn't immediately reboot so make sure to power cycle the server. The Rescue mode runs from a RAM disk, so make also sure that you have enough RAM. Temporarily rescaling to 32 GiB of RAM (the RAM disk will be half of the available RAM) during the bootstrapping process helps. Make sure to select your SSH public key. SSH into the machine:

# The installer needs sudo
apt install -y sudo

# Let root run the nix installer
mkdir -p /etc/nix
echo "build-users-group =" > /etc/nix/nix.conf

# Install Nix in single-user mode
curl -L https://nixos.org/nix/install | sh
. $HOME/.nix-profile/etc/profile.d/nix.sh

# Install nixos-generators
# This might take a while, so the verbose flag `-v` is included to monitor progress
nix-env -f https://github.com/nix-community/nixos-generators/archive/master.tar.gz -i -v

# Create a initial config, just to kexec into
cat <<EOF > /root/config.nix
{
  services.openssh.enable = true;
  users.users.root.openssh.authorizedKeys.keys = [
    # Replace with your public key
    "ssh-rsa AAAA..."
  ];
}
EOF

# Generate the kexec script
nixos-generate -o /root/result  -f kexec-bundle -c /root/config.nix

# Switch to the new system
/root/result

At this point the shell should stop responding. Kill the shell and ssh back into the machine. The server public key will have changed.

format() {
  parted -s "$1" -- mklabel msdos
  parted -s "$1" -- mkpart primary 1MiB 512MiB
  parted -s "$1" -- set 1 boot on
  parted -s "$1" -- mkpart primary 512MiB 100%
  parted -s "$1" -- print
}

# In this particular machine we have two NVMe disks
format /dev/nvme0n1
format /dev/nvme1n1

# Here we create a single btrfs volume using both disks. Change as needed

# TODO: Use boot.loader.grub.mirroredBoots
mkfs.ext2 /dev/nvme0n1p1
mkfs.btrfs -d raid0 -m raid1 -L nixos /dev/nvme0n1p2 /dev/nvme1n1p2

# Mount the disks
mount /dev/disk/by-label/nixos /mnt
mkdir /mnt/boot
mount /dev/nvme0n1p1 /mnt/boot

# Generate the NixOS configuration.
nixos-generate-config --root /mnt

At this point, edit the /mnt/etc/nixos/configuration.nix and tune as needed. I just added the following lines:

boot.loader.grub.device = "/dev/nvme0n1";
services.openssh.enable = true;
users.users.root.openssh.authorizedKeys.keys = [
  "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGB1Pog97SWdV2UEA40V+3bML+lSZXEd48zCRlS/eGbY3rsXfgUXb5FIBulN9cET9g0OOAKeCZBR1Y2xXofiHDYkhk298rHDuir6cINuoMGUO7VsygUfKguBy63QMPHYnJBE1h+6sQGu/3X9G2o/0Ys2J+lZv4+N7Hqolhbg/Cu6/LUCsJM/udqTVwJGEqszDWPtuuTAIS6utB1QdL9EZT5WBb1nsNyHnIlCnoDKZvrrO9kM0FGKhjJG2skd3+NqmLhYIDhRhZvRnL9c8U8uozjbtj/N8L/2VCRzgzKmvu0Y1cZMWeAAdyqG6LoyE7xGO+SF4Vz1x6JjS9VxnZipIB zimbatm@nixos"
];

Finally run nixos-install, and then reboot the machine.

Voila! (after 1000 steps)