Install NixOS on Hetzner Cloud

Note: This article is about installing NixOS on Hetzner Cloud, which provides VPS cloud servers. It is not to be confused with Hetzner Online, which provides dedicated bare-metal servers.

Hetzner Cloud in 2023 offers amd64 instances on both Intel and AMD hardware, as well as arm64 instanced based on the Ampere Altra Q80, in three locations in Europe, as well as two in America.

The pricing of the arm64 instances is generally considered very competitive, compared to similarly specced amd64 instances.

Installation

There are several ways to install NixOS, such as the "traditional" ISO installation, nixos-infect or nixos-anywhere.

Tradition ISO installation

TODO

nixos-anywhere

The tutorial assumes you already have an account on Hetzner Cloud, and no prior access to a system with NixOS or nix CLI utility installed:

Create a temp folder for future use. Run:
```
mkdir /tmp/my-first-flake
```
Note: this folder will be mounted into docker container on the next step. Having the folder on the host system enables editing files using a familiar editor, available on the host system, such as VS Code or neovim.
Enter docker container. Run:
```
docker run --rm --interactive --tty --mount type=bind,source=/tmp/my-first-flake,target=/tmp/my-first-flake alpine:3.20 ash
```
Note: this is done in a container in order to reduce the "setup footprint and residue", allowing to throw away this setup environment quickly.
Install nix and hcloud CLI utilities. Run:
```
apk add nix hcloud
```
Authenticate hcloud CLI utility. Run:
```
hcloud context create my-first-context
```
When asked, enter value of the token in the prompt. Note: the token with "Read/Write" permissions can be obtained on a project page inside Hetzner Cloud: https://console.hetzner.cloud/projects/0000000/security/tokens
Create a VM on Hetzner. Run:
```
hcloud server create --name my-hetzner-vm --type cpx21 --image ubuntu-24.04 --location fsn1
```
Write down IP and generated user password. Note 1: this tutorial uses cpx21 VM instance type which corresponds to an x86 architecture marchine with 3 CPU cores and 4GB of RAM, and fsn1 location which corresponds to a data center in the city of Falkenstein in Germany. A list of all instance types can be obtained by running command hcloud server-type list, while a list of all locations can be obtained by running hcloud location list command. Note 2: Hopefully, Hetzner Cloud team will support NixOS disk images soon, see Could we convince Hetzner to add Nixos as a standard image choice.
SSH onto the server using IP and root password obtained after the previous step:
```
ssh root@0.0.0.0
```
Change the password to something temporal, for example: my-temp-password-123.

On your host computer, create a folder. Run:

mkdir -p /tmp/my-first-flake/my-vms/my-hetzner-vm/

Using a code editor on your host computer, create 4 files. File contents, as well as the location of where to put corresponding file are indicated below:

# /tmp/my-first-flake/my-vms/my-hetzner-vm/hardware-configuration.nix

{ config, lib, pkgs, modulesPath, ... }:

{
  imports = [
    (modulesPath + "/profiles/qemu-guest.nix")
  ];

  boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ ];
  boot.extraModulePackages = [ ];
  swapDevices = [ ];
  networking.useDHCP = lib.mkDefault true;
  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}

# /tmp/my-first-flake/my-vms/my-hetzner-vm/disko-config.nix

{
  disko.devices = {
    disk = {
      main = {
        type = "disk";
        device = "/dev/sda";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02";
              priority = 1;
            };
            ESP = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            };
            root = {
              size = "100%";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/";
              };
            };
          };
        };
      };
    };
  };
}

# /tmp/my-first-flake/my-vms/my-hetzner-vm/configuration.nix

{ config, lib, pkgs, ... }:

{
  imports =
    [
      ./hardware-configuration.nix
      ./disko-config.nix
    ];

  boot.loader.grub.enable = true;
  boot.loader.grub.device = "/dev/sda1";

  services.openssh.enable = true;

  users.users.eugene = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
    initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123
  };

  programs.neovim = {
    enable = true;
    defaultEditor = true;
    configure = {
      customRC = ''
        colorscheme habamax
      '';

      packages.packages = {
        start = [
          pkgs.vimPlugins.nerdtree
        ];
      };
    };
  };

  system.stateVersion = "24.05";
}

Note: the value of initialHashedPassword was obtained using mkpasswd command in Linux, and corresponds to Password.123 string used as password.

# /tmp/my-first-flake/flake.nix

{
  inputs = {
    nixpkgs = {
      url = "github:NixOS/nixpkgs/nixos-24.05";
    };

    disko = {
      url = "github:nix-community/disko";
      inputs = {
        nixpkgs = {
          follows = "nixpkgs";
        };
      };
    };
  };

  outputs = inputs@{ self, nixpkgs, ... }: {
    nixosConfigurations = {
      my-hetzner-vm = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";

        modules = [
          ./my-vms/my-hetzner-vm/configuration.nix
          inputs.disko.nixosModules.disko
        ];
      };
    };
  };
}

Note: all these files constitute what's known as a nix flake. The flake in question is small, though not exactly a minimal one.

Build NixOS from flake. Run:

nix run --extra-experimental-features 'nix-command flakes' github:nix-community/nixos-anywhere -- --flake /tmp/my-first-flake#my-hetzner-vm root@0.0.0.0 --build-on-remote

Note: replace 0.0.0.0 with an IP address obtained during an earlier step.

The NixOS on Hetzner is installed!

Let's do a few more steps to customize the installation.

Copy flake files onto the server. Run:
```
scp -r /tmp/my-first-flake eugene@0.0.0.0:~/
```
Note: again, replace 0.0.0.0 with an IP address obtained via hcloud server ip my-hetzner-vm.

Using neovim editor on the VM, modify configuration.nix to include a package containing Elixir programming language runtime for eugene user. Run:

nvim my-first-flake/my-vms/my-hetzner-vm/configuration.nix

Edit the configuration.nix so that users block looks like this:

# ~/my-first-flake/my-vms/my-hetzner-vm/configuration.nix
# ...
users.users.eugene = {
  isNormalUser = true;
  extraGroups = [ "wheel" ];
  initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123

  packages = [
    pkgs.beam.packages.erlang_26.elixir_1_16
  ];
};
# ...

Re-build NixOS. Run:

sudo nixos-rebuild switch --flake ./my-first-flake#my-hetzner-vm

disko

TODO: it would be neat to document how to boot from the NixOS ISO and create the machine based on an online description including https://github.com/nix-community/disko specs - should be a quick way to set up bespoke 'throwaway' machines.

references:

nixos-infect

Beside the manual installation, one way to setup NixOS is to replace an existing installation, for example the latest Ubuntu image, with nixos-infect.

Cloud-init

You don't even need to log into the Ubuntu image, you can run nixos-infect from the 'could-init' as documented at https://github.com/elitak/nixos-infect/tree/master#hetzner-cloud

Manually

Boot into the existing operating system, preferably Ubuntu or Debian
Login as root or with root permissions
Deploy your SSH public key for the current root user. This key will be used later for authentication into the NixOS system.
Run following script. Replace NIX_CHANNEL variable with the version string you wish to install.

curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-22.11 bash -x

Reboot into NixOS

Declarative

Network configuration

Hetzner Cloud offers both IPv4 (/32 subnet) and IPv6 (/64 subnet) connectivity to each machine. The assigned addresses can be looked up on the Hetzner Cloud Console from the "Networking" tab on the instance details. The public IPv4 address of the server can automatically obtained be via DHCP. For IPv6 you have to statically configure both address and gateway.

  systemd.network.enable = true;
  systemd.network.networks."30-wan" = {
    matchConfig.Name = "ens3"; # either ens3 (amd64) or enp1s0 (arm64)
    networkConfig.DHCP = "ipv4";
    address = [
      # replace this subnet with the one assigned to your instance
      "2a01:4f8:aaaa:bbbb::1/64"
    ];
    routes = [ { routeConfig.Gateway = "fe80::1"; } ];
  };

Static IPv4 configuration

The IPv4 address can also be configured statically. The trick here is, that the gateway needs to be configured with the onlink flag, because it is not in the same subnet as your public IP address, but still very much on that same link.

  systemd.network.networks."30-wan" = {
    matchConfig.Name = "ens3"; # either ens3 (amd64) or enp1s0 (arm64)
    networkConfig.DHCP = "no";
    address = [
      # replace this address with the one assigned to your instance
      "A.B.C.D/32"
      # replace this subnet with the one assigned to your instance
      "2a01:4f8:AAAA:BBBB::1/64"
    ];
    routes = [ {
      routeConfig = { Gateway = "172.31.1.1"; GatewayOnLink = true; }; }
      { routeConfig.Gateway = "fe80::1"; }
    ];
  };

AArch64 (CAX instance type) specifics

If the screen goes blank after selecting the boot option in the bootloader, the following snippet makes sure that GPU drivers are available in initrd, and the correct device gets used for the serial console:

boot.initrd.kernelModules = [ "virtio_gpu" ];
boot.kernelParams = [ "console=tty" ];