Install NixOS on Hetzner Cloud

From NixOS Wiki
Revision as of 08:42, 3 December 2024 by Mic92 (talk | contribs) (update example to 24.11)
Note: This article is about installing NixOS on Hetzner Cloud, which provides VPS cloud servers. It is not to be confused with Hetzner Online, which provides dedicated bare-metal servers.

Hetzner Cloud in 2023 offers amd64 instances on both Intel and AMD hardware, as well as arm64 instanced based on the Ampere Altra Q80, in three locations in Europe, as well as two in America.

The pricing of the arm64 instances is generally considered very competitive, compared to similarly specced amd64 instances.

Installation

There are several ways to install NixOS, such as the "traditional" ISO installation, nixos-infect or nixos-anywhere.

Traditional ISO installation

In the Hetzner console, mount the NixOS minimal ISO into your server, and use the console to install NixOS.

x86_64

At time of writing, Hetnzer's x86_64 servers use legacy boot.

First, create a new MBR partition table. 

 parted /dev/sda --script mklabel msdos

Then create a 512MB boot partition with ext4 

 parted /dev/sda --script mkpart primary ext4 1MiB 513MiB
 parted /dev/sda --script set 1 boot on
 mkfs.ext4 -L boot /dev/sda1

Create a swap partition. This example uses 8GB, you may want to research the correct amount for your server size. Note the end of the swap partition in this command is 8577MiB, this is the value used in the next command. 

 parted /dev/sda --script mkpart primary linux-swap 513MiB 8577MiB
 mkswap -L swap /dev/sda2
 swapon /dev/sda2

Create a root partition using the rest of the disk with ext4. 

 parted /dev/sda --script mkpart primary ext4 8577MiB 100%
 mkfs.ext4 -L nixos /dev/sda3

If you don't mount the partitions you've just created, the NixOS installer will produce an error in the form `Failed to get blkid info (returned 512) for on tmpfs at <path>/<prefix>-install-grub.pl`. 

 # Mount the partitions to /mnt and /mnt/boot.
 mount /dev/disk/by-label/nixos /mnt
 mkdir /mnt/boot
 mount /dev/disk/by-label/boot /mnt/boot

Finally, install. Install from a remote flake: 

 sudo nixos-install --flake github:<username>/<repo>#<id>

Once installed, unmount the ISO and reboot.

Hetnzer base configuration

In the example below, the id would be `hetzner-x86_64`.

The `flake.nix` file in the repo should be of the form: 

 {
   inputs = {
     nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11";
   };
 
   outputs = { nixpkgs, ... }: {
     nixosConfigurations = {
       hetzner-x86_64 = nixpkgs.lib.nixosSystem {
         system = "x86_64-linux";
         modules = [
           ./configuration.nix
         ];
       };
     };
   };
 }

With the referenced `configuration.nix` in the form.

Note the filesystems configuration, which matches the partition scheme, and the `availableKernelModules` section which includes modules that enable ext4 at boot.

Also note the user `username` which is configured to be part of the `wheel` group, and can therefore use `sudo`. 

 { pkgs, ... }:
 
 {
   nix.settings = {
     experimental-features = "nix-command flakes";
     auto-optimise-store = true;
   };
   
   environment.systemPackages = [
     pkgs.vim
     pkgs.git
     pkgs.zip
     pkgs.unzip
     pkgs.wget
   ];
   
   fileSystems."/" = {
     device = "/dev/disk/by-label/nixos";
     fsType = "ext4";
   };
   fileSystems."/boot" = {
     device = "/dev/disk/by-label/boot";
     fsType = "ext4";
   };
   swapDevices = [
     {
       device = "/dev/disk/by-label/swap";
     }
   ];
   
   documentation.nixos.enable = false;
   time.timeZone = "Europe/London";
   i18n.defaultLocale = "en_GB.UTF-8";
   console.keyMap = "us";
   nix.settings.trusted-users = [ "@wheel" ];
   
   boot.loader.grub.enable = true;
   boot.loader.grub.device = "/dev/sda";
   boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" "ext4" ];
   
   users.users = {
     root.hashedPassword = "!"; # Disable root login
     username = {
       isNormalUser = true;
       extraGroups = [ "wheel" ];
       openssh.authorizedKeys.keys = [
         ssh-rsa <your_ssh_public_key>
       ];
     };
   };
   
   security.sudo.wheelNeedsPassword = false;
   
   services.openssh = {
     enable = true;
     settings = {
       PermitRootLogin = "no";
       PasswordAuthentication = false;
       KbdInteractiveAuthentication = false;
     };
   };
   networking.firewall.allowedTCPPorts = [ 22 ];
   
   # This value determines the NixOS release from which the default
   # settings for stateful data, like file locations and database versions
   # on your system were taken. It‘s perfectly fine and recommended to leave
   # this value at the release version of the first install of this system.
   # Before changing this value read the documentation for this option
   # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
   system.stateVersion = "24.11"; # Did you read the comment?
 }


To access the system, you will need to ensure that port 22 on the VM is available via the Hetzner firewall.

nixos-anywhere

The tutorial assumes you already have an account on Hetzner Cloud, and no prior access to a system with NixOS or nix CLI utility installed:

  1. Create a temp folder for future use. Run:
    mkdir /tmp/my-first-flake
    Note: this folder will be mounted into docker container on the next step. Having the folder on the host system enables editing files using a familiar editor, available on the host system, such as VS Code or neovim.
  2. Enter docker container. Run:
    docker run --rm --interactive --tty --mount type=bind,source=/tmp/my-first-flake,target=/tmp/my-first-flake alpine:3.20 ash
    Note: this is done in a container in order to reduce the "setup footprint and residue", allowing to throw away this setup environment quickly.
  3. Install nix and hcloud CLI utilities. Run:
    apk add nix hcloud openssh
  4. Authenticate hcloud CLI utility. Run:
    hcloud context create my-first-context
  5. When asked, enter value of the token in the prompt. Note: the token with "Read/Write" permissions can be obtained on a project page inside Hetzner Cloud: https://console.hetzner.cloud/projects/0000000/security/tokens
  6. Create an SSH key:
    ssh-keygen -t ed25519 -f /root/.ssh/id_ed25519 -N ''
  7. Upload your public key to Hetzner:
    hcloud ssh-key create --name 'My public key' --public-key-from-file /root/.ssh/id_ed25519.pub
  8. Create a VM on Hetzner. Run:
    hcloud server create --name my-hetzner-vm --type cpx21 --image ubuntu-24.04 --location fsn1 --ssh-key 'My public key'
    Write down the IP. Note 1: this tutorial uses cpx21 VM instance type which corresponds to an x86 architecture marchine with 3 CPU cores and 4GB of RAM, and fsn1 location which corresponds to a data center in the city of Falkenstein in Germany. A list of all instance types can be obtained by running command hcloud server-type list, while a list of all locations can be obtained by running hcloud location list command. Note 2: Hopefully, Hetzner Cloud team will support NixOS disk images soon, see Could we convince Hetzner to add Nixos as a standard image choice.
  9. On your host computer, create a folder. Run:
    mkdir -p /tmp/my-first-flake/my-vms/my-hetzner-vm/
  10. Using a code editor on your host computer, create 4 files. File contents, as well as the location of where to put corresponding file are indicated below:
    # /tmp/my-first-flake/my-vms/my-hetzner-vm/hardware-configuration.nix

{ config, lib, pkgs, modulesPath, ... }:

{
  imports = [
    (modulesPath + "/profiles/qemu-guest.nix")
  ];

  boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ ];
  boot.extraModulePackages = [ ];
  swapDevices = [ ];
  networking.useDHCP = lib.mkDefault true;
  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}

    # /tmp/my-first-flake/my-vms/my-hetzner-vm/disko-config.nix

{
  disko.devices = {
    disk = {
      main = {
        type = "disk";
        device = "/dev/sda";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02";
              priority = 1;
            };
            ESP = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            };
            root = {
              size = "100%";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/";
              };
            };
          };
        };
      };
    };
  };
}

    # /tmp/my-first-flake/my-vms/my-hetzner-vm/configuration.nix

{ config, lib, pkgs, ... }:

{
  imports =
    [
      ./hardware-configuration.nix
      ./disko-config.nix
    ];

  boot.loader.grub.enable = true;

  services.openssh.enable = true;

  users.users.eugene = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
    initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123
  };

  programs.neovim = {
    enable = true;
    defaultEditor = true;
    configure = {
      customRC = ''
        colorscheme habamax
      '';

      packages.packages = {
        start = [
          pkgs.vimPlugins.nerdtree
        ];
      };
    };
  };

  system.stateVersion = "24.05";
}
    Note: the value of initialHashedPassword above was obtained using mkpasswd command in Linux, and corresponds to Password.123 string used as password.
    # /tmp/my-first-flake/flake.nix

{
  inputs = {
    nixpkgs = {
      url = "github:NixOS/nixpkgs/nixos-24.05";
    };

    disko = {
      url = "github:nix-community/disko";
      inputs = {
        nixpkgs = {
          follows = "nixpkgs";
        };
      };
    };
  };

  outputs = inputs@{ self, nixpkgs, ... }: {
    nixosConfigurations = {
      my-hetzner-vm = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";

        modules = [
          ./my-vms/my-hetzner-vm/configuration.nix
          inputs.disko.nixosModules.disko
        ];
      };
    };
  };
}
    Note: all these files constitute what's known as a nix flake. The flake in question is small, though not exactly a minimal one.
  11. Build NixOS from flake. Run:
    nix run --extra-experimental-features 'nix-command flakes' github:nix-community/nixos-anywhere/1.3.0 -- --flake /tmp/my-first-flake#my-hetzner-vm root@0.0.0.0 --build-on-remote
    Note: replace 0.0.0.0 with an IP address obtained during an earlier step.

The NixOS on Hetzner is installed!

Let's do a few more steps to customize the installation.

  1. Copy flake files onto the server. Run:
    scp -r /tmp/my-first-flake eugene@0.0.0.0:~/
    Note: again, replace 0.0.0.0 with an IP address obtained via hcloud server ip my-hetzner-vm.
  2. Using neovim editor on the VM, modify configuration.nix to include a package containing Elixir programming language runtime for eugene user. Run:
    nvim my-first-flake/my-vms/my-hetzner-vm/configuration.nix
    Edit the configuration.nix so that users block looks like this:
    # ~/my-first-flake/my-vms/my-hetzner-vm/configuration.nix
# ...
users.users.eugene = {
  isNormalUser = true;
  extraGroups = [ "wheel" ];
  initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123

  packages = [
    pkgs.beam.packages.erlang_26.elixir_1_16
  ];
};
# ...
  3. Re-build NixOS. Run:
    sudo nixos-rebuild switch --flake ./my-first-flake#my-hetzner-vm

disko

TODO: it would be neat to document how to boot from the NixOS ISO and create the machine based on an online description including https://github.com/nix-community/disko specs - should be a quick way to set up bespoke 'throwaway' machines.

references:

nixos-infect

Beside the manual installation, one way to setup NixOS is to replace an existing installation, for example the latest Ubuntu image, with nixos-infect.

Cloud-init

You don't even need to log into the Ubuntu image, you can run nixos-infect from the 'could-init' as documented at https://github.com/elitak/nixos-infect/tree/master#hetzner-cloud

Manually

  • Boot into the existing operating system, preferably Ubuntu or Debian
  • Login as root or with root permissions
  • Deploy your SSH public key for the current root user. This key will be used later for authentication into the NixOS system.
  • Run following script. Replace NIX_CHANNEL variable with the version string you wish to install.
curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-22.11 bash -x
  • Reboot into NixOS

Declarative

Network configuration

Hetzner Cloud offers both IPv4 (/32 subnet) and IPv6 (/64 subnet) connectivity to each machine. The assigned addresses can be looked up on the Hetzner Cloud Console from the "Networking" tab on the instance details. The public IPv4 address of the server can automatically obtained be via DHCP. For IPv6 you have to statically configure both address and gateway. 

  systemd.network.enable = true;
  systemd.network.networks."30-wan" = {
    matchConfig.Name = "ens3"; # either ens3 or enp1s0, check 'ip addr'
    networkConfig.DHCP = "ipv4";
    address = [
      # replace this subnet with the one assigned to your instance
      "2a01:4f8:aaaa:bbbb::1/64"
    ];
    routes = [
      { Gateway = "fe80::1"; }
    ];

  };

Static IPv4 configuration

The IPv4 address can also be configured statically. The trick here is, that the gateway needs to be configured with the onlink flag, because it is not in the same subnet as your public IP address, but still very much on that same link. 

  systemd.network.networks."30-wan" = {
    matchConfig.Name = "ens3"; # either ens3 (amd64) or enp1s0 (arm64)
    networkConfig.DHCP = "no";
    address = [
      # replace this address with the one assigned to your instance
      "A.B.C.D/32"
      # replace this subnet with the one assigned to your instance
      "2a01:4f8:AAAA:BBBB::1/64"
    ];
    routes = [ {
      routeConfig = { Gateway = "172.31.1.1"; GatewayOnLink = true; }; }
      { routeConfig.Gateway = "fe80::1"; }
    ];
  };

AArch64 (CAX instance type) specifics

If the screen goes blank after selecting the boot option in the bootloader, the following snippet makes sure that GPU drivers are available in initrd, and the correct device gets used for the serial console: 

boot.initrd.kernelModules = [ "virtio_gpu" ];
boot.kernelParams = [ "console=tty" ];
Retrieved from "https://wiki.nixos.org/w/index.php?title=Install_NixOS_on_Hetzner_Cloud&oldid=18997"