Install NixOS on Hetzner Cloud: Difference between revisions
| Line 427: | Line 427: | ||
[[Category:Cookbook]] | [[Category:Cookbook]] | ||
[[Category:Deployment]] | |||
Latest revision as of 09:13, 26 October 2024
Hetzner Cloud in 2023 offers amd64 instances on both Intel and AMD hardware, as well as arm64 instanced based on the Ampere Altra Q80, in three locations in Europe, as well as two in America.
The pricing of the arm64 instances is generally considered very competitive, compared to similarly specced amd64 instances.
Installation
There are several ways to install NixOS, such as the "traditional" ISO installation, nixos-infect or nixos-anywhere.
Traditional ISO installation
In the Hetzner console, mount the NixOS minimal ISO into your server, and use the console to install NixOS.
x86_64
At time of writing, Hetnzer's x86_64 servers use legacy boot.
First, create a new MBR partition table.
parted /dev/sda --script mklabel msdos
Then create a 512MB boot partition with ext4
parted /dev/sda --script mkpart primary ext4 1MiB 513MiB parted /dev/sda --script set 1 boot on mkfs.ext4 -L boot /dev/sda1
Create a swap partition. This example uses 8GB, you may want to research the correct amount for your server size. Note the end of the swap partition in this command is 8577MiB, this is the value used in the next command.
parted /dev/sda --script mkpart primary linux-swap 513MiB 8577MiB mkswap -L swap /dev/sda2 swapon /dev/sda2
Create a root partition using the rest of the disk with ext4.
parted /dev/sda --script mkpart primary ext4 8577MiB 100% mkfs.ext4 -L nixos /dev/sda3
If you don't mount the partitions you've just created, the NixOS installer will produce an error in the form `Failed to get blkid info (returned 512) for on tmpfs at <path>/<prefix>-install-grub.pl`.
# Mount the partitions to /mnt and /mnt/boot. mount /dev/disk/by-label/nixos /mnt mkdir /mnt/boot mount /dev/disk/by-label/boot /mnt/boot
Finally, install. Install from a remote flake:
sudo nixos-install --flake github:<username>/<repo>#<id>
Once installed, unmount the ISO and reboot.
Hetnzer base configuration
In the example below, the id would be `hetzner-x86_64`.
The `flake.nix` file in the repo should be of the form:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
};
outputs = { nixpkgs, ... }: {
nixosConfigurations = {
hetzner-x86_64 = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
./configuration.nix
];
};
};
};
}
With the referenced `configuration.nix` in the form.
Note the filesystems configuration, which matches the partition scheme, and the `availableKernelModules` section which includes modules that enable ext4 at boot.
Also note the user `username` which is configured to be part of the `wheel` group, and can therefore use `sudo`.
{ pkgs, ... }:
{
nix.settings = {
experimental-features = "nix-command flakes";
auto-optimise-store = true;
};
environment.systemPackages = [
pkgs.vim
pkgs.git
pkgs.zip
pkgs.unzip
pkgs.wget
];
fileSystems."/" = {
device = "/dev/disk/by-label/nixos";
fsType = "ext4";
};
fileSystems."/boot" = {
device = "/dev/disk/by-label/boot";
fsType = "ext4";
};
swapDevices = [
{
device = "/dev/disk/by-label/swap";
}
];
documentation.nixos.enable = false;
time.timeZone = "Europe/London";
i18n.defaultLocale = "en_GB.UTF-8";
console.keyMap = "us";
nix.settings.trusted-users = [ "@wheel" ];
boot.loader.grub.enable = true;
boot.loader.grub.device = "/dev/sda";
boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" "ext4" ];
users.users = {
root.hashedPassword = "!"; # Disable root login
username = {
isNormalUser = true;
extraGroups = [ "wheel" ];
openssh.authorizedKeys.keys = [
ssh-rsa <your_ssh_public_key>
];
};
};
security.sudo.wheelNeedsPassword = false;
services.openssh = {
enable = true;
settings = {
PermitRootLogin = "no";
PasswordAuthentication = false;
KbdInteractiveAuthentication = false;
};
};
networking.firewall.allowedTCPPorts = [ 22 ];
# This value determines the NixOS release from which the default
# settings for stateful data, like file locations and database versions
# on your system were taken. It‘s perfectly fine and recommended to leave
# this value at the release version of the first install of this system.
# Before changing this value read the documentation for this option
# (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
system.stateVersion = "24.05"; # Did you read the comment?
}
To access the system, you will need to ensure that port 22 on the VM is available via the Hetzner firewall.
nixos-anywhere
The tutorial assumes you already have an account on Hetzner Cloud, and no prior access to a system with NixOS or nix CLI utility installed:
- Create a temp folder for future use. Run:Note: this folder will be mounted into docker container on the next step. Having the folder on the host system enables editing files using a familiar editor, available on the host system, such as VS Code or neovim.
mkdir /tmp/my-first-flake - Enter docker container. Run:Note: this is done in a container in order to reduce the "setup footprint and residue", allowing to throw away this setup environment quickly.
docker run --rm --interactive --tty --mount type=bind,source=/tmp/my-first-flake,target=/tmp/my-first-flake alpine:3.20 ash
- Install
nixandhcloudCLI utilities. Run:apk add nix hcloud openssh
- Authenticate
hcloudCLI utility. Run:hcloud context create my-first-context
- When asked, enter value of the token in the prompt. Note: the token with "Read/Write" permissions can be obtained on a project page inside Hetzner Cloud: https://console.hetzner.cloud/projects/0000000/security/tokens
- Create an SSH key:
ssh-keygen -t ed25519 -f /root/.ssh/id_ed25519 -N ''
- Upload your public key to Hetzner:
hcloud ssh-key create --name 'My public key' --public-key-from-file /root/.ssh/id_ed25519.pub
- Create a VM on Hetzner. Run:Write down the IP. Note 1: this tutorial uses
hcloud server create --name my-hetzner-vm --type cpx21 --image ubuntu-24.04 --location fsn1 --ssh-key 'My public key'
cpx21VM instance type which corresponds to an x86 architecture marchine with 3 CPU cores and 4GB of RAM, andfsn1location which corresponds to a data center in the city of Falkenstein in Germany. A list of all instance types can be obtained by running commandhcloud server-type list, while a list of all locations can be obtained by runninghcloud location listcommand. Note 2: Hopefully, Hetzner Cloud team will support NixOS disk images soon, see Could we convince Hetzner to add Nixos as a standard image choice. - On your host computer, create a folder. Run:
mkdir -p /tmp/my-first-flake/my-vms/my-hetzner-vm/
- Using a code editor on your host computer, create 4 files. File contents, as well as the location of where to put corresponding file are indicated below:
# /tmp/my-first-flake/my-vms/my-hetzner-vm/hardware-configuration.nix { config, lib, pkgs, modulesPath, ... }: { imports = [ (modulesPath + "/profiles/qemu-guest.nix") ]; boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ]; boot.initrd.kernelModules = [ ]; boot.kernelModules = [ ]; boot.extraModulePackages = [ ]; swapDevices = [ ]; networking.useDHCP = lib.mkDefault true; nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux"; }
# /tmp/my-first-flake/my-vms/my-hetzner-vm/disko-config.nix { disko.devices = { disk = { main = { type = "disk"; device = "/dev/sda"; content = { type = "gpt"; partitions = { boot = { size = "1M"; type = "EF02"; priority = 1; }; ESP = { size = "512M"; type = "EF00"; content = { type = "filesystem"; format = "vfat"; mountpoint = "/boot"; }; }; root = { size = "100%"; content = { type = "filesystem"; format = "ext4"; mountpoint = "/"; }; }; }; }; }; }; }; }
Note: the value of# /tmp/my-first-flake/my-vms/my-hetzner-vm/configuration.nix { config, lib, pkgs, ... }: { imports = [ ./hardware-configuration.nix ./disko-config.nix ]; boot.loader.grub.enable = true; services.openssh.enable = true; users.users.eugene = { isNormalUser = true; extraGroups = [ "wheel" ]; initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123 }; programs.neovim = { enable = true; defaultEditor = true; configure = { customRC = '' colorscheme habamax ''; packages.packages = { start = [ pkgs.vimPlugins.nerdtree ]; }; }; }; system.stateVersion = "24.05"; }
initialHashedPasswordabove was obtained usingmkpasswdcommand in Linux, and corresponds toPassword.123string used as password.Note: all these files constitute what's known as a nix flake. The flake in question is small, though not exactly a minimal one.# /tmp/my-first-flake/flake.nix { inputs = { nixpkgs = { url = "github:NixOS/nixpkgs/nixos-24.05"; }; disko = { url = "github:nix-community/disko"; inputs = { nixpkgs = { follows = "nixpkgs"; }; }; }; }; outputs = inputs@{ self, nixpkgs, ... }: { nixosConfigurations = { my-hetzner-vm = nixpkgs.lib.nixosSystem { system = "x86_64-linux"; modules = [ ./my-vms/my-hetzner-vm/configuration.nix inputs.disko.nixosModules.disko ]; }; }; }; }
- Build NixOS from flake. Run:Note: replace
nix run --extra-experimental-features 'nix-command flakes' github:nix-community/nixos-anywhere/1.3.0 -- --flake /tmp/my-first-flake#my-hetzner-vm root@0.0.0.0 --build-on-remote
0.0.0.0with an IP address obtained during an earlier step.
The NixOS on Hetzner is installed!
Let's do a few more steps to customize the installation.
- Copy flake files onto the server. Run:Note: again, replace
scp -r /tmp/my-first-flake eugene@0.0.0.0:~/
0.0.0.0with an IP address obtained viahcloud server ip my-hetzner-vm. - Using
neovimeditor on the VM, modifyconfiguration.nixto include a package containing Elixir programming language runtime foreugeneuser. Run:Edit thenvim my-first-flake/my-vms/my-hetzner-vm/configuration.nixconfiguration.nixso thatusersblock looks like this:# ~/my-first-flake/my-vms/my-hetzner-vm/configuration.nix # ... users.users.eugene = { isNormalUser = true; extraGroups = [ "wheel" ]; initialHashedPassword = "$y$j9T$2DyEjQxPoIjTkt8zCoWl.0$3mHxH.fqkCgu53xa0vannyu4Cue3Q7xL4CrUhMxREKC"; # Password.123 packages = [ pkgs.beam.packages.erlang_26.elixir_1_16 ]; }; # ...
- Re-build NixOS. Run:
sudo nixos-rebuild switch --flake ./my-first-flake#my-hetzner-vm
disko
TODO: it would be neat to document how to boot from the NixOS ISO and create the machine based on an online description including https://github.com/nix-community/disko specs - should be a quick way to set up bespoke 'throwaway' machines.
references:
nixos-infect
Beside the manual installation, one way to setup NixOS is to replace an existing installation, for example the latest Ubuntu image, with nixos-infect.
Cloud-init
You don't even need to log into the Ubuntu image, you can run nixos-infect from the 'could-init' as documented at https://github.com/elitak/nixos-infect/tree/master#hetzner-cloud
Manually
- Boot into the existing operating system, preferably Ubuntu or Debian
- Login as root or with root permissions
- Deploy your SSH public key for the current root user. This key will be used later for authentication into the NixOS system.
- Run following script. Replace
NIX_CHANNELvariable with the version string you wish to install.
curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-22.11 bash -x
- Reboot into NixOS
Declarative
Network configuration
Hetzner Cloud offers both IPv4 (/32 subnet) and IPv6 (/64 subnet) connectivity to each machine. The assigned addresses can be looked up on the Hetzner Cloud Console from the "Networking" tab on the instance details. The public IPv4 address of the server can automatically obtained be via DHCP. For IPv6 you have to statically configure both address and gateway.
systemd.network.enable = true;
systemd.network.networks."30-wan" = {
matchConfig.Name = "ens3"; # either ens3 or enp1s0, check 'ip addr'
networkConfig.DHCP = "ipv4";
address = [
# replace this subnet with the one assigned to your instance
"2a01:4f8:aaaa:bbbb::1/64"
];
routes = [
{ Gateway = "fe80::1"; }
];
};
Static IPv4 configuration
The IPv4 address can also be configured statically. The trick here is, that the gateway needs to be configured with the onlink flag, because it is not in the same subnet as your public IP address, but still very much on that same link.
systemd.network.networks."30-wan" = {
matchConfig.Name = "ens3"; # either ens3 (amd64) or enp1s0 (arm64)
networkConfig.DHCP = "no";
address = [
# replace this address with the one assigned to your instance
"A.B.C.D/32"
# replace this subnet with the one assigned to your instance
"2a01:4f8:AAAA:BBBB::1/64"
];
routes = [ {
routeConfig = { Gateway = "172.31.1.1"; GatewayOnLink = true; }; }
{ routeConfig.Gateway = "fe80::1"; }
];
};
AArch64 (CAX instance type) specifics
If the screen goes blank after selecting the boot option in the bootloader, the following snippet makes sure that GPU drivers are available in initrd, and the correct device gets used for the serial console:
boot.initrd.kernelModules = [ "virtio_gpu" ];
boot.kernelParams = [ "console=tty" ];