Install NixOS on Scaleway X86 Virtual Cloud Server

Preparation

Build NixOS kexec Tarball

remote build

First you will need to build a NixOS kexec tarball. To do this follow the steps outlined below:

git clone https://github.com/cleverca22/nix-tests.git
cd nix-tests/kexec/
# Edit configuration.nix according to your needs. An example can be found below:
nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

local (Scaleway instance) build

# apt-get update
# apt-get install -y git
# curl https://nixos.org/nix/install | sh
# source ~/.nix-profile/etc/profile.d/nix.sh
# git clone https://github.com/cleverca22/nix-tests.git
# cd nix-tests/kexec/
# vim configuration.nix

remove autoreboot.nix, comment out boot.supportedFilesystems = [ "zfs" ]; (as in the sample config below)

# nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4
# cp result/tarball/nixos-system-x86_64-linux.tar.xz /nixos.tar.xz
# cd /
# tar -xf nixos.tar.xz
# ./kexec_nixos

continue in browser shell.

Example configuration.nix

(removed autoreboot.nix from imports, added SSH key)

# new cmd: nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

{ lib, pkgs, config, ... }:

with lib;

{
  imports = [ <nixpkgs/nixos/modules/installer/netboot/netboot-minimal.nix> ./kexec.nix ./justdoit.nix ];

  #boot.supportedFilesystems = [ "zfs" ];
  boot.loader.grub.enable = false;
  boot.kernelParams = [
    "console=ttyS0,115200"          # allows certain forms of remote access, if the hardware is setup right
    "panic=30" "boot.panic_on_fail" # reboot the machine upon fatal boot issues
  ];
  systemd.services.sshd.wantedBy = mkForce [ "multi-user.target" ];
  networking.hostName = "kexec";
  # example way to embed an ssh pubkey into the tar
  users.users.root.openssh.authorizedKeys.keys = [ "ssh-rsa AAAAB3.... Your-SSH-key" ];
}

Create Scaleway Instance

If you know the Scaleway API you can do equivalent steps via API instead of using the webinterface.

Open Scaleway "Create a server" page
Enter name
Select region
Select server type
- This will only work for the V and X types of servers (e.g. VC1M or X64-15GB)
Select "Debian Sid" from the distributions tab

Note: At the time of writing of this guide the Ubuntu Xenial did not have a kexec enabled kernel and will therefore not work!

Create server
Wait until server has been provisioned and is done booting
- The status of the system can be checked on by looking at the console in the webinterface

Start NixOS kexec System

Copy the prepared NixOS kexec tarball to the server

scp result/tarball/nixos-system-x86_64-linux.tar.xz root@51.YY.XX.93:

Use ssh to access the server
- Use the corrensponding SSH key you configured at Scaleway before

Extract the tarball into /

cd /; tar -xf /root/nixos-system-x86_64-linux.tar.xz

Execute the kexec_nixos script
```
./kexec_nixos
```
The output should look something like this:

root@scalenix:/# ./kexec_nixos 
++ mktemp -d
+ cd /tmp/tmp.iDXuzu8Ec4
+ pwd
/tmp/tmp.iDXuzu8Ec4
+ mkdir initrd
+ pushd initrd
/tmp/tmp.iDXuzu8Ec4/initrd /tmp/tmp.iDXuzu8Ec4
+ cat /ssh_pubkey
cat: /ssh_pubkey: No such file or directory
+ find -type f
+ cpio -o -H newc
+ gzip -9
1 block
+ popd
/tmp/tmp.iDXuzu8Ec4
+ cat /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/initrd extra.gz
+ kexec -l /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/kernel --initrd=final.gz '--append=init=/nix/store/sv9hndbkrdxr1psi2jr82hkm1ba0j8bx-nixos-system-kexec-17.09.git.f3841ab/init loglevel=4 console=ttyS0,115200 panic=30 boot.panic_on_fail'
+ sync
+ echo 'executing kernel, filesystems will be improperly umounted'
executing kernel, filesystems will be improperly umounted
+ kexec -e
packet_write_wait: Connection to 51.XX.XX.93 port 22: Broken pipe

Once the ssh connection is broken take a look at the servers console in the webinterface. After a while you should see a NixOS root session:

Installing NixOS

Scaleway peculiarities

Scaleway starts up VMs differently than other providers. This results in the following things that need to be done so that NixOS can boot:

Scaleway start scripts mount /dev/vda -> No partitioning of the root volume possible
You have to configure 3 "tags" in the Scaleway configuration of the system to tell the start script which kernel and initrd to load
- KEXEC_KERNEL=/boot/nixos-kernel
- KEXEC_INITRD=/boot/nixos-initrd
- KEXEC_APPEND=init=/boot/nixos-init
  File:Scaleway-tags.png
  Screenshot of Scaleway webinterface tag setting showing 2 of the 3 tags
You need to configure NixOS with the following options

boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;

Step by Step Guide

Mount /dev/vda
```
mount /dev/vda /mnt/
```
Delete all existing data from the previous Debian system
```
rm -rf /mnt/*
```
Create mountpoint for /nix on the 2nd volume (I want to use it for the nix store)
```
mkdir /mnt/nix
```
Create filesystem on the 2nd volume
```
mkfs.ext4 /dev/vdb -L nixstore
```
Mount 2nd volume
```
mount /dev/vdb /mnt/nix/
```
Generate NixOS config
```
nixos-generate-config --root /mnt
```
Edit /mnt/etc/nixos/configuration.nix
- Remove the default boot.loader.grub entries
- Add the following

boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;

- Make sure to add users/keys to access the system later via SSH
Install NixOS nixos-install --root /mnt
- This might take some time
- Set root password at the end of
reboot