Install NixOS on Scaleway X86 Virtual Cloud Server

From NixOS Wiki
Revision as of 08:32, 19 April 2018 by imported>Excogitation (added instructions for local build)

Preparation

Build NixOS kexec Tarball

remote build

First you will need to build a NixOS kexec tarball. To do this follow the steps outlined below:

git clone https://github.com/cleverca22/nix-tests.git
cd nix-tests/kexec/
# Edit configuration.nix according to your needs. An example can be found below:
nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

local (Scaleway instance) build

# apt-get update
# apt-get install -y git
# curl https://nixos.org/nix/install | sh
# source ~/.nix-profile/etc/profile.d/nix.sh
# git clone https://github.com/cleverca22/nix-tests.git
# cd nix-tests/kexec/
# vim configuration.nix

remove autoreboot.nix, comment out boot.supportedFilesystems = [ "zfs" ]; (as in the sample config below)

# nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4
# cp result/tarball/nixos-system-x86_64-linux.tar.xz /nixos.tar.xz
# cd /
# tar -xf nixos.tar.xz
# ./kexec_nixos

continue in browser shell.

Example configuration.nix

(removed autoreboot.nix from imports, added SSH key)

# new cmd: nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

{ lib, pkgs, config, ... }:

with lib;

{
  imports = [ <nixpkgs/nixos/modules/installer/netboot/netboot-minimal.nix> ./kexec.nix ./justdoit.nix ];

  #boot.supportedFilesystems = [ "zfs" ];
  boot.loader.grub.enable = false;
  boot.kernelParams = [
    "console=ttyS0,115200"          # allows certain forms of remote access, if the hardware is setup right
    "panic=30" "boot.panic_on_fail" # reboot the machine upon fatal boot issues
  ];
  systemd.services.sshd.wantedBy = mkForce [ "multi-user.target" ];
  networking.hostName = "kexec";
  # example way to embed an ssh pubkey into the tar
  users.users.root.openssh.authorizedKeys.keys = [ "ssh-rsa AAAAB3.... Your-SSH-key" ];
}

Create Scaleway Instance

If you know the Scaleway API you can do equivalent steps via API instead of using the webinterface.

  • Open Scaleway "Create a server" page
  • Enter name
  • Select region
  • Select server type
    • This will only work for the V and X types of servers (e.g. VC1M or X64-15GB)
  • Select "Debian Sid" from the distributions tab
Note: At the time of writing of this guide the Ubuntu Xenial did not have a kexec enabled kernel and will therefore not work!
  • Create server
  • Wait until server has been provisioned and is done booting
    • The status of the system can be checked on by looking at the console in the webinterface

Start NixOS kexec System

  • Copy the prepared NixOS kexec tarball to the server
    scp result/tarball/nixos-system-x86_64-linux.tar.xz root@51.YY.XX.93:
    
  • Use ssh to access the server
    • Use the corrensponding SSH key you configured at Scaleway before
  • Extract the tarball into /
    cd /; tar -xf /root/nixos-system-x86_64-linux.tar.xz
    
  • Execute the kexec_nixos script
    ./kexec_nixos
    
  • The output should look something like this:
root@scalenix:/# ./kexec_nixos 
++ mktemp -d
+ cd /tmp/tmp.iDXuzu8Ec4
+ pwd
/tmp/tmp.iDXuzu8Ec4
+ mkdir initrd
+ pushd initrd
/tmp/tmp.iDXuzu8Ec4/initrd /tmp/tmp.iDXuzu8Ec4
+ cat /ssh_pubkey
cat: /ssh_pubkey: No such file or directory
+ find -type f
+ cpio -o -H newc
+ gzip -9
1 block
+ popd
/tmp/tmp.iDXuzu8Ec4
+ cat /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/initrd extra.gz
+ kexec -l /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/kernel --initrd=final.gz '--append=init=/nix/store/sv9hndbkrdxr1psi2jr82hkm1ba0j8bx-nixos-system-kexec-17.09.git.f3841ab/init loglevel=4 console=ttyS0,115200 panic=30 boot.panic_on_fail'
+ sync
+ echo 'executing kernel, filesystems will be improperly umounted'
executing kernel, filesystems will be improperly umounted
+ kexec -e
packet_write_wait: Connection to 51.XX.XX.93 port 22: Broken pipe
  • Once the ssh connection is broken take a look at the servers console in the webinterface. After a while you should see a NixOS root session:

Screenshot of console showing root session

Installing NixOS

Scaleway peculiarities

Scaleway starts up VMs differently than other providers. This results in the following things that need to be done so that NixOS can boot:

  • Scaleway start scripts mount /dev/vda -> No partitioning of the root volume possible
  • You have to configure 3 "tags" in the Scaleway configuration of the system to tell the start script which kernel and initrd to load
    • KEXEC_KERNEL=/boot/nixos-kernel
    • KEXEC_INITRD=/boot/nixos-initrd
    • KEXEC_APPEND=init=/boot/nixos-init
      File:Scaleway-tags.png
      Screenshot of Scaleway webinterface tag setting showing 2 of the 3 tags
  • You need to configure NixOS with the following options
boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;

Step by Step Guide

  • Mount /dev/vda
    mount /dev/vda /mnt/
    
  • Delete all existing data from the previous Debian system
    rm -rf /mnt/*
    
  • Create mountpoint for /nix on the 2nd volume (I want to use it for the nix store)
    mkdir /mnt/nix
    
  • Create filesystem on the 2nd volume
    mkfs.ext4 /dev/vdb -L nixstore
    
  • Mount 2nd volume
    mount /dev/vdb /mnt/nix/
    
  • Generate NixOS config
    nixos-generate-config --root /mnt
    
  • Edit /mnt/etc/nixos/configuration.nix
    • Remove the default boot.loader.grub entries
    • Add the following
boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;
    • Make sure to add users/keys to access the system later via SSH
  • Install NixOS nixos-install --root /mnt
    • This might take some time
    • Set root password at the end of
  • reboot