NixOS Wiki

Install NixOS on Scaleway X86 Virtual Cloud Server

Revision as of 18:55, 5 October 2018 by imported>Volth (→Scaleway Aarch64 peculiarities: fix script)

Create Scaleway Instance

If you know the Scaleway API you can do equivalent steps via API instead of using the webinterface.

  • Open Scaleway "Create a server" page
  • Enter name
  • Select region
  • Select server type
    • This will only work for the V and X types of servers (e.g. VC1M or X64-15GB)
  • Select "Debian Sid" from the distributions tab
Note: At the time of writing of this guide the Ubuntu Xenial did not have a kexec enabled kernel and will therefore not work!
  • Create server
  • Wait until server has been provisioned and is done booting
    • The status of the system can be checked on by looking at the console in the webinterface

Build NixOS kexec Tarball

remote build

First you will need to build a NixOS kexec tarball. To do this follow the steps outlined below: 

git clone https://github.com/cleverca22/nix-tests.git
cd nix-tests/kexec/
# Edit configuration.nix according to your needs. An example can be found below:
nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

copy the NixOS tarball

  • Copy the prepared NixOS kexec tarball to the server
scp result/tarball/nixos-system-x86_64-linux.tar.xz root@51.YY.XX.93:
  • Use ssh to access the server
    • Use the corrensponding SSH key you configured at Scaleway before
  • Extract the tarball into /
cd /; tar -xf /root/nixos-system-x86_64-linux.tar.xz

local (Scaleway instance) build

apt-get update
apt-get install -y git
curl https://nixos.org/nix/install | sh
source ~/.nix-profile/etc/profile.d/nix.sh
git clone https://github.com/cleverca22/nix-tests.git
cd nix-tests/kexec/
vim configuration.nix

remove autoreboot.nix, comment out boot.supportedFilesystems = [ "zfs" ]; (as in the sample config below) 

nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4
cp result/tarball/nixos-system-x86_64-linux.tar.xz /nixos.tar.xz
cd /
tar -xf nixos.tar.xz

Example configuration.nix

(removed autoreboot.nix from imports, added SSH key) 

# new cmd: nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./configuration.nix -Q -j 4

{ lib, pkgs, config, ... }:

with lib;

{
  imports = [ <nixpkgs/nixos/modules/installer/netboot/netboot-minimal.nix> ./kexec.nix ./justdoit.nix ];

  #boot.supportedFilesystems = [ "zfs" ];
  boot.loader.grub.enable = false;
  boot.kernelParams = [
    "console=ttyS0,115200"          # allows certain forms of remote access, if the hardware is setup right
    "panic=30" "boot.panic_on_fail" # reboot the machine upon fatal boot issues
  ];
  systemd.services.sshd.wantedBy = mkForce [ "multi-user.target" ];
  networking.hostName = "kexec";
  # example way to embed an ssh pubkey into the tar
  users.users.root.openssh.authorizedKeys.keys = [ "ssh-rsa AAAAB3.... Your-SSH-key" ];
}

Start NixOS kexec System

  • Execute the kexec_nixos script
./kexec_nixos
  • The output should look something like this:
root@scalenix:/# ./kexec_nixos 
++ mktemp -d
+ cd /tmp/tmp.iDXuzu8Ec4
+ pwd
/tmp/tmp.iDXuzu8Ec4
+ mkdir initrd
+ pushd initrd
/tmp/tmp.iDXuzu8Ec4/initrd /tmp/tmp.iDXuzu8Ec4
+ cat /ssh_pubkey
cat: /ssh_pubkey: No such file or directory
+ find -type f
+ cpio -o -H newc
+ gzip -9
1 block
+ popd
/tmp/tmp.iDXuzu8Ec4
+ cat /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/initrd extra.gz
+ kexec -l /nix/store/2lmw78k2ralvpn6fa270b53nz1xgqk8b-image/kernel --initrd=final.gz '--append=init=/nix/store/sv9hndbkrdxr1psi2jr82hkm1ba0j8bx-nixos-system-kexec-17.09.git.f3841ab/init loglevel=4 console=ttyS0,115200 panic=30 boot.panic_on_fail'
+ sync
+ echo 'executing kernel, filesystems will be improperly umounted'
executing kernel, filesystems will be improperly umounted
+ kexec -e
packet_write_wait: Connection to 51.XX.XX.93 port 22: Broken pipe
  • Once the ssh connection is broken take a look at the servers console in the webinterface. After a while you should see a NixOS root session:

 

Installing NixOS

Scaleway peculiarities

Scaleway starts up VMs differently than other providers. This results in the following things that need to be done so that NixOS can boot:

  • Scaleway start scripts mount /dev/vda -> No partitioning of the root volume possible
  • You have to configure 3 "tags" in the Scaleway configuration of the system to tell the start script which kernel and initrd to load
    • KEXEC_KERNEL=/boot/nixos-kernel
    • KEXEC_INITRD=/boot/nixos-initrd
    • KEXEC_APPEND=init=/boot/nixos-init
      File:Scaleway-tags.png
      Screenshot of Scaleway webinterface tag setting showing 2 of the 3 tags
  • You need to configure NixOS with the following options
boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;

Scaleway Aarch64 peculiarities

If your server is aarch64, there is no working kexec on initrd image provided by Scaleway. So KEXEC_KERNEL tag won't work. This can be worked around by compiling static kexec and copying in to /boot and also by providing fake /sbin/init (a file which Scaleway's initrd launches at the end) 

system.build.installBootLoader = pkgs.writeScript "installBootLoader.sh" ''
  #!/bin/sh

  export TOPLEVEL="$1"
  export PATH=${pkgs.coreutils}/bin

  cp -f $TOPLEVEL/kernel /boot/kernel
  cp -f $TOPLEVEL/initrd /boot/initrd
  cp -f ${pkgs.pkgsMusl.kexectools.overrideDerivation(old: {configureFlags = old.configureFlags ++ ["LDFLAGS=-static"]; }) }/bin/kexec /boot/kexec
  cp -f ${pkgs.pkgsMusl.busybox.override {enableStatic = true;}}/bin/busybox /boot/sh

  mkdir -p /sbin
  cat > /sbin/init <<EOF
  #!/boot/sh
  /boot/kexec -d -l /boot/kernel --initrd=/boot/initrd --append="init=$TOPLEVEL/init systemConfig=$TOPLEVEL $(cat $TOPLEVEL/kernel-params)"
  /boot/kexec -d -e
  EOF
  chmod 0777 /sbin/init
'';
boot.loader.grub.enable = false;

Step by Step Guide

  • Mount /dev/vda 
    mount /dev/vda /mnt/
  • Delete all existing data from the previous Debian system 
    rm -rf /mnt/*
  • Create mountpoint for /nix on the 2nd volume (I want to use it for the nix store)
    mkdir /mnt/nix
  • Create filesystem on the 2nd volume 
    mkfs.ext4 /dev/vdb -L nixstore
  • Mount 2nd volume 
    mount /dev/vdb /mnt/nix/
  • Generate NixOS config 
    nixos-generate-config --root /mnt
  • Edit /mnt/etc/nixos/configuration.nix
    • Remove the default boot.loader.grub entries
    • Add the following
boot.loader.initScript.enable = true;
boot.loader.generationsDir.enable = true;
boot.loader.generationsDir.copyKernels = true;
boot.loader.grub.enable = false;
    • Make sure to add users/keys to access the system later via SSH
  • Install NixOS nixos-install --root /mnt
    • This might take some time
    • Set root password at the end of
  • reboot
Retrieved from "https://wiki.nixos.org/w/index.php?title=Install_NixOS_on_Scaleway_X86_Virtual_Cloud_Server&oldid=4553"