Libvirt

libvirt is a toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). It does so by providing a common API to different virtualization backends.

Setup

Enable libvirt daemon

/etc/nixos/configuration.nix

services.libvirtd = {
  enable = true;
  qemu = {
    package = pkgs.qemu_kvm;
    runAsRoot = true;
    swtpm.enable = true;
    ovmf = {
      enable = true;
      packages = [(pkgs.unstable.OVMF.override {
        secureBoot = true;
        tpmSupport = true;
      }).fd];
    };
  };
};

To enable local user access to libvirt, for example by using virt-manager or gnome-boxes, add yourself to the libvirtd group

/etc/nixos/configuration.nix

users.users.myuser = {
  extraGroups = [ "libvirtd" ];
};

Configuration

Nested virtualization

If you would like to enable nested virtualization for your guests to run KVM hypervisors inside them, you should enable it as follows: boot.extraModprobeConfig, for example:

/etc/nixos/configuration.nix

boot.extraModprobeConfig = "options kvm_intel nested=1";

UEFI with OVMF

Enable UEFI firmware support, following Qemu configuration for your local user is required

~/.config/libvirt/qemu.conf

# Adapted from /var/lib/libvirt/qemu.conf
# Note that AAVMF and OVMF are for Aarch64 and x86 respectively
nvram = [ "/run/libvirt/nix-ovmf/AAVMF_CODE.fd:/run/libvirt/nix-ovmf/AAVMF_VARS.fd", "/run/libvirt/nix-ovmf/OVMF_CODE.fd:/run/libvirt/nix-ovmf/OVMF_VARS.fd" ]

See this tutorial on how to run a guest machine in UEFI mode using virt-manager.

Clients

NixOS provides some packages that can make use of libvirt or are useful with libvirt.

libguestfs

libguestfs is a set of tools for accessing and modifying virtual machines disk images.

Following are notes regarding the use of some of those tools

virt-builder

virt-builder is installed with libguestfs, but has some issues from its packaging.

It is possible to work around those issues without modifying the package (when a pristine nixpkgs is needed).

$ mkdir -p ~/.config/virt-builder
$ cd ~/.config/virt-builder
$ ln -s /run/current-system/sw/etc/xdg/virt-builder/repos.d
$ cd ~/.config
$ ln -s virt-builder/ .virt-builder-wrapped

This will make your user use the shipped repo configurations, and works around the fact that virt-builder reads its executable name to build its configuration path. The executable being wrapped, it is named differently.

error: cannot find any suitable libguestfs supermin

Use use the package libguestfs-with-appliance. See https://github.com/NixOS/nixpkgs/issues/37540

guestfs-tools

Includes virt-sysprep, used to prepare a VM image for use. Review the manpage of virt-sysprep, virt-clone, and virt-builder.

Accessing QEMU VMs through Webbrowser

I have a need that I can access some created VMs through a web browser. There's several SPICE html5 clients out there one from EyeOS works the best in my opinon.

In order to access the VM in a browser, we need to do several things.

Make VM SPICE accessible

In virt-manager (or whatever tool you use) you can add the Spice server as display. In virt-manager it's the Graphics new hardware. However - at least in virt-manager - you can't set everything as it needs to be. So after adding the Spice server through virt-manager, fire up your console and edit the xml file using virsh edit {vmname}.

Go to the graphics section and edit your you entry to something like this:

<graphics type='spice' port='5900' autoport='no' listen='0.0.0.0' keymap='de-ch' defaultMode='insecure'>
  <listen type='address' address='0.0.0.0'/>
  <image compression='auto_lz'/>
</graphics>

Add Websockify

Since libvirt doesn't support websockets on its own, we'll need to add websockify to your configuration.nix

services.networking.websockify = {
  enable = true;
  sslCert = "/https-cert.pem";
  sslKey = "/https-key.pem";
  portMap = {
    "5959" = 5900;
  };
};

The port mapping 5959 -> 5900 is the websocket forward from nginx 5959 to the spice server. If you used another port for the spice server, then adjust accordingly.

Also, I use letsencrypt dns mode to get https cert and key. Nginx i nixos can get the certs on its own. Since I use the same certs also for other things, I just put them in the root (/) folder. Use what is best for you.

Get EyeOS Spice Web Client

As said, the experience with the EyeOS Spice Web Client has been the best so far. Another client would be the spice-html5 from freedesktop.org.

1. Download the EyeOS Spice Web Client and unpack it (if necessary) or , as example, just git clone https://github.com/eyeos/spice-web-client/ /var/www/spice

2. Once downloaded (and unpacked), edit the run.js file and search for 'ws' (around line 213) and change it to 'wss'

Setup nginx for access

As last part, you'll need to setup nginx so serve files from the EyeOS Spice Web Client and use websockify to communicate with the VM.

services.nginx = {
  enable = true;
  virtualHosts."mydomain.tld" = {
    forceSSL = true;
    root = "/var/www/";
    locations."/spice/" = {
      index = "index.html index.htm";
    };
    locations."/websockify/" = {
      proxyWebsockets = true;
      proxyPass = "https://127.0.0.1:5959";
      extraConfig = ''
        proxy_read_timeout 61s;
        proxy_buffering off;
      '';
    };
    sslCertificate = "/https-cert.pem";
    sslCertificateKey = "/https-key.pem";
    listen = [ { addr = "*"; port = 45000; ssl = true; } ];
  };
};

So, in the above example we access the nginx installation on port 45000 (use whatever you want, you could also just use normal ports like 80/445). We tell it to use port 5959 for websockify which is mapped to port 5900. And we tell it to access the mydomain.tld/spice folder as /var/www/spice (where we did download the EyeOS Spice Web Client to).

Access the VM through the browser

In order to access the VM through the browser, you'll also need to open ports in your firewall (port for nginx, websockify and spice; 4500, 5959, 5900 in the example).

Then you'll need to start the vm, you can do it by sshing into the computer and run virsh start {vmname}.

And finally you can access the VMs GUI through https://mydomain.tld:4500/spice/index.html?host=mydomain.tld&port=5959