IGVT-g

From NixOS Wiki
(Redirected from KVMGT)

Intel GVT-g is a technology that allows to "slice" an Intel GPU into virtualized GPUs that can be then passed into virtual machines.

Note that Intel GVT-g only supports Intel Broadwell (5th gen) to Comet Lake (10th gen), where in 10th gen support for IVGT-g was removed. For 11th gen, and 12th gen, there is SR-IOV coming up for virtualized GPUs, but that did not yet arrive in Linux mainline.

Configuring GPU

First, enable iGVT-g with: 

/etc/nixos/configuration.nix

virtualisation.kvmgt.enable = true;

... then rebuild and reboot.


After rebooting, check if the driver was successfully loaded: 

 $ ls /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/

... if you get No such file or directory, it means you might be using an older CPU that needs an extra configuration option: 

/etc/nixos/configuration.nix

boot.extraModprobeConfig = "options i915 enable_guc=2";

... then rebuild and reboot once again (unless that previous ls succeeded, in which case you don't have to modify that extraModprobeConfig at all).


Now, using ls you can list which variants your Intel GPU is able to virtualise: 

 $ ls /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/
 i915-GVTg_V5_4/  i915-GVTg_V5_8/

E.g.: 

 $ cat /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_8/description 
 low_gm_size: 64MB
 high_gm_size: 384MB
 fence: 4
 resolution: 1024x768
 weight: 2

Find some variant that matches your expectations (resolution, memory size etc.), note down its name, generate a random UUID: 

 # If you're using Nix Flakes:
 $ nix shell nixpkgs#libossp_uuid -c uuid
 a297db4a-f4c2-11e6-90f6-d3b88d6c9525

 # If you're not using Nix Flakes:
 $ nix run nixpkgs.libossp_uuid -c uuid
 a297db4a-f4c2-11e6-90f6-d3b88d6c9525

... and add that variant's name into your configuration: 

/etc/nixos/configuration.nix

virtualisation.kvmgt.enable = true; 
virtualisation.kvmgt.vgpus = {
  "i915-GVTg_V5_8" = {
    uuid = [ "a297db4a-f4c2-11e6-90f6-d3b88d6c9525" ];
  };
};

environment.systemPackages = with pkgs; [
  virtmanager
];

virtualisation.libvirtd.enable = true;
users.extraUsers.user.extraGroups = [ "libvirtd" ];

Rebuild & voilá - your virtualized GPU is ready; now you just have to attach it to some virtual machine.

Attaching GPU

Bare Qemu

 qemu-system-x86_64 \
    -enable-kvm \
    -m 1G \
    -nodefaults \
    -display gtk,gl=on \
    -device vfio-pci,sysfsdev=/sys/bus/mdev/devices/a297db4a-f4c2-11e6-90f6-d3b88d6c9525,display=on,x-igd-opregion=on,driver=vfio-pci-nohotplug,ramfb=on,xres=1920,yres=1080

libvirt

If using virt-manager, create new or open existing VM. Change existing <graphics> and <video> sections.

sudo -E virsh edit win10 

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <devices>
    <graphics type='spice'>
      <listen type='none'/>
      <gl enable='yes'/>
    </graphics>
    <video>
      <model type='none'/>
    </video>
    <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='on'>
      <source>
        <address uuid='a297db4a-f4c2-11e6-90f6-d3b88d6c9525'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-igd-opregion=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.ramfb=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.driver=vfio-pci-nohotplug'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.xres=1920'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.yres=1080'/>
    <qemu:env name="MESA_LOADER_DRIVER_OVERRIDE" value="i965"/>
  </qemu:commandline>
</domain>

FAQ

  • No video output
    • ensure that the recent Intel graphics driver is installed in the guest
  • (libvirtd) "Element domain has extra content: qemu:commandline" error after editing via virsh
    • you forgot to add xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'
  • (libvirtd) "no drm render node available" error in virt-manager
    • in virt-manager change SPICE display render node from auto to available one
  • "write_loop: No space left on device" error when creating mdev device
    • check whether available instances are left
 $ cat /sys/bus/pci/devices/0000\:00\:02.0/mdev_supported_types/i915-GVTg_V5_4/available_instances 
 1

also check dmesg output for gvt related error, most likely there is not enough VRAM

  • (libvirtd) VM stops immediately with no error other than "internal error: process exited while connecting to monitor"

Try disabling seccomp sandboxing in qemu like so: 

  virtualisation.libvirtd = {
    qemuVerbatimConfig = ''
      seccomp_sandbox = 0
    '';
  };

Useful sources

Retrieved from "https://wiki.nixos.org/w/index.php?title=IGVT-g&oldid=5843"