Jump to content

PCI passthrough

From NixOS Wiki

This guide details the configuration of PCI device passthrough using VFIO and OVMF (an open-source UEFI implementation for QEMU) on NixOS. This setup allows you to assign a physical PCI device (typically a GPU) to a virtual machine (VM) at native hardware performance.

Prerequisites:

  • Hardware with Intel VT-d or AMD-Vi enabled in BIOS.
  • A dedicated PCI device such as a GPU that is not required by the host system.
  • Kernel supporting IOMMU and VFIO. The default NixOS is sufficient.

Configuration

For more in-depth information on PCI passthrough that is not specific to NixOS, refer to the PCI passthrough via OVMF article on the Arch Wiki.

VFIO Modules

The following kernel modules are required for PCI passthrough:

  • vfio_pci
  • vfio
  • vfio_iommu_type1

If other drivers (e.g., for early modesetting such as i915, amdgpu, radeon, nouveau, etc.) are in use, they must be loaded after the VFIO modules.

❄︎ /etc/nixos/configuration.nix
  boot.initrd.kernelModules = [ 
    "vfio_pci"
    "vfio"
    "vfio_iommu_type1"

    "i915" # replace or remove with your device's driver as needed
   ];

After updating the configuration, run nixos-rebuild switch and reboot to ensure the required modules are loaded at boot.

Kernel Parameters

To enable IOMMU functionality, the following kernel parameters are required:

  • intel_iommu=on — for Intel CPUs
  • amd_iommu=on — for AMD CPUs

Additionally, specify the PCI vendor and device IDs of the devices to passthrough using the vfio-pci.ids parameter. These IDs can be obtained using the lspci -nn command.

$ lspci -nn
...
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] [1002:67b0]
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]

In this example, an AMD GPU is passed through to the virtual machine. Note that the ids to both the VGA controller and it's associated audio device get passed to the kernel parameters

❄︎ /etc/nixos/configuration.nix
  boot.kernelParams = [ 
    "intel_iommu=on" # or "amd_iommu=on"
    "vfio-pci.ids=1002:67b0,1002:aac8,144d:a80a"
  ];

libvirtd / QEMU

Enable the libvirt daemon with QEMU and OVMF (UEFI firmware) support. Secure Boot and TPM emulation are also enabled via swtpm and OVMF configuration.

Additionally, the Virt-manager application can be enabled for graphical VM management.

❄︎ /etc/nixos/configuration.nix
  programs.virt-manager.enable = true;
  virtualisation.spiceUSBRedirection.enable = true;
  virtualisation.libvirtd = {
    enable = true;
    qemu = {
      package = pkgs.qemu_kvm;
      runAsRoot = true;
      swtpm.enable = true;
      ovmf = {
        enable = true;
        packages = [(pkgs.OVMF.override {
          secureBoot = true;
          tpmSupport = true;
        }).fd];
      };
    };
  };

For more information on configuring virtualization, refer to the virtualisation.libvirtd module options.

Adding user to libvirtd group

For non-root users to manage virtual machines, add your user to the libvirtd group. Replace myUser with your actual username:

❄︎ /etc/nixos/configuration.nix
  users.extraUsers.myUser.extraGroups = [ "libvirtd" ];

VM setup and configuration

To configure networking for your virtual machine, you will need to set up a network. If you choose to use the default libvirt network, refer to the libvirt#Default networking section for detailed instructions.

For instructions on setting up a virtual machine with attached PCI devices using virt-manager, refer to the Setting up an OVMF-based guest virtual machine page on the Arch Wiki.

See Also