Jump to content

PCI passthrough: Difference between revisions

From NixOS Wiki
Klinger (talk | contribs)
mNo edit summary
Pigs (talk | contribs)
Detailed configuration instructions on how to set everything required to configure pci passthrough
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
Good Resources:
This guide details the configuration of PCI device passthrough using [https://docs.kernel.org/driver-api/vfio.html VFIO] and [https://github.com/tianocore/tianocore.github.io/wiki/OVMF OVMF] (an open-source [https://en.wikipedia.org/wiki/UEFI UEFI] implementation for [https://www.qemu.org/ QEMU]) on NixOS. This setup allows you to assign a physical PCI device (typically a GPU) to a virtual machine (VM) at native hardware performance.


Prerequisites:


https://gist.github.com/techhazard/1be07805081a4d7a51c527e452b87b26
* Hardware with Intel VT-d or AMD-Vi enabled in BIOS.


https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
* A dedicated PCI device such as a GPU that is not required by the host system.
 
* Kernel supporting IOMMU and VFIO. The default NixOS is sufficient.
 
= Configuration =
 
For more in-depth information on PCI passthrough that is not specific to NixOS, refer to the [https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF PCI passthrough via OVMF] article on the Arch Wiki.
 
== VFIO Modules ==
 
The following kernel modules are required for PCI passthrough:
 
* <code>vfio_pci</code>
* <code>vfio</code>
* <code>vfio_iommu_type1</code>
 
If other drivers (e.g., for early modesetting such as <code>i915</code>, <code>amdgpu</code>, <code>radeon</code>, <code>nouveau</code>, etc.) are in use, they must be loaded after the VFIO modules.
 
{{file|/etc/nixos/configuration.nix|nix|
<nowiki>
  boot.initrd.kernelModules = [
    "vfio_pci"
    "vfio"
    "vfio_iommu_type1"
 
    "i915" # replace or remove with your device's driver as needed
  ];
</nowiki>
}}
 
After updating the configuration, run <code>nixos-rebuild switch</code> and reboot to ensure the required modules are loaded at boot.
 
== Kernel Parameters ==
 
To enable IOMMU functionality, the following [[Linux kernel#Customizing kernel module parameters|kernel parameters]] are required:
 
* <code>intel_iommu=on</code> — for Intel CPUs
 
* <code>amd_iommu=on</code> — for AMD CPUs
 
Additionally, specify the PCI vendor and device IDs of the devices to passthrough using the <code>vfio-pci.ids</code> parameter.  These IDs can be obtained using the <code>lspci -nn</code> command.
 
<syntaxhighlight lang=console>
$ lspci -nn
...
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] [1002:67b0]
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]
</syntaxhighlight>
 
In this example, an AMD GPU is passed through to the virtual machine. Note that the ids to both the VGA controller and it's associated audio device get passed to the kernel parameters
 
{{file|/etc/nixos/configuration.nix|nix|
<nowiki>
  boot.kernelParams = [
    "intel_iommu=on" # or "amd_iommu=on"
    "vfio-pci.ids=1002:67b0,1002:aac8,144d:a80a"
  ];
</nowiki>
}}
 
== libvirtd / QEMU ==
 
Enable the [[libvirt]] daemon with [[QEMU]] and OVMF (UEFI firmware) support. Secure Boot and TPM emulation are also enabled via swtpm and OVMF configuration.
 
Additionally, the [[Virt-manager]] application can be enabled for graphical VM management.
 
{{file|/etc/nixos/configuration.nix|nix|
<nowiki>
  programs.virt-manager.enable = true;
  virtualisation.spiceUSBRedirection.enable = true;
  virtualisation.libvirtd = {
    enable = true;
    qemu = {
      package = pkgs.qemu_kvm;
      runAsRoot = true;
      swtpm.enable = true;
      ovmf = {
        enable = true;
        packages = [(pkgs.OVMF.override {
          secureBoot = true;
          tpmSupport = true;
        }).fd];
      };
    };
  };
</nowiki>
}}
 
For more information on configuring virtualization, refer to the {{nixos:option|virtualisation.libvirtd}} module options. 
 
=== Adding user to libvirtd group ===
 
For non-root users to manage virtual machines, add your user to the <code>libvirtd</code> group. Replace <code>myUser</code> with your actual username:
 
{{file|/etc/nixos/configuration.nix|nix|
<nowiki>
  users.extraUsers.myUser.extraGroups = [ "libvirtd" ];
</nowiki>
}}
 
== VM setup and configuration ==
 
To configure networking for your virtual machine, you will need to set up a network. If you choose to use the default libvirt network, refer to the [[libvirt#Default networking]] section for detailed instructions.
 
For instructions on setting up a virtual machine with attached PCI devices using <code>virt-manager</code>, refer to the [https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Setting_up_an_OVMF-based_guest_virtual_machine Setting up an OVMF-based guest virtual machine] page on the Arch Wiki.
 
= See Also =
 
* [https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF PCI passthrough via OVMF - ArchWiki]
* [https://pigs.dev/posts/2025-04-15-gaming-in-vm-with-nixos.html Gaming in a Windows VM with NixOS - pigs.dev (April, 2025)]
* [https://astrid.tech/2022/09/22/0/nixos-gpu-vfio/ A GPU Passthrough Setup for NixOS (with VR passthrough too!) - astrid.tech (September, 2022)]
* [https://alexbakker.me/post/nixos-pci-passthrough-qemu-vfio.html Notes on PCI Passthrough on NixOS using QEMU and VFIO - alexbakker.me (September, 2019)]
* [https://gist.github.com/techhazard/1be07805081a4d7a51c527e452b87b26 PCI Passthrough - GitHub Gist by techhazard (April, 2017)]


[[Category:Hardware]]
[[Category:Hardware]]
[[Category:Virtualization]]
[[Category:Guide]]

Latest revision as of 21:08, 12 May 2025

This guide details the configuration of PCI device passthrough using VFIO and OVMF (an open-source UEFI implementation for QEMU) on NixOS. This setup allows you to assign a physical PCI device (typically a GPU) to a virtual machine (VM) at native hardware performance.

Prerequisites:

  • Hardware with Intel VT-d or AMD-Vi enabled in BIOS.
  • A dedicated PCI device such as a GPU that is not required by the host system.
  • Kernel supporting IOMMU and VFIO. The default NixOS is sufficient.

Configuration

For more in-depth information on PCI passthrough that is not specific to NixOS, refer to the PCI passthrough via OVMF article on the Arch Wiki.

VFIO Modules

The following kernel modules are required for PCI passthrough:

  • vfio_pci
  • vfio
  • vfio_iommu_type1

If other drivers (e.g., for early modesetting such as i915, amdgpu, radeon, nouveau, etc.) are in use, they must be loaded after the VFIO modules.

❄︎ /etc/nixos/configuration.nix
  boot.initrd.kernelModules = [ 
    "vfio_pci"
    "vfio"
    "vfio_iommu_type1"

    "i915" # replace or remove with your device's driver as needed
   ];

After updating the configuration, run nixos-rebuild switch and reboot to ensure the required modules are loaded at boot.

Kernel Parameters

To enable IOMMU functionality, the following kernel parameters are required:

  • intel_iommu=on — for Intel CPUs
  • amd_iommu=on — for AMD CPUs

Additionally, specify the PCI vendor and device IDs of the devices to passthrough using the vfio-pci.ids parameter. These IDs can be obtained using the lspci -nn command.

$ lspci -nn
...
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] [1002:67b0]
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]

In this example, an AMD GPU is passed through to the virtual machine. Note that the ids to both the VGA controller and it's associated audio device get passed to the kernel parameters

❄︎ /etc/nixos/configuration.nix
  boot.kernelParams = [ 
    "intel_iommu=on" # or "amd_iommu=on"
    "vfio-pci.ids=1002:67b0,1002:aac8,144d:a80a"
  ];

libvirtd / QEMU

Enable the libvirt daemon with QEMU and OVMF (UEFI firmware) support. Secure Boot and TPM emulation are also enabled via swtpm and OVMF configuration.

Additionally, the Virt-manager application can be enabled for graphical VM management.

❄︎ /etc/nixos/configuration.nix
  programs.virt-manager.enable = true;
  virtualisation.spiceUSBRedirection.enable = true;
  virtualisation.libvirtd = {
    enable = true;
    qemu = {
      package = pkgs.qemu_kvm;
      runAsRoot = true;
      swtpm.enable = true;
      ovmf = {
        enable = true;
        packages = [(pkgs.OVMF.override {
          secureBoot = true;
          tpmSupport = true;
        }).fd];
      };
    };
  };

For more information on configuring virtualization, refer to the virtualisation.libvirtd module options.

Adding user to libvirtd group

For non-root users to manage virtual machines, add your user to the libvirtd group. Replace myUser with your actual username:

❄︎ /etc/nixos/configuration.nix
  users.extraUsers.myUser.extraGroups = [ "libvirtd" ];

VM setup and configuration

To configure networking for your virtual machine, you will need to set up a network. If you choose to use the default libvirt network, refer to the libvirt#Default networking section for detailed instructions.

For instructions on setting up a virtual machine with attached PCI devices using virt-manager, refer to the Setting up an OVMF-based guest virtual machine page on the Arch Wiki.

See Also