Official NixOS Wiki

AMD GPU: Difference between revisions

← Older edit
Artturin (talk | contribs)
trusted
24 edits
VisualWikitext
m Fix headers
Artturin (talk | contribs)
trusted
24 edits
amdgpu.ids
 
(27 intermediate revisions by 17 users not shown)
Line 1: Line 1:
This guide is about setting up NixOS to correctly use your AMD Graphics card if it is relatively new (aka, after the GCN architecture).
[[Wikipedia:AMDgpu (Linux kernel module)|AMDGPU]] is an open source graphics driver for AMD Radeon graphics cards. It supports AMD GPUs based on the [[Wikipedia:Graphics Core Next|Graphics Core Next (GCN)]] architecture and later, covering hardware released from approximately 2012 onward. This guide is about configuration of NixOS to correctly use AMD GPUs supported by the AMDGPU driver.


== Basic Setup ==
== Basic Setup ==
For ordinary desktop / gaming usage, AMD GPUs are expected to work out of the box. As with any desktop configuration though, graphics acceleration does need to be enabled.
For ordinary desktop / gaming usage, AMD GPUs are expected to work out of the box. As with any desktop configuration though, graphics acceleration does need to be enabled.
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">hardware.graphics = {
# as of 24.11
hardware.graphics = {
   enable = true;
   enable = true;
   enable32Bit = true;
   enable32Bit = true;
};
};</syntaxhighlight>There is also the [https://search.nixos.org/options?channel=unstable&query=hardware.amdgpu amdgpu nixos module available for common configuration options], such as enabling opencl, legacy support, overdrive/overclocking and loading during initrd.


# pre 24.11
== Special Configuration ==
hardware.opengl = {
The following configurations are only required if you have a specific reason for needing them. They are not expected to be necessary for a typical desktop / gaming setup.
  enable = true;
  driSupport = true;
  driSupport32Bit = true;
};


</syntaxhighlight>
=== AMD iGPU with high amount of RAM ===
The iGPU uses system RAM and has no dedicated VRAM. On many systems its possible to set the amount of VRAM in BIOS: "Auto" or the lowest amount is enough. To set the maximum RAM that the iGPU use, you can add <code>ttm pages_limit</code> and <code>ttm page_pool_size</code> in <code>boot.extraModprobeConfig</code> option.<syntaxhighlight lang="nix"># Sets a high amount of GTT size for LLM usage.
boot.extraModprobeConfig = ''
  # Specified as 4KiB pages: 120 GB GTT.
  options ttm pages_limit=31457280


== Problems ==
  # Specified as 4KiB pages: 60 GB pre-allocated.
  options ttm page_pool_size=15728640


=== Dual Monitors ===
  # `amdgpu gttsize' is deprecated and should not be used.
 
  # options amdgpu gttsize=120000
If you encounter problems having multiple monitors connected to your GPU, adding `video` parameters for each connector to the kernel command line sometimes helps.
'';</syntaxhighlight>


For example:
=== Enable Southern Islands (SI) and Sea Islands (CIK) support (eg. HD 7000/8000) ===
The oldest architectures that AMDGPU supports are [[wikipedia:Radeon_HD_7000_series|Southern Islands (SI, i.e. GCN 1)]] and [[wikipedia:Radeon_HD_8000_series|Sea Islands (CIK, i.e. GCN 2)]], but support for them is disabled by default. To use AMDGPU instead of the <code>radeon</code> driver, you can set the legacySupport option in the amdgpu module.<syntaxhighlight lang="nix">
hardware.amdgpu.legacySupport.enable = true;
</syntaxhighlight>This will set the kernel parameters as follows (this is redundant if you set the above option)


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
boot.kernelParams = [
boot.kernelParams = [
   "video=DP-1:2560x1440@144"
  # For Southern Islands (SI i.e. GCN 1) cards
   "video=DP-2:2560x1440@144"
   "amdgpu.si_support=1"
   "radeon.si_support=0"
  # For Sea Islands (CIK i.e. GCN 2) cards
  "amdgpu.cik_support=1"
  "radeon.cik_support=0"
];
];
</syntaxhighlight>
</syntaxhighlight>


With the connector names (like `DP-1`), the resolution and frame rate adjusted accordingly.
Doing this is required to use [[#Vulkan|Vulkan]] on these cards, as the <code>radeon</code> driver doesn't support it.
 
To figure out the connector names, execute the following command while your monitors are connected:
 
<syntaxhighlight lang="bash">
head /sys/class/drm/*/status
</syntaxhighlight>
 
== Special Configuration ==
The following configurations are only required if you have a specific reason for needing them. They are not expected to be necessary for a typical desktop / gaming setup.
 
=== Enable Southern Islands (SI) and Sea Islands (CIK) support ===
The oldest architectures that AMDGPU supports are [[wikipedia:Radeon_HD_7000_series|Southern Islands (SI, i.e. GCN 1)]] and [[wikipedia:Radeon_HD_8000_series|Sea Islands (CIK, i.e. GCN 2)]], but support for them is disabled by default. To use AMDGPU instead of the <code>radeon</code> driver, you can set the kernel parameters:
 
<syntaxhighlight lang="nix">
# For Southern Islands (SI i.e. GCN 1) cards
boot.kernelParams = [ "radeon.si_support=0" "amdgpu.si_support=1" ];
# For Sea Islands (CIK i.e. GCN 2) cards
boot.kernelParams = [ "radeon.cik_support=0" "amdgpu.cik_support=1" ];
</syntaxhighlight>


Doing this is required to use [[#Vulkan|Vulkan]] on these cards, as the <code>radeon</code> driver doesn't support it.
Please note this also removes support for analog video outputs, which is only available with the <code>radeon</code> driver.


=== HIP ===
=== HIP ===
Line 61: Line 48:


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
  systemd.tmpfiles.rules =  
systemd.tmpfiles.rules =
   let
   let
     rocmEnv = pkgs.symlinkJoin {
     rocmEnv = pkgs.symlinkJoin {
Line 71: Line 58:
       ];
       ];
     };
     };
   in [
   in
  [
     "L+    /opt/rocm  -    -    -    -    ${rocmEnv}"
     "L+    /opt/rocm  -    -    -    -    ${rocmEnv}"
   ];
   ];
</syntaxhighlight>


=== Enabling ROCm & HIP For Packages ===
Whether or not a package is built with ROCm support is controlled by the <code>rocmSupport</code> nixpkgs config variable. As HIP is a component of ROCm, anything that needs HIP support (e.g. Blender) gets that enabled through <code>rocmSupport</code> too.
You can set it globally with this line
<syntaxhighlight lang="nix">
nixpkgs.config.rocmSupport = true;
</syntaxhighlight>
</syntaxhighlight>


==== Blender ====
Or override specific packages
Hardware accelerated rendering can be achieved by using the package <syntaxhighlight lang="nix" inline="">blender-hip</syntaxhighlight>.


Currently, you need to [[Linux kernel|use the latest kernel]] for <syntaxhighlight lang="nix" inline="">blender-hip</syntaxhighlight> to work.
<syntaxhighlight lang="nix">
environment.systemPackages = with pkgs; [
  (ffmpeg-full.override { config.rocmSupport = true; })
  pkgsRocm.ffmpeg-full # equivalent to (ffmpeg-full.override {config.rocmSupport=true;}) for packages in ROCm Release attrPaths
];
</syntaxhighlight>
 
While most if not all packages that support ROCm should be in the [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/rocm-modules/release-attrPaths.json ROCm Release attrPaths], and therefore built by Hydra and cached in cache.nixos.org with <code>rocmSupport</code>, enabling it globally still has a slight chance of pointless compiling on your machine.
 
For Blender-specific information on setting up HIP support, see: [[Blender#HIP]].


=== OpenCL ===
=== OpenCL ===
<syntaxhighlight lang="nix">
OpenCL support using the ROCM runtime library can be enabled via the amdgpu module.<syntaxhighlight lang="nix">
hardware.graphics.extraPackages = with pkgs; [ rocmPackages.clr.icd ];
hardware.amdgpu.opencl.enable = true;
</syntaxhighlight>
</syntaxhighlight>


Line 123: Line 127:


=== Vulkan ===
=== Vulkan ===
Vulkan is already enabled by default (using Mesa RADV) on 64 bit applications. The settings to control it are:
Vulkan is already enabled by default (using Mesa RADV) on 64 bit applications. The settings to control it are now the same as those shown in the basic setup:


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
hardware.opengl.driSupport = true; # This is already enabled by default
hardware.graphics.enable = true;
hardware.opengl.driSupport32Bit = true; # For 32 bit applications
hardware.graphics.enable32Bit = true; # Replaced 'driSupport32Bit'
</syntaxhighlight>
</syntaxhighlight>


==== AMDVLK ====
==== AMDVLK ====
The AMDVLK drivers can be used in addition to the Mesa RADV drivers. The program will choose which one to use:
 
{{Warning|1=AMDVLK is being discontinued, so the only reason you may want to use it is if you have issues with RADV. See: https://github.com/GPUOpen-Drivers/AMDVLK/discussions/416.}}
 
The AMDVLK drivers can be used in addition to the Mesa RADV drivers. The program will choose which one to use (mostly defaulting to AMDVLK):


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
#24.11
hardware.graphics.extraPackages = with pkgs; [
hardware.graphics.extraPackages = with pkgs; [
   amdvlk
   amdvlk
Line 142: Line 148:
   driversi686Linux.amdvlk
   driversi686Linux.amdvlk
];
];
</syntaxhighlight>
More information can be found here: https://nixos.org/manual/nixos/unstable/index.html#sec-gpu-accel-vulkan


#24.05 and below
One way to avoid installing it system-wide, is creating a wrapper like this:
hardware.opengl.extraPackages = with pkgs; [
 
   amdvlk
<syntaxhighlight lang="nix">
];
with pkgs; writeShellScriptBin "amdvlk-run" ''
# For 32 bit applications
   export VK_DRIVER_FILES="${amdvlk}/share/vulkan/icd.d/amd_icd64.json:${pkgsi686Linux.amdvlk}/share/vulkan/icd.d/amd_icd32.json"
hardware.opengl.extraPackages32 = with pkgs; [
   exec "$@"
   driversi686Linux.amdvlk
'';
];
</syntaxhighlight>
</syntaxhighlight>


More information can be found here: https://nixos.org/manual/nixos/unstable/index.html#sec-gpu-accel-vulkan
This wrapper can be used per-game inside Steam (<syntaxhighlight lang="sh" inline>amdvlk-run %command%</syntaxhighlight>) and lets RADV to be the system's default.
 
===== Performance Issues with AMDVLK =====
Some games choose AMDVLK over RADV, which can cause noticeable performance issues (e.g. <50% less FPS in games)
 
To force RADV<syntaxhighlight lang="nix">
environment.variables.AMD_VULKAN_ICD = "RADV";
</syntaxhighlight>


=== GUI tools ===
=== GUI tools ===


==== LACT - Linux AMDGPU Controller ====
==== LACT - Linux AMDGPU Controller ====
This application allows you to overclock, undervolt, set fans curves of AMD GPUs on a Linux system.
This application allows you to overclock, undervolt, set fans curves of AMD GPUs on a Linux system.


In order to install the daemon service you need to add the package to <code>systemd.packages</code>. Also the <code>wantedBy</code> field should be set to <code>multi-user.target</code> to start the service during boot.
In order to install the daemon service you need to add the package to <code>systemd.packages</code>. Also the <code>wantedBy</code> field should be set to <code>multi-user.target</code> to start the service during boot.
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
environment.systemPackages = with pkgs; [ lact ];
environment.systemPackages = with pkgs; [ lact ];
Line 166: Line 183:
systemd.services.lactd.wantedBy = ["multi-user.target"];
systemd.services.lactd.wantedBy = ["multi-user.target"];
</syntaxhighlight>
</syntaxhighlight>
Simple version is:
<syntaxhighlight lang="nix">
services.lact.enable = true;
# https://github.com/NixOS/nixpkgs/blob/nixos-unstable/nixos/modules/services/hardware/lact.nix
</syntaxhighlight>
== Troubleshooting ==
=== Low resolution during initramfs phase ===
If you encounter a low resolution output during early boot phases, you can load the amdgpu module in the initial ramdisk
<syntaxhighlight lang="nix">
hardware.amdgpu.initrd.enable = true; # sets boot.initrd.kernelModules = ["amdgpu"];
</syntaxhighlight>
=== Dual Monitors ===
If you encounter problems having multiple monitors connected to your GPU, adding `video` parameters for each connector to the kernel command line sometimes helps.
For example:
<syntaxhighlight lang="nix">
boot.kernelParams = [
  "video=DP-1:2560x1440@144"
  "video=DP-2:2560x1440@144"
];
</syntaxhighlight>
With the connector names (like `DP-1`), the resolution and frame rate adjusted accordingly.
To figure out the connector names, execute the following command while your monitors are connected:
<syntaxhighlight lang="console">
$ head /sys/class/drm/*/status
</syntaxhighlight>
=== System Hang with Vega Graphics (and select GPUs) ===
Currently on the latest kernel/mesa (currently 6.13 and 24.3.4 respectively), Vega integrated graphics (and other GPUs like the RX 6600<ref>https://bbs.archlinux.org/viewtopic.php?pid=2224147#p2224147</ref>) will have a possibility to hang due to context-switching between Graphics and Compute.<ref>https://bbs.archlinux.org/viewtopic.php?id=301798</ref> There are currently two sets of patches to choose between stability or speed that can be applied: [https://github.com/SeryogaBrigada/linux/commits/v6.13-amdgpu amdgpu-stable] and [https://github.com/SeryogaBrigada/linux/commits/v6.13-amdgpu-testing amdgpu-testing].
See [[Linux Kernel#Patching a single In-tree kernel module]], keep in mind how to make [https://stackoverflow.com/a/23525893 patch diffs from commits from GitHub], and consider this example configuration:<syntaxhighlight lang="nix">
{ config, pkgs, ... }:
let
  amdgpu-kernel-module = pkgs.callPackage ./packages/amdgpu-kernel-module.nix {
    # Make sure the module targets the same kernel as your system is using.
    kernel = config.boot.kernelPackages.kernel;
  };
  # linuxPackages_latest 6.13 (or linuxPackages_zen 6.13)
  amdgpu-stability-patch = pkgs.fetchpatch {
    name = "amdgpu-stability-patch";
    url = "https://github.com/torvalds/linux/compare/ffd294d346d185b70e28b1a28abe367bbfe53c04...SeryogaBrigada:linux:4c55a12d64d769f925ef049dd6a92166f7841453.diff";
    hash = "sha256-q/gWUPmKHFBHp7V15BW4ixfUn1kaeJhgDs0okeOGG9c=";
  };
  /*
  # linuxPackages_zen 6.12
  amdgpu-stability-patch = pkgs.fetchpatch {
    name = "amdgpu-stability-patch-zen";
    url = "https://github.com/zen-kernel/zen-kernel/compare/fd00d197bb0a82b25e28d26d4937f917969012aa...WhiteHusky:zen-kernel:f4c32ca166ad55d7e2bbf9adf121113500f3b42b.diff";
    hash = "sha256-bMT5OqBCyILwspWJyZk0j0c8gbxtcsEI53cQMbhbkL8=";
  };
  */
in
{
  # amdgpu instability with context switching between compute and graphics
  # https://bbs.archlinux.org/viewtopic.php?id=301798
  # side-effects: plymouth fails to show at boot, but does not interfere with booting
  boot.extraModulePackages = [
    (amdgpu-kernel-module.overrideAttrs (_: {
      patches = [
        amdgpu-stability-patch
      ];
    }))
  ];
}
</syntaxhighlight>
=== Sporadic Crashes ===
If getting error messages in <code>dmesg</code> with <code>page fault</code> or <code>GCVM_L2_PROTECTION_FAULT_STATUS</code> it might be from AMD GPU boosting too high without enough voltage
Use a tool like LACT to increase power usage limit to 15%, undervolt by moderate amount (e.g. -50mV for 7900 XTX) and optionally decrease maximum GPU clock.
* https://wiki.gentoo.org/wiki/AMDGPU#Frequent_and_Sporadic_Crashes
* https://gitlab.freedesktop.org/mesa/mesa/-/issues/11532
* https://gitlab.freedesktop.org/drm/amd/-/issues/3067
=== Error: <code>amdgpu: Failed to get gpu_info firmware</code> ===
Solution:
<syntaxhighlight lang="nix">
hardware.firmware = [ pkgs.linux-firmware ];
</syntaxhighlight>
=== PyTorch does not show the the GPU model and prints <code>(null): No such file or directory</code> ===
<syntaxhighlight lang="shell">$ python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
(null): No such file or directory
(null): No such file or directory
True AMD Radeon Graphics</syntaxhighlight>It happens because torch cannot find <code>amdgpu.ids</code><syntaxhighlight lang="nix">systemd.tmpfiles.rules = [
  "L+    /opt/amdgpu/share/libdrm/amdgpu.ids  -    -    -    -    ${pkgs.libdrm}/share/libdrm/amdgpu.ids"
];</syntaxhighlight><code>True AMD Radeon RX 9060 XT</code>
== See Also ==
* https://wiki.archlinux.org/title/AMDGPU
* https://wiki.gentoo.org/wiki/AMDGPU
== References ==
[[Category:Video]]
[[Category:Video]]
Retrieved from "https://wiki.nixos.org/wiki/AMD_GPU"