Official NixOS Wiki

AMD GPU: Difference between revisions

← Older edit
Artturin (talk | contribs)
trusted
24 edits
VisualWikitext
Smudgebun (talk | contribs)
65 edits
→Blender: Converted to a general section on ROCm and HIP enabling for packages, linked to the Blender page HIP section for Blender-specific information.
Artturin (talk | contribs)
trusted
24 edits
amdgpu.ids
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[https://en.wikipedia.org/wiki/AMDgpu_(Linux_kernel_module) AMDGPU] is an open source graphics driver for AMD Radeon graphics cards. It supports AMD GPUs based on the [https://en.wikipedia.org/wiki/Graphics_Core_Next GCN architecture Graphics Core Next (GCN)] architecture and later, covering hardware released from approximately 2012 onward. This guide is about configuration of NixOS to correctly use AMD GPUs supported by the AMDGPU driver.
[[Wikipedia:AMDgpu (Linux kernel module)|AMDGPU]] is an open source graphics driver for AMD Radeon graphics cards. It supports AMD GPUs based on the [[Wikipedia:Graphics Core Next|Graphics Core Next (GCN)]] architecture and later, covering hardware released from approximately 2012 onward. This guide is about configuration of NixOS to correctly use AMD GPUs supported by the AMDGPU driver.


== Basic Setup ==
== Basic Setup ==
Line 11: Line 11:
The following configurations are only required if you have a specific reason for needing them. They are not expected to be necessary for a typical desktop / gaming setup.
The following configurations are only required if you have a specific reason for needing them. They are not expected to be necessary for a typical desktop / gaming setup.


=== AMD iGPU with high amount of RAM (usecase: large language models) ===
=== AMD iGPU with high amount of RAM ===
The iGPU uses system RAM and has no dedicated VRAM. It can use up to the full available system RAM for example for large LLM models. On many systems its possible to set the amount of VRAM in BIOS: „Auto“ or the lowest amount is enough. The driver knows, to expand with GTT.
The iGPU uses system RAM and has no dedicated VRAM. On many systems its possible to set the amount of VRAM in BIOS: "Auto" or the lowest amount is enough. To set the maximum RAM that the iGPU use, you can add <code>ttm pages_limit</code> and <code>ttm page_pool_size</code> in <code>boot.extraModprobeConfig</code> option.<syntaxhighlight lang="nix"># Sets a high amount of GTT size for LLM usage.
boot.extraModprobeConfig = ''
  # Specified as 4KiB pages: 120 GB GTT.
  options ttm pages_limit=31457280


Documentation:
  # Specified as 4KiB pages: 60 GB pre-allocated.
* [https://docs.kernel.org/gpu/amdgpu/module-parameters.html amdgpu gttsize]
   options ttm page_pool_size=15728640
* [https://www.kernel.org/doc/html/v4.14/gpu/drm-mm.html#the-translation-table-manager-ttm ttm pages_limit and ttm pages_pool]
Example for 128GB system RAM, in this example the LLM can use 120 GB „VRAM/GTT“:
<syntaxhighlight lang="nix">
boot.kernelParams = [
  # The kernel module parameter gttsize is a is deprecated and will be removed in the future.
   options amdgpu gttsize=120000


   # specified as 4KiB pages: 120 GB GTT
   # `amdgpu gttsize' is deprecated and should not be used.
  options ttm pages_limit=31457280
   # options amdgpu gttsize=120000
   # specified as 4KiB pages: 60 GB pre-allocated
'';</syntaxhighlight>
  options ttm page_pool_size=15728640
];</syntaxhighlight>


=== Enable Southern Islands (SI) and Sea Islands (CIK) support (eg. HD 7000/8000) ===
=== Enable Southern Islands (SI) and Sea Islands (CIK) support (eg. HD 7000/8000) ===
Line 37: Line 31:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
boot.kernelParams = [
boot.kernelParams = [
    # For Southern Islands (SI i.e. GCN 1) cards
  # For Southern Islands (SI i.e. GCN 1) cards
    "amdgpu.si_support=1"
  "amdgpu.si_support=1"
    "radeon.si_support=0"
  "radeon.si_support=0"
    # For Sea Islands (CIK i.e. GCN 2) cards
  # For Sea Islands (CIK i.e. GCN 2) cards
    "amdgpu.cik_support=1"
  "amdgpu.cik_support=1"
    "radeon.cik_support=0"
  "radeon.cik_support=0"
];
];
</syntaxhighlight>
</syntaxhighlight>
Line 54: Line 48:


<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
  systemd.tmpfiles.rules =  
systemd.tmpfiles.rules =
   let
   let
     rocmEnv = pkgs.symlinkJoin {
     rocmEnv = pkgs.symlinkJoin {
Line 64: Line 58:
       ];
       ];
     };
     };
   in [
   in
  [
     "L+    /opt/rocm  -    -    -    -    ${rocmEnv}"
     "L+    /opt/rocm  -    -    -    -    ${rocmEnv}"
   ];
   ];
</syntaxhighlight>
</syntaxhighlight>


=== Enabling ROCm & HIP For Packages ===
=== Enabling ROCm & HIP For Packages ===
Whether or not a package is built with ROCm support is controlled by the <code>rocmSupport</code> nixpkgs config variable. As HIP is a component of ROCm, anything that needs HIP support (e.g. Blender) gets that enabled through <code>rocmSupport</code> too.
Whether or not a package is built with ROCm support is controlled by the <code>rocmSupport</code> nixpkgs config variable. As HIP is a component of ROCm, anything that needs HIP support (e.g. Blender) gets that enabled through <code>rocmSupport</code> too.


Line 83: Line 77:
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
environment.systemPackages = with pkgs; [
environment.systemPackages = with pkgs; [
(ffmpeg-full.override {config.rocmSupport=true;})
  (ffmpeg-full.override { config.rocmSupport = true; })
pkgsRocm.ffmpeg-full # equivalent to (ffmpeg-full.override {config.rocmSupport=true;}) for packages in ROCm Release attrPaths
  pkgsRocm.ffmpeg-full # equivalent to (ffmpeg-full.override {config.rocmSupport=true;}) for packages in ROCm Release attrPaths
];
];
</syntaxhighlight>
</syntaxhighlight>


Line 136: Line 129:
Vulkan is already enabled by default (using Mesa RADV) on 64 bit applications. The settings to control it are now the same as those shown in the basic setup:
Vulkan is already enabled by default (using Mesa RADV) on 64 bit applications. The settings to control it are now the same as those shown in the basic setup:


<syntaxhighlight lang="nix">hardware.graphics.enable = true;
<syntaxhighlight lang="nix">
hardware.graphics.enable = true;
hardware.graphics.enable32Bit = true; # Replaced 'driSupport32Bit'
hardware.graphics.enable32Bit = true; # Replaced 'driSupport32Bit'
</syntaxhighlight>
</syntaxhighlight>
Line 188: Line 182:
systemd.packages = with pkgs; [ lact ];
systemd.packages = with pkgs; [ lact ];
systemd.services.lactd.wantedBy = ["multi-user.target"];
systemd.services.lactd.wantedBy = ["multi-user.target"];
</syntaxhighlight>Simple version is:<syntaxhighlight lang="nix">
</syntaxhighlight>
Simple version is:
<syntaxhighlight lang="nix">
services.lact.enable = true;
services.lact.enable = true;
# https://github.com/NixOS/nixpkgs/blob/nixos-unstable/nixos/modules/services/hardware/lact.nix
# https://github.com/NixOS/nixpkgs/blob/nixos-unstable/nixos/modules/services/hardware/lact.nix
Line 196: Line 192:


=== Low resolution during initramfs phase ===
=== Low resolution during initramfs phase ===
If you encounter a low resolution output during early boot phases, you can load the amdgpu module in the initial ramdisk<syntaxhighlight lang="nix">
If you encounter a low resolution output during early boot phases, you can load the amdgpu module in the initial ramdisk
<syntaxhighlight lang="nix">
hardware.amdgpu.initrd.enable = true; # sets boot.initrd.kernelModules = ["amdgpu"];
hardware.amdgpu.initrd.enable = true; # sets boot.initrd.kernelModules = ["amdgpu"];
</syntaxhighlight>
</syntaxhighlight>
Line 217: Line 214:
To figure out the connector names, execute the following command while your monitors are connected:
To figure out the connector names, execute the following command while your monitors are connected:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="console">
head /sys/class/drm/*/status
$ head /sys/class/drm/*/status
</syntaxhighlight>
</syntaxhighlight>


Line 274: Line 271:


Solution:
Solution:
hardware.firmware = [ pkgs.linux-firmware ];
<syntaxhighlight lang="nix">
hardware.firmware = [ pkgs.linux-firmware ];
</syntaxhighlight>
 
=== PyTorch does not show the the GPU model and prints <code>(null): No such file or directory</code> ===
<syntaxhighlight lang="shell">$ python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
(null): No such file or directory
(null): No such file or directory
True AMD Radeon Graphics</syntaxhighlight>It happens because torch cannot find <code>amdgpu.ids</code><syntaxhighlight lang="nix">systemd.tmpfiles.rules = [
  "L+    /opt/amdgpu/share/libdrm/amdgpu.ids  -    -    -    -    ${pkgs.libdrm}/share/libdrm/amdgpu.ids"
];</syntaxhighlight><code>True AMD Radeon RX 9060 XT</code>


== See Also ==
== See Also ==
Retrieved from "https://wiki.nixos.org/wiki/AMD_GPU"