NVIDIA: Difference between revisions
Completely rewrote the entire damn thing |
Focusgraph (talk | contribs) mNo edit summary |
||
| (30 intermediate revisions by 17 users not shown) | |||
| Line 4: | Line 4: | ||
=== Kernel modules from NVIDIA === | === Kernel modules from NVIDIA === | ||
Kernel modules from NVIDIA offer better performance than other alternatives, but make the system unfree by requiring proprietary userspace libraries that can interface with the kernel modules. Users that want to have a fully free and open-source system should use [[ | Kernel modules from NVIDIA offer better performance than other alternatives, but make the system unfree by requiring proprietary userspace libraries that can interface with the kernel modules. Users that want to have a fully free and open-source system should use [[NVIDIA#Nouveau|Nouveau]] instead. | ||
To enable them, add <code>"nvidia"</code> to the list of enabled video drivers defined by the <code>services.xserver.videoDrivers</code> option. | To enable them, add <code>"nvidia"</code> to the list of enabled video drivers defined by the <code>services.xserver.videoDrivers</code> option. | ||
| Line 10: | Line 10: | ||
{{Note|<code>hardware.graphics.enable</code> was named <code>hardware.opengl.enable</code> '''until NixOS 24.11'''.}} | {{Note|<code>hardware.graphics.enable</code> was named <code>hardware.opengl.enable</code> '''until NixOS 24.11'''.}} | ||
{{Note|Since driver version 560, you also will need to decide whether to use the open-source or proprietary modules by setting the <code>hardware.nvidia.open</code> option to either <code>true</code> or <code>false</code> respectively.<br><br>Open-source kernel modules are preferred over and planned to steadily replace proprietary modules<ref>https://developer.nvidia.com/blog/nvidia-transitions-fully-towards-open-source-gpu-kernel-modules/</ref>, although they only support GPUs of the Turing architecture or newer (from GeForce RTX 20 series and GeForce GTX 16 series onwards). Data center GPUs starting from Grace Hopper or | {{Note|Since driver version 560, you also will need to decide whether to use the open-source or proprietary modules by setting the <code>hardware.nvidia.open</code> option to either <code>true</code> or <code>false</code> respectively.<br><br>Open-source kernel modules are preferred over and planned to steadily replace proprietary modules<ref>https://developer.nvidia.com/blog/nvidia-transitions-fully-towards-open-source-gpu-kernel-modules/</ref>, although they only support GPUs of the Turing architecture or newer (from GeForce RTX 20 series and GeForce GTX 16 series onwards). Data center GPUs starting from Grace Hopper or Blackwell must use open-source modules — proprietary modules are no longer supported.<br><br>Make sure to allow [[Unfree software|unfree software]] even when using the open module as the user space part of the driver is still proprietary. Other unfree NVIDIA packages include <code>nvidia-x11</code>, <code>nvidia-settings</code>, and <code>nvidia-persistenced</code>. | ||
}}{{file|configuration.nix|nix|<nowiki> | }}{{Warning|If you use a laptop with both dedicated and integrated GPUs, remember to [[#Hybrid_graphics_with_PRIME|configure PRIME]] in order to make your dedicated NVIDIA GPU work properly with your integrated GPU. Your configuration '''might not work''' if you skip this step.}}{{file|configuration.nix|nix|<nowiki> | ||
{ | { | ||
hardware.graphics.enable = true; | hardware.graphics.enable = true; | ||
services.xserver.videoDrivers = [ "nvidia" ]; | services.xserver.videoDrivers = [ "nvidia" ]; | ||
hardware.nvidia.open = true; # | hardware.nvidia.open = true; # see the note above | ||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
==== Legacy branches ==== | ==== Legacy branches ==== | ||
GPUs of the Kepler architecture or older (most GeForce 600/700/800M cards and older) are no longer supported by latest proprietary modules. | GPUs of the Kepler architecture or older (most GeForce 600/700/800M cards and older) are no longer supported by latest proprietary modules. Instead, users of these GPUs must use legacy branches that may still receive updates, as long as the GPUs themselves remain supported by NVIDIA. You can find which legacy branch you need to use by searching for your GPU model on [https://www.nvidia.com/en-us/drivers/unix/legacy-gpu/ NVIDIA's official legacy driver support list]. | ||
To use legacy branches, you need to set the <code>hardware.nvidia.package</code> option to a package set named <code>config.boot.kernelPackages.nvidiaPackages.legacy_<branch></code>.{{file|configuration.nix|nix|<nowiki> | To use legacy branches, you need to set the <code>hardware.nvidia.package</code> option to a package set named <code>config.boot.kernelPackages.nvidiaPackages.legacy_<branch></code>.{{file|configuration.nix|nix|<nowiki> | ||
{ config, ... }: # | { config, ... }: # ← Required to get the packages used by the currently configured kernel, including drivers | ||
{ | { | ||
# Last version that supports Kepler GPUs | # Last version that supports Kepler GPUs | ||
| Line 61: | Line 59: | ||
Laptops often feature both an integrated GPU (iGPU) and a dedicated GPU (dGPU) in order to strive a balance between performance and power consumption — while the dGPU is used for performance-intensive tasks such as gaming, video editing, 3D rendering, compute jobs, etc., the iGPU can be used to render common 2D elements like application windows and the desktop environment. | Laptops often feature both an integrated GPU (iGPU) and a dedicated GPU (dGPU) in order to strive a balance between performance and power consumption — while the dGPU is used for performance-intensive tasks such as gaming, video editing, 3D rendering, compute jobs, etc., the iGPU can be used to render common 2D elements like application windows and the desktop environment. | ||
PRIME, therefore is a technology developed to facilitate the cooperation between the two GPUs and is critical for the laptop's graphical performance. Depending on your needs, you can configure PRIME in one of three modes, which have different tradeoffs in terms of performance and battery life. | PRIME, therefore, is a technology developed to facilitate the cooperation between the two GPUs and is critical for the laptop's graphical performance. Depending on your needs, you can configure PRIME in one of three modes, which have different tradeoffs in terms of performance and battery life. | ||
==== Common setup ==== | ==== Common setup ==== | ||
All PRIME configurations require setting the PCI bus IDs of the two GPUs. One easy way to do find their IDs is by running <code>lspci</code> from the <code>pciutils</code> package and then finding devices that are classified as VGA controllers. After double checking that the listed devices are indeed your integrated and dedicated GPUs, you can then find the PCI IDs at the beginning of each line. Exact results may vary, but an example output might look like:<syntaxhighlight lang="console"> | All PRIME configurations require setting the PCI bus IDs of the two GPUs. One easy way to do find their IDs is by running <code>lspci</code> from the <code>pciutils</code> package, and then finding devices that are classified as VGA controllers. After double checking that the listed devices are indeed your integrated and dedicated GPUs, you can then find the PCI IDs at the beginning of each line. Exact results may vary, but an example output might look like:<syntaxhighlight lang="console"> | ||
$ nix shell nixpkgs#pciutils -c | $ nix shell nixpkgs#pciutils -c lspci -D -d ::03xx | ||
0000:00:02.0 VGA compatible controller: Intel Corporation TigerLake-H GT1 [UHD Graphics] (rev 01) | 0000:00:02.0 VGA compatible controller: Intel Corporation TigerLake-H GT1 [UHD Graphics] (rev 01) | ||
0000:01:00.0 VGA compatible controller: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] (rev a1) | 0000:01:00.0 VGA compatible controller: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] (rev a1) | ||
</syntaxhighlight>Before putting them into your configuration, however, '''they must first be reformatted''' — | </syntaxhighlight>Before putting them into your configuration, however, '''they must first be reformatted''' — assuming the bus address is <code><domain>:<bus>:<device>.<func></code>, convert all numbers from hexadecimal to decimal, then the formatted string is <code>PCI:<bus>@<domain>:<device>:<func></code>. They can be set under <code>intelBusId</code>, <code>nvidiaBusId</code>, or <code>amdgpuBusId</code> in <code>hardware.nvidia.prime</code>, depending on the manufacturer of the GPU:{{file|configuration.nix|nix|<nowiki> | ||
{ | { | ||
hardware.nvidia.prime = { | hardware.nvidia.prime = { | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0@0:2:0"; | ||
nvidiaBusId = "PCI:1:0:0"; | nvidiaBusId = "PCI:1@0:0:0"; | ||
#amdgpuBusId = "PCI: | # amdgpuBusId = "PCI:5@0:0:0"; # If you have an AMD iGPU | ||
}; | }; | ||
} | } | ||
| Line 79: | Line 77: | ||
==== Offload mode ==== | ==== Offload mode ==== | ||
{{Note|Offload mode is available '''since NixOS 20.09 and NVIDIA driver version 435.21''', and requires an NVIDIA GPU of the Turing generation, or newer and a compatible CPU — either an Intel CPU from the Coffee Lake generation or newer, or an AMD Ryzen.}} | {{Note|Offload mode is available '''since NixOS 20.09 and NVIDIA driver version 435.21''', and requires an NVIDIA GPU of the Turing generation, or newer and a compatible CPU — either an Intel CPU from the Coffee Lake generation or newer, or an AMD Ryzen. Offload mode is '''incompatible''' with sync mode.}} | ||
Offload mode puts your dGPU to sleep and lets the iGPU handle all tasks, except if you call the dGPU specifically by "offloading" an application to it. For example, you can run your laptop normally and it will use the energy-efficient iGPU all day, and then you can offload a game from Steam onto the dGPU to make the dGPU run that game only. For many, this is the most desirable option. | Offload mode puts your dGPU to sleep and lets the iGPU handle all tasks, except if you call the dGPU specifically by "offloading" an application to it. For example, you can run your laptop normally and it will use the energy-efficient iGPU all day, and then you can offload a game from Steam onto the dGPU to make the dGPU run that game only. For many, this is the most desirable option. | ||
To enable offload mode, set the <code>hardware.nvidia.prime.offload.enable</code> option to <code>true</code>: | To enable offload mode, set the <code>hardware.nvidia.prime.offload.enable</code> option to <code>true</code>: | ||
| Line 86: | Line 84: | ||
{{file|configuration.nix|nix|<nowiki> | {{file|configuration.nix|nix|<nowiki> | ||
{ | { | ||
# For offloading, `modesetting` is needed additionally, | |||
# otherwise the X-server will be running permanently on nvidia, | |||
# thus keeping the GPU always on (see `nvidia-smi`). | |||
services.xserver.videoDrivers = [ | |||
"modesetting" # example for Intel iGPU; use "amdgpu" here instead if your iGPU is AMD | |||
"nvidia" | |||
]; | |||
hardware.nvidia.prime = { | hardware.nvidia.prime = { | ||
offload.enable = true; | offload.enable = true; | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0@0:2:0"; | ||
nvidiaBusId = "PCI:1:0:0"; | nvidiaBusId = "PCI:1@0:0:0"; | ||
#amdgpuBusId = "PCI: | # amdgpuBusId = "PCI:5@0:0:0"; # If you have an AMD iGPU | ||
}; | }; | ||
} | } | ||
| Line 112: | Line 118: | ||
sync.enable = true; | sync.enable = true; | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0@0:2:0"; | ||
nvidiaBusId = "PCI:1:0:0"; | nvidiaBusId = "PCI:1@0:0:0"; | ||
#amdgpuBusId = "PCI: | # amdgpuBusId = "PCI:5@0:0:0"; # If you have an AMD iGPU | ||
}; | }; | ||
} | } | ||
| Line 120: | Line 126: | ||
==== Reverse sync mode ==== | ==== Reverse sync mode ==== | ||
{{Note|Reverse sync mode is available '''since NixOS 23.05 and NVIDIA driver version 460.39''' and is still an experimental, buggy feature<ref>https://forums.developer.nvidia.com/t/the-all-new-outputsink-feature-aka-reverse-prime/129828/67</ref>. '''Your mileage may vary.''' Reverse sync mode | {{Note|Reverse sync mode is available '''since NixOS 23.05 and NVIDIA driver version 460.39''' and is still an experimental, buggy feature<ref>https://forums.developer.nvidia.com/t/the-all-new-outputsink-feature-aka-reverse-prime/129828/67</ref>. '''Your mileage may vary.''' Reverse sync mode is '''incompatible''' with sync mode and requires using a desktop manager that respects the <code>services.xserver.displayManager.setupCommands</code> option, including LightDM, GDM and SDDM.}}The difference between regular sync mode and reverse sync mode is that the '''dGPU''' is configured as the primary output device, allowing displaying to external displays wired to it and not the iGPU (more common). | ||
To enable sync mode, set the <code>hardware.nvidia.prime.reverseSync.enable</code> option to <code>true</code>: | To enable reverse sync mode, set the <code>hardware.nvidia.prime.reverseSync.enable</code> option to <code>true</code>: | ||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
| Line 129: | Line 135: | ||
reverseSync.enable = true; | reverseSync.enable = true; | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0@0:2:0"; | ||
nvidiaBusId = "PCI:1:0:0"; | nvidiaBusId = "PCI:1@0:0:0"; | ||
#amdgpuBusId = "PCI: | # amdgpuBusId = "PCI:5@0:0:0"; # If you have an AMD iGPU | ||
}; | }; | ||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
=== | === Wayland === | ||
==== Requirements ==== | |||
Wayland requires kernel mode setting (KMS) to be enabled (Highly Recommended): | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
{ | |||
hardware.nvidia.modesetting.enable = true; | |||
} | |||
</nowiki>}} | |||
==== Supported Compositors ==== | |||
* '''GNOME (Wayland)''' | |||
Fully supported on recent drivers (≥ 535 recommended, ≥ 555 strongly recommended). | |||
* '''KDE Plasma (Wayland)''' | |||
Usable since Plasma 6 with recent NVIDIA drivers, though some issues may remain. | |||
* '''Hyprland''' | |||
Generally works with recent NVIDIA drivers, but support is not officially guaranteed. Regressions may occur after driver or compositor updates. | |||
==== PRIME and Wayland ==== | |||
* PRIME '''sync''' and '''reverse sync''' modes are '''X11-only''' and do not work under Wayland. | |||
* PRIME '''offload''' works under Wayland, but application offloading may behave differently depending on the compositor. | |||
==== Explict Sync ==== | |||
Drivers ≥ 555 introduce explicit sync support, which greatly improves frame pacing and reduces flickering and stuttering under Wayland. For the best Wayland experience, recent NVIDIA drivers are strongly recommended. | |||
== Tips and tricks == | == Tips and tricks == | ||
=== Check nixos-hardware === | === Check nixos-hardware === | ||
The [https://github.com/NixOS/nixos-hardware nixos-hardware] project attempts to provide configurations that address specific hardware quirks for different devices. It is possible that someone already wrote a hardware configuration for your device and that usually takes care of drivers. If so, follow the upstream documentation to enable the required modules. | |||
=== Multiple boot configurations === | === Multiple boot configurations === | ||
| Line 182: | Line 216: | ||
{{file|/etc/nixos/nvidia.nix|nix|<nowiki> | {{file|/etc/nixos/nvidia.nix|nix|<nowiki> | ||
package = config.boot.kernelPackages.nvidiaPackages.mkDriver { | { config, ... }: | ||
{ | |||
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.mkDriver { | |||
version = "555.58.02"; | version = "555.58.02"; | ||
sha256_64bit = "sha256-xctt4TPRlOJ6r5S54h5W6PT6/3Zy2R4ASNFPu8TSHKM="; | sha256_64bit = "sha256-xctt4TPRlOJ6r5S54h5W6PT6/3Zy2R4ASNFPu8TSHKM="; | ||
| Line 190: | Line 226: | ||
persistencedSha256 = lib.fakeSha256; | persistencedSha256 = lib.fakeSha256; | ||
}; | }; | ||
}; | |||
</nowiki>}} | </nowiki>}} | ||
| Line 199: | Line 236: | ||
This allows you to pin the specific driver version being used in your NixOS installation. | This allows you to pin the specific driver version being used in your NixOS installation. | ||
You might want to do this if you are running the newest kernel, as the packaged drivers may fail to build otherwise<ref>https://github.com/NixOS/nixpkgs/issues/429624#issuecomment-3189861599</ref>. | |||
== Troubleshooting == | == Troubleshooting == | ||
=== Booting to text mode === | === Booting to text mode === | ||
If you encounter the problem of booting to text mode you might try adding the | If you encounter the problem of booting to text mode you might try adding the NVIDIA kernel module manually with: | ||
<syntaxHighlight lang="nix"> | <syntaxHighlight lang="nix"> | ||
| Line 230: | Line 268: | ||
<code>powerManagement.enable = true</code> can sometimes fix this, but is itself unstable and is known to cause suspend issues. | <code>powerManagement.enable = true</code> can sometimes fix this, but is itself unstable and is known to cause suspend issues. | ||
If you have a modern | <code>hardware.nvidia.powerManagement.enable = true</code> can also sometimes fix this issue; it is <code>false</code> by default. | ||
{{Note|When the <code>hardware.nvidia.powerManagement.enable</code> option is enabled, the driver saves video memory to <code>/tmp</code> by default. If <code>/tmp</code> is backed by tmpfs (RAM) and the GPU VRAM usage exceeds the available space, the system will not resume and you will see a blank screen instead. | |||
To resolve this, redirect the temporary file to a storage location with sufficient capacity (e.g., <code>/var/tmp</code>) using kernel parameters: | |||
{{file|configuration.nix|nix|<nowiki> | |||
boot.kernelParams = [ "nvidia.NVreg_TemporaryFilePath=/var/tmp" ]; | |||
</nowiki>}} | |||
}} | |||
If you have a modern NVIDIA GPU (Turing [https://en.wikipedia.org/wiki/Turing_(microarchitecture)#Products_using_Turing] or later), you may also want to investigate the <code>hardware.nvidia.powerManagement.finegrained</code> option: [https://download.nvidia.com/XFree86/Linux-x86_64/460.73.01/README/dynamicpowermanagement.html] | |||
[https://discourse.nixos.org/t/suspend-resume-cycling-on-system-resume/32322/12 A potential fix] that Interrupts the gnome-shell in time so it’s not trying to access the graphics hardware. <ref>https://discourse.nixos.org/t/suspend-resume-cycling-on-system-resume/32322/12</ref> The entire purpose is to manually "pause" the GNOME Shell process just before the system sleeps and "un-pause" it just after the system wakes up. | |||
<hr> | |||
If you have graphical corruption upon waking from suspend, and the above causes the system to go back to sleep ~20-30 seconds after wakeup, the following may solve both issues: | |||
{{File|3={ | |||
# https://discourse.nixos.org/t/black-screen-after-suspend-hibernate-with-nvidia/54341/6 | |||
# https://discourse.nixos.org/t/suspend-problem/54033/28 | |||
systemd = { | |||
# Uncertain if this is still required or not. | |||
services.systemd-suspend.environment.SYSTEMD_SLEEP_FREEZE_USER_SESSIONS = "false"; | |||
services."gnome-suspend" = { | |||
description = "suspend gnome shell"; | |||
before = [ | |||
"systemd-suspend.service" | |||
"systemd-hibernate.service" | |||
"nvidia-suspend.service" | |||
"nvidia-hibernate.service" | |||
]; | |||
wantedBy = [ | |||
"systemd-suspend.service" | |||
"systemd-hibernate.service" | |||
]; | |||
serviceConfig = { | |||
Type = "oneshot"; | |||
ExecStart = ''${pkgs.procps}/bin/pkill -f -STOP ${pkgs.gnome-shell}/bin/gnome-shell''; | |||
}; | |||
}; | |||
services."gnome-resume" = { | |||
description = "resume gnome shell"; | |||
after = [ | |||
"systemd-suspend.service" | |||
"systemd-hibernate.service" | |||
"nvidia-resume.service" | |||
]; | |||
wantedBy = [ | |||
"systemd-suspend.service" | |||
"systemd-hibernate.service" | |||
]; | |||
serviceConfig = { | |||
Type = "oneshot"; | |||
ExecStart = ''${pkgs.procps}/bin/pkill -f -CONT ${pkgs.gnome-shell}/bin/gnome-shell''; | |||
}; | |||
}; | |||
}; | |||
# https://discourse.nixos.org/t/black-screen-after-suspend-hibernate-with-nvidia/54341/23 | |||
hardware.nvidia.powerManagement.enable = true; | |||
}|name=configuration.nix|lang=nix}} | |||
=== Black screen or 'nothing works' on laptops === | === Black screen or 'nothing works' on laptops === | ||
The kernel module<code>i915</code>for Intel or<code>amdgpu</code>for AMD may interfere with the | The kernel module<code>i915</code>for Intel or<code>amdgpu</code>for AMD may interfere with the NVIDIA driver. This may result in a black screen when switching to the virtual terminal, or when exiting the X session. A possible workaround is to disable the integrated GPU by blacklisting the module, using the following configuration option (see also [https://discourse.nixos.org/t/nvidia-gpu-and-i915-kernel-module/21307/3]): | ||
<syntaxHighlight lang="nix"> | <syntaxHighlight lang="nix"> | ||
| Line 241: | Line 341: | ||
boot.kernelParams = [ "module_blacklist=amdgpu" ]; | boot.kernelParams = [ "module_blacklist=amdgpu" ]; | ||
</syntaxHighlight> | </syntaxHighlight> | ||
=== NVIDIA Docker Containers === | |||
See: [[Docker#NVIDIA Docker Containers]] | |||
== Disabling == | == Disabling == | ||
| Line 258: | Line 362: | ||
=== Nouveau === | === Nouveau === | ||
Nouveau can be disabled by blacklisting the <code>nouveau</code> kernel module:<syntaxhighlight lang="nix"> | Nouveau can be disabled by blacklisting the <code>nouveau</code> kernel module: | ||
Note: This is done by default when using proprietary drivers<syntaxhighlight lang="nix"> | |||
{ | { | ||
boot.blacklistedKernelModules = [ "nouveau" ]; | boot.blacklistedKernelModules = [ "nouveau" ]; | ||
} | } | ||
</syntaxhighlight>Note that disabling both NVIDIA kernel modules and Nouveau effectively disables the GPU entirely. | </syntaxhighlight>Note that disabling both NVIDIA kernel modules and Nouveau effectively disables the GPU entirely. | ||
== Footnotes == | |||
[[Category:Video]] | [[Category:Video]] | ||
<references />4. https://discourse.nixos.org/t/nvidia-open-breaks-hardware-acceleration/58770/1 | |||