NVIDIA: Difference between revisions
Loosetooth (talk | contribs) m Correct broken bullet points |
Completely rewrote the entire damn thing |
||
Line 1: | Line 1: | ||
This page attempts to cover everything related to the use of NVIDIA GPUs on NixOS. | |||
== Enabling == | |||
=== Kernel modules from NVIDIA === | |||
Kernel modules from NVIDIA offer better performance than other alternatives, but make the system unfree by requiring proprietary userspace libraries that can interface with the kernel modules. Users that want to have a fully free and open-source system should use [[Nvidia#Nouveau|Nouveau]] instead. | |||
To enable them, add <code>"nvidia"</code> to the list of enabled video drivers defined by the <code>services.xserver.videoDrivers</code> option. | |||
{{Note|<code>hardware.graphics.enable</code> was named <code>hardware.opengl.enable</code> '''until NixOS 24.11'''.}} | |||
{{Note|Since driver version 560, you also will need to decide whether to use the open-source or proprietary modules by setting the <code>hardware.nvidia.open</code> option to either <code>true</code> or <code>false</code> respectively.<br><br>Open-source kernel modules are preferred over and planned to steadily replace proprietary modules<ref>https://developer.nvidia.com/blog/nvidia-transitions-fully-towards-open-source-gpu-kernel-modules/</ref>, although they only support GPUs of the Turing architecture or newer (from GeForce RTX 20 series and GeForce GTX 16 series onwards). Data center GPUs starting from Grace Hopper or Blackwall must use open-source modules — proprietary modules are no longer supported.<br><br>''If you're using proprietary modules'', make sure to allow [[Unfree software|unfree software]]. The unfree NVIDIA packages include nvidia-x11, nvidia-settings, and nvidia-persistenced. | |||
}}{{file|configuration.nix|nix|<nowiki> | |||
{{file| | |||
{ | { | ||
hardware.graphics.enable = true; | hardware.graphics.enable = true; | ||
services.xserver.videoDrivers = [ "nvidia" ]; | |||
services.xserver.videoDrivers = ["nvidia"]; | hardware.nvidia.open = true; # Set to false for proprietary drivers | ||
} | |||
</nowiki>}}{{Warning|If you use a laptop with both dedicated and integrated GPUs, you need to [[Nvidia#Hybrid_graphics_with_PRIME|configure PRIME]] in order to make your dedicated NVIDIA GPU work properly with your integrated GPU. Your configuration '''may not work''' if you skip this step.}} | |||
==== Legacy branches ==== | |||
GPUs of the Kepler architecture or older (most GeForce 600/700/800M cards and older) are no longer supported by latest proprietary modules. Users of these GPUs must use legacy branches of proprietary modules that may still receive updates, as long as the GPUs themselves remain supported by NVIDIA. You can find which legacy branch you need to use by searching for your GPU model on [https://www.nvidia.com/en-us/drivers/unix/legacy-gpu/ NVIDIA's official legacy driver support list]. | |||
To use legacy branches, you need to set the <code>hardware.nvidia.package</code> option to a package set named <code>config.boot.kernelPackages.nvidiaPackages.legacy_<branch></code>.{{file|configuration.nix|nix|<nowiki> | |||
{ config, ... }: # <- required to get the packages used by the currently configured kernel, including drivers | |||
{ | |||
# Last version that supports Kepler GPUs | |||
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.legacy_470; | |||
} | |||
</nowiki>}}Nixpkgs does not endeavor to support all legacy branches since older, unmaintained legacy branches can become incompatible with newer kernel and X server versions, and at some point it becomes infeasible to patch them to cooperate with modern software. You can find the list of supported legacy branches under [https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/nvidia-x11/default.nix in the Nixpkgs repository]. | |||
==== Beta/production branches ==== | |||
By default, modules from the stable branch are used, which come from latest release available for the current architecture — while x86-64 and aarch64 systems follow the "New Feature Branch" releases as they remain actively developed, 32-bit x86 and ARM systems remain on the 390.xx branch as that is the last branch with 32-bit support. | |||
Instead of the stable branch, users can also switch to the beta branch, which has more new features and experimental changes at the risk of containing more bugs, or the production branch, a more conservative, well-tested stable version that is suitable for production use with minimum breakage, at the cost of being behind in terms of new features. | |||
Using the beta and production branches are similar to how one would use legacy branches:{{file|configuration.nix|nix|<nowiki> | |||
{ config, ... }: | |||
{ | |||
#hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.stable; # Default | |||
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.beta; | |||
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.production; | |||
} | |||
</nowiki>}} | |||
=== Nouveau === | |||
Nouveau is a set of free and open-source drivers for NVIDIA GPUs that provide 2D/3D acceleration for all NVIDIA GPUs. Its use is in general not recommended due to its considerably worse performance compared to NVIDIA's kernel modules, as it does not support reclocking (changing the GPU clock frequency on-demand) for many NVIDIA GPUs<ref>https://www.phoronix.com/forums/forum/linux-graphics-x-org-drivers/open-source-nvidia-linux-nouveau/998310-nouveau-persevered-in-2017-for-open-source-nvidia-but-2018-could-be-much-better#post998316</ref>. Nevertheless it remains a viable option for users who want a fully free and open-source operating system as it does not contain any proprietary components, unlike NVIDIA's kernel modules and userspace libraries. | |||
Nouveau is enabled by default whenever graphics are enabled, and does not need any extra configuration.{{file|configuration.nix|nix|<nowiki> | |||
{ | |||
hardware.graphics.enable = true; | |||
.. | |||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
=== | == Configuring == | ||
=== Power management === | |||
== | === Hybrid graphics with PRIME === | ||
Laptops often feature both an integrated GPU (iGPU) and a dedicated GPU (dGPU) in order to strive a balance between performance and power consumption — while the dGPU is used for performance-intensive tasks such as gaming, video editing, 3D rendering, compute jobs, etc., the iGPU can be used to render common 2D elements like application windows and the desktop environment. | |||
PRIME, therefore is a technology developed to facilitate the cooperation between the two GPUs and is critical for the laptop's graphical performance. Depending on your needs, you can configure PRIME in one of three modes, which have different tradeoffs in terms of performance and battery life. | |||
==== Common setup ==== | |||
All PRIME configurations require setting the PCI bus IDs of the two GPUs. One easy way to do find their IDs is by running <code>lspci</code> from the <code>pciutils</code> package and then finding devices that are classified as VGA controllers. After double checking that the listed devices are indeed your integrated and dedicated GPUs, you can then find the PCI IDs at the beginning of each line. Exact results may vary, but an example output might look like:<syntaxhighlight lang="console"> | |||
=== | $ nix shell nixpkgs#pciutils -c "lspci" | grep "VGA" | ||
0000:00:02.0 VGA compatible controller: Intel Corporation TigerLake-H GT1 [UHD Graphics] (rev 01) | |||
0000:01:00.0 VGA compatible controller: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] (rev a1) | |||
</syntaxhighlight>Before putting them into your configuration, however, '''they must first be reformatted''' — take the last three numbers, convert them from hexadecimal to decimal, remove the leading zeroes, concatenate them with colons, and then add a <code>PCI:</code> prefix. Then, they can be set under <code>intelBusId</code>, <code>nvidiaBusId</code>, or <code>amdgpuBusId</code> in <code>hardware.nvidia.prime</code>, depending on the manufacturer of the GPU:{{file|configuration.nix|nix|<nowiki> | |||
<syntaxhighlight lang=" | |||
</syntaxhighlight> | |||
{ | { | ||
hardware.nvidia.prime = { | |||
intelBusId = "PCI:0:2:0"; | |||
nvidiaBusId = "PCI:1:0:0"; | |||
#amdgpuBusId = "PCI:54:0:0"; # If you have an AMD iGPU | |||
}; | |||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
==== Offload mode ==== | ==== Offload mode ==== | ||
{{Note|Offload mode is available '''since NixOS 20.09 and NVIDIA driver version 435.21''', and requires an NVIDIA GPU of the Turing generation, or newer and a compatible CPU — either an Intel CPU from the Coffee Lake generation or newer, or an AMD Ryzen.}} | |||
Offload mode puts your dGPU to sleep and lets the iGPU handle all tasks, except if you call the dGPU specifically by "offloading" an application to it. For example, you can run your laptop normally and it will use the energy-efficient iGPU all day, and then you can offload a game from Steam onto the dGPU to make the dGPU run that game only. For many, this is the most desirable option. '''Offload mode is incompatible with sync mode.''' | |||
To enable offload mode, set the <code>hardware.nvidia.prime.offload.enable</code> option to <code>true</code>: | |||
{{file|configuration.nix|nix|<nowiki> | |||
{ | |||
hardware.nvidia.prime = { | |||
offload.enable = true; | |||
{{file|nvidia-offload|bash|<nowiki> | |||
intelBusId = "PCI:0:2:0"; | |||
nvidiaBusId = "PCI:1:0:0"; | |||
#amdgpuBusId = "PCI:54:0:0"; # If you have an AMD iGPU | |||
}; | |||
} | |||
</nowiki>}}When you want to run a program on the dGPU, you only need to set a few environment variables for the driver to recognize that offload mode should be used. If <code>hardware.nvidia.prime.offload.enableOffloadCmd</code> is set to true, NixOS will generate a wrapper script named <code>nvidia-offload</code> that sets the right variables for you:{{file|nvidia-offload|bash|<nowiki> | |||
export __NV_PRIME_RENDER_OFFLOAD=1 | export __NV_PRIME_RENDER_OFFLOAD=1 | ||
export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0 | export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0 | ||
Line 170: | Line 100: | ||
export __VK_LAYER_NV_optimus=NVIDIA_only | export __VK_LAYER_NV_optimus=NVIDIA_only | ||
exec "$@" | exec "$@" | ||
</nowiki>}} | </nowiki>}}If everything is configured correctly, then running a program like <code>glxgears</code> should use the iGPU, while running <code>nvidia-offload glxgears</code> should only use the dGPU. | ||
==== Sync mode ==== | ==== Sync mode ==== | ||
{{Note|Sync mode is available '''since NixOS 19.03 and NVIDIA driver version 390.67''', and is '''incompatible''' with both offload and reverse sync modes. Sync mode also requires using a desktop manager that respects the <code>services.xserver.displayManager.setupCommands</code> option, including LightDM, GDM and SDDM.}}In sync mode, rendering is completely delegated to the dGPU, while the iGPU only displays the rendered framebuffers copied from the dGPU. Sync mode offers better performance and greatly reduces screen tearing, at the expense of higher power consumption since the dGPU will not go to sleep when not needed, as is the case in offload mode. Sync mode may also solve some issues with connecting a display in clamshell mode directly to the GPU. | |||
To enable sync mode, set the <code>hardware.nvidia.prime.sync.enable</code> option to <code>true</code>: | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
Line 199: | Line 111: | ||
hardware.nvidia.prime = { | hardware.nvidia.prime = { | ||
sync.enable = true; | sync.enable = true; | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0:2:0"; | ||
# amdgpuBusId = "PCI:54:0:0"; | nvidiaBusId = "PCI:1:0:0"; | ||
#amdgpuBusId = "PCI:54:0:0"; # If you have an AMD iGPU | |||
}; | }; | ||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
==== Reverse sync mode | ==== Reverse sync mode ==== | ||
{{Note|Reverse sync mode is available '''since NixOS 23.05 and NVIDIA driver version 460.39''' and is still an experimental, buggy feature<ref>https://forums.developer.nvidia.com/t/the-all-new-outputsink-feature-aka-reverse-prime/129828/67</ref>. '''Your mileage may vary.''' Reverse sync mode also requires using a desktop manager that respects the <code>services.xserver.displayManager.setupCommands</code> option, including LightDM, GDM and SDDM.}}The difference between regular sync mode and reverse sync mode is that the '''dGPU''' is configured as the primary output device, allowing displaying to external displays wired to it and not the iGPU (more common). | |||
To enable sync mode, set the <code>hardware.nvidia.prime.reverseSync.enable</code> option to <code>true</code>: | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
Line 216: | Line 128: | ||
hardware.nvidia.prime = { | hardware.nvidia.prime = { | ||
reverseSync.enable = true; | reverseSync.enable = true; | ||
intelBusId = "PCI:0:2:0"; | intelBusId = "PCI:0:2:0"; | ||
nvidiaBusId = "PCI: | nvidiaBusId = "PCI:1:0:0"; | ||
# amdgpuBusId = "PCI:54:0:0"; | #amdgpuBusId = "PCI:54:0:0"; # If you have an AMD iGPU | ||
}; | }; | ||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
== | === Other settings === | ||
== Tips and tricks == | |||
=== Check nixos-hardware === | === Check nixos-hardware === | ||
Line 241: | Line 152: | ||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
{ | { | ||
specialisation.on-the-go.configuration = { | |||
specialisation | |||
system.nixos.tags = [ "on-the-go" ]; | system.nixos.tags = [ "on-the-go" ]; | ||
hardware.nvidia = { | hardware.nvidia.prime = { | ||
offload = { | |||
enable = lib.mkForce true; | |||
enableOffloadCmd = lib.mkForce true; | |||
}; | |||
sync.enable = lib.mkForce false; | |||
}; | }; | ||
}; | }; | ||
} | } | ||
</nowiki>}} | </nowiki>}} | ||
(You can also add other settings here totally unrelated to | (You can also add other settings here totally unrelated to NVIDIA, such as power profiles, etc.) | ||
After rebuilding and rebooting, you'll see in your boot menu under each Generation an "on-the-go" option, which will let you boot into the on-to-go specialisation for that generation. | After rebuilding and rebooting, you'll see in your boot menu under each Generation an "on-the-go" option, which will let you boot into the on-to-go specialisation for that generation. | ||
Line 263: | Line 172: | ||
If you're using Nix-packaged software on a non-NixOS system, you'll need a workaround to get everything up-and-running. The [https://github.com/guibou/nixGL nixGL project] provides wrapper to use GL drivers on non-NixOS systems. You need to have GPU drivers installed on your distro (for kernel modules). With nixGL installed, you'll run <code>nixGL foobar</code> instead of <code>foobar</code>. | If you're using Nix-packaged software on a non-NixOS system, you'll need a workaround to get everything up-and-running. The [https://github.com/guibou/nixGL nixGL project] provides wrapper to use GL drivers on non-NixOS systems. You need to have GPU drivers installed on your distro (for kernel modules). With nixGL installed, you'll run <code>nixGL foobar</code> instead of <code>foobar</code>. | ||
Note that nixGL is not specific to | Note that nixGL is not specific to NVIDIA GPUs, and should work with just about any GPU. | ||
=== CUDA and using your GPU for compute === | === CUDA and using your GPU for compute === | ||
See the [[CUDA]] wiki page. | See the [[CUDA]] wiki page. | ||
=== Running Specific NVIDIA Driver Versions === | === Running Specific NVIDIA Driver Versions === | ||
To run a specific version of the NVIDIA driver in NixOS, you can customize your configuration by specifying the desired version along with the corresponding SHA256 hashes. Below is an example configuration for using NVIDIA driver version | To run a specific version of the NVIDIA driver in NixOS, you can customize your configuration by specifying the desired version along with the corresponding SHA256 hashes. Below is an example configuration for using NVIDIA driver version <code>555.58.02</code>: | ||
{{file|/etc/nixos/nvidia.nix|nix|<nowiki> | {{file|/etc/nixos/nvidia.nix|nix|<nowiki> | ||
Line 307: | Line 194: | ||
In this configuration: | In this configuration: | ||
* Replace | * Replace <code>version</code> with the desired driver version. | ||
* Update the SHA256 hashes to match the new version you want to use. | * Update the SHA256 hashes to match the new version you want to use. | ||
* After updating the configuration, run <code>sudo nixos-rebuild switch</code> to apply the changes and load the specified NVIDIA driver version. | * After updating the configuration, run <code>sudo nixos-rebuild switch</code> to apply the changes and load the specified NVIDIA driver version. | ||
This allows you to pin the specific driver version being used in your NixOS installation. | This allows you to pin the specific driver version being used in your NixOS installation. | ||
== Troubleshooting == | == Troubleshooting == | ||
Line 414: | Line 242: | ||
</syntaxHighlight> | </syntaxHighlight> | ||
= | == Disabling == | ||
=== Kernel modules from NVIDIA === | |||
Normally, NVIDIA's kernel modules should be completely disabled by removing <code>"nvidia"</code> from <code>services.xserver.videoDrivers</code>. If that fails to work, you can also manually blacklist the corresponding kernel modules:<syntaxhighlight lang="nix"> | |||
{ | |||
boot.blacklistedKernelModules = [ | |||
"nvidia" | |||
"nvidiafb" | |||
"nvidia-drm" | |||
"nvidia-uvm" | |||
"nvidia-modeset" | |||
= | ]; | ||
} | |||
</syntaxhighlight> | |||
</ | |||
=== Nouveau === | |||
Nouveau can be disabled by blacklisting the <code>nouveau</code> kernel module:<syntaxhighlight lang="nix"> | |||
{ | |||
boot.blacklistedKernelModules = [ "nouveau" ]; | |||
} | |||
</syntaxhighlight>Note that disabling both NVIDIA kernel modules and Nouveau effectively disables the GPU entirely. | |||
[[Category:Video]] | [[Category:Video]] |