NVIDIA

From NixOS Wiki
Revision as of 16:18, 8 July 2023 by imported>Jmq (Added imports line to explicitly show 'lib' is required.)

Installing NVIDIA official drivers on NixOS

If you're using NixOS, installing and using the official NVIDIA drivers is as simple as, 

/etc/nixos/configuration.nix

{ config, lib, pkgs, ... }
{

  # Make sure opengl is enabled
  hardware.opengl = {
    enable = true;
    driSupport = true;
    driSupport32Bit = true;
  };

  # NVIDIA drivers are unfree.
  nixpkgs.config.allowUnfreePredicate = pkg:
    builtins.elem (lib.getName pkg) [
      "nvidia-x11"
    ];

  # Tell Xorg to use the nvidia driver
  services.xserver.videoDrivers = ["nvidia"];

  hardware.nvidia = {

    # Modesetting is needed for most wayland compositors
    modesetting.enable = true;

    # Use the open source version of the kernel module
    # Only available on driver 515.43.04+
    open = true;

    # Enable the nvidia settings menu
    nvidiaSettings = true;

    # Optionally, you may need to select the appropriate driver version for your specific GPU.
    package = config.boot.kernelPackages.nvidiaPackages.stable;
  };
  ...
}

As noted in the final comment, you'll need to determine the appropriate driver version for your card. For "legacy" cards, you can consult nvidia official legacy driver list. You can consult the set of possible options in the source here.

(The services.xserver.videoDrivers setting is also valid for wayland installations despite it's name.)

Using GPUs on non-NixOS

If you're using Nix-packaged software on a non-NixOS system, you'll need a workaround to get everything up-and-running. The nixGL project provides wrapper to use GL drivers on non-NixOS systems. You need to have GPU drivers installed on your distro (for kernel modules). With nixGL installed, you'll run nixGL foobar instead of foobar.

Note that nixGL is not specific to NVIDIA GPUs, and should work with just about any GPU.

CUDA and using your GPU for compute

See the CUDA wiki page!

Using your GPU for graphics

Please note that, if you are setting up PRIME offloading, you must set the single value of "nvidia" even though it would be more conceptually correct to also include the driver for your other GPU. Doing otherwise will cause a broken xorg.conf to be generated. This is because NixOS doesn't actually handle multiple GPUs / GPU drivers properly, as per https://github.com/NixOS/nixpkgs/issues/108018.

Determining the type of your GPU

  • MXM / output-providing card (shows as VGA Controller in lspci), i.e. graphics card in desktop computer or in some laptops
  • muxless/non-MXM Optimus cards have no display outputs and show as 3D Controller in lspci output, seen in most modern consumer laptops

MXM cards allow you to use the Nvidia card standalone, in Non-Optimus mode. Non-MXM cards require Optimus, Nvidia's integrated-vs-discrete GPU switching technology.

Non-Optimus mode

You need an MXM card (see above) for Non-Optimus mode. Follow NVIDIA Graphics Cards section in official manual.

In case of laptop you may also need to use a BIOS option to select which card to use for the internal display.

Optimus

Mostly useful for laptops. There are currently two solutions available under NixOS, described in detail below:

  • Official solution: Nvidia PRIME (in on-demand "offload" mode, and always-on "sync" mode)
  • Previous open-source solution: Bumblebee (now deprecated)

Nvidia PRIME

The Official solution by nvidia.

offload mode

Available since 20.09 (see #66601).

In this mode the Nvidia card is only activated on demand, however a Nvidia card of the Turing generation or newer and an Intel Coffee Lake chipset is required for a complete poweroff of the Nvidia card (see discussion).

Offload mode is enabled by running your program(s) with specific environment variables, i.e., here's a sample script called nvidia-offload that you can run wrapped around your exacutable, for example nvidia-offload glxgears: 

nvidia-offload

export __NV_PRIME_RENDER_OFFLOAD=1
export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0
export __GLX_VENDOR_LIBRARY_NAME=nvidia
export __VK_LAYER_NV_optimus=NVIDIA_only
exec "$@"

To configure Offload mode, you firstly you need to enable the proprietary Nvidia driver: 

/etc/nixos/configuration.nix

{
  services.xserver.videoDrivers = [ "nvidia" ];
  ...

Note that on certain laptops and/or if you are using a custom kernel version, you may have issues with your NixOS system finding the primary display. In this case you should use hardware.nvidia.modesetting.enable, i.e.: 

/etc/nixos/configuration.nix

 	
{
  hardware.nvidia.modesetting.enable = true;
  services.xserver.videoDrivers = [ "nvidia" ];
  // ...

Then you need to setup the Bus ID's of the cards as seen below.

Note: Bus ID is important and needs to be formatted properly

The Nvidia driver expects the bus ID to be in decimal format; There are two ways you can get the bus IDs, one is with lspci, which shows the bus IDs in hexadecimal format and the other with lshw, which shows it in decimal format, as wanted by nixos.

lspci

You can convert the value by:

  • Stripping any leading zeros from the bus numbers or if the number is above 09, convert it to decimal and use that value.
  • Replacing any full stops with colons.
  • Prefix the final value with "PCI".

For example:

Output from lspci

09:1f.0 VGA compatible controller: NVIDIA Corporation Device 1f91 (rev a1)

Converted and correct format

PCI:9:31:0

lshw

If you don't have lshw installed, you can get it temporarily in a ephemeral shell by running:

nix-shell -p lshw --run "lshw -c display"

The two bus ID's will be in the corresponding "bus info" field.

For example:

bus info: pci@0000:01:00.0

and

bus info: pci@0000:00:02.0

Now you have to take everything after the first colon, and replace the . with another colon.

01:00:0

and

00:02:0

A possible configuration: 

/etc/nixos/configuration.nix

{
  services.xserver.videoDrivers = ["nvidia"];

  hardware.nvidia.prime = {
    offload = {
      enable = true;
      enableOffloadCmd = true;
    };

    intelBusId = "PCI:0:2:0";
    nvidiaBusId = "PCI:1:0:0";
  };
}

booting with an external display

Most Optimus laptops have the HDMI port for an external display wired directly to the Nvidia chip, in which case you need a configuration to use the Nvidia driver directly without offload mode. Fortunately, NixOS has an amazing feature called specialisations which allows you to do this easily. Here is an example configuration: 

/etc/nixos/configuration.nix

  specialisation = {
    external-display.configuration = {
      system.nixos.tags = [ "external-display" ];
      hardware.nvidia = {
        prime.offload.enable = lib.mkForce false;
        powerManagement.enable = lib.mkForce false;
      };
    };
  };

Once you rebuild your configuration, an extra external-display configuration will be built and placed in your boot menu.

To use this, boot your laptop with the lid open, choose the external-display configuration in the boot menu, and continue to keep the lid open until your desktop appears on the external display. At this point you can close the lid.

offloading steam

First, add this to your ~/.bashrc : 

export XDG_DATA_HOME="$HOME/.local/share"

.

For NixOS Steam run: 

mkdir -p ~/.local/share/applications
sed 's/^Exec=/&nvidia-offload /' /run/current-system/sw/share/applications/steam.desktop > ~/.local/share/applications/steam.desktop

.

For Flatpak Steam run: 

mkdir -p ~/.local/share/applications
sed 's/^Exec=/&nvidia-offload /' /var/lib/flatpak/exports/share/applications/com.valvesoftware.Steam.desktop > ~/.local/share/applications/com.valvesoftware.steam.desktop

.

Then restart your graphical environment session.

sync mode

In this mode the Nvidia card is turned on constantly, having impact on laptop battery and health.

Possible issues:

  • Hangs of applications after resume from suspend
  • Wrong DPI calculation (in this case provide dpi manually services.xserver.dpi = 96;)
  • Black screen after system upgrade (e.g. nixos-rebuild switch; use nixos-rebuild boot instead and reboot)
  • No video playback acceleration available (vaapi)
/etc/nixos/configuration.nix

{
  hardware.nvidia.modesetting.enable = true;
  services.xserver.videoDrivers = [ "nvidia" ];
  hardware.nvidia.prime = {
    sync.enable = true;

    nvidiaBusId = "PCI:1:0:0";
    intelBusId = "PCI:0:2:0";
  };
}

reverse sync mode

Warning: This feature is relatively new, depending on your system this might work poorly. AMD support, especially so. See: https://forums.developer.nvidia.com/t/the-all-new-outputsink-feature-aka-reverse-prime/129828

Only works with services.xserver.displayManager.setupCommands compatible Display Managers (LightDM, GDM and SDDM). 

/etc/nixos/configuration.nix

{
  services.xserver.videoDrivers = ["nvidia"];

  hardware.nvidia = {
    # Modesetting should be enabled to prevent screen tearing
    modesetting.enable = true;

    # Reverse sync is not compatible with the open source kernel module
    open = false;

    prime = {
      reverseSync.enable = true;

      #enable if using an external GPU
      allowExternalGpu = false;

      intelBusId = "PCI:0:2:0";

      nvidiaBusId = "PCI:1:0:0";
    };
  };
}

Bumblebee

Deprecated solution. You should use offload mode instead.

Use option 

hardware.bumblebee.enable = true;


Troubleshooting

Booting to Terminal

If you encounter the Problem of booting to The Terminal you might try adding the nvidia kernel modul manually with: 

boot.initrd.kernelModules = [ "nvidia" ];
boot.extraModulePackages = [ config.boot.kernelPackages.nvidia_x11 ];

Fix screen tearing

You may often incounter screen tearing or artifacts when using proprietary Nvidia drivers. You can fix that by forcing full composition pipeline.

Note: This has been reported to reduce the performance of some OpenGL applications and may produce issues in WebGL. It also drastically increases the time the driver needs to clock down after load.
/etc/nixos/configuration.nix

hardware.nvidia.forceFullCompositionPipeline = true;

Fix app flickering with Picom

~/.config/picom/picom.conf

unredir-if-possible = false;
backend = "xrender"; # try "glx" if xrender doesn't help
vsync = true;

Fix graphical corruption on suspend/resume

By default only a small portion of VRAM is saved when suspending the system [1]. This can cause graphical issues in some applications when resuming from suspend. To fix it, enable systemd-based suspend, which will save and restore all of VRAM: 

/etc/nixos/configuration.nix

hardware.nvidia.powerManagement.enable = true;

If you have a modern Nvidia GPU (Turing [2] or later), you may also want to investigate the hardware.nvidia.powerManagement.finegrained option: [3]

Fix black screen on a system with an integrated GPU

If you are using non-Optimus mode on a system with an integrated GPU, the kernel module i915 for intel or amdgpu for AMD may interfere with the NVIDIA driver. This may result in a black screen when switching to the virtual terminal, or when exiting the X session. A possible workaround is to disable the integrated GPU by blacklisting the module, using the following configuration option (see also [4]): 

# intel
boot.kernelParams = [ "module_blacklist=i915" ];
# AMD
boot.kernelParams = [ "module_blacklist=amdgpu" ];
Retrieved from "https://wiki.nixos.org/w/index.php?title=NVIDIA&oldid=5654"