CUDA: Difference between revisions

From NixOS Wiki
imported>Samuela
Created page with "NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud insta..."
 
Razushi (talk | contribs)
→‎cudatoolkit, cudnn, and related packages: Updated broken links to cuDNN and cuTENSOR, please verify. Update date refers to the last verification, not example updates.
 
(17 intermediate revisions by 11 users not shown)
Line 1: Line 1:
NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud instances. These use cases are supported by the [https://github.com/orgs/NixOS/teams/cuda-maintainers @NixOS/cuda-maintainers team] on GitHub. If you have an issue using your NVIDIA GPU for computing purposes [https://github.com/nixos/nixpkgs/issues/new open an issue] on GitHub and tag @NixOS/cuda-maintainers.
NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud instances. These use cases are supported by the [https://github.com/orgs/NixOS/projects/27 @NixOS/cuda-maintainers team] on GitHub. If you have an issue using your NVIDIA GPU for computing purposes [https://github.com/nixos/nixpkgs/issues/new open an issue] on GitHub and tag @NixOS/cuda-maintainers.


Note that you may need to adjust your driver version to use "data center" GPUs like V100/A100s. See [https://discourse.nixos.org/t/how-to-use-nvidia-v100-a100-gpus/17754 this thread] for more info.
{{tip|1='''Cache''': Using the [https://app.cachix.org/cache/cuda-maintainers#pull cuda-maintainers cache] is recommended! It will save you valuable time and electrons. Getting set up should be as simple as <code>cachix use cuda-maintainers</code>. Click [[#Setting up CUDA Binary Cache|here]] for more details.}}


== CUDA (cudatoolkit) ==
{{tip|1='''Data center GPUs''': Note that you may need to adjust your driver version to use "data center" GPUs like V100/A100s. See [https://discourse.nixos.org/t/how-to-use-nvidia-v100-a100-gpus/17754 this thread] for more info.}}


The CUDA toolkit is available in a [https://search.nixos.org/packages?channel=unstable&from=0&size=50&buckets=%7B%22package_attr_set%22%3A%5B%22cudaPackages%22%5D%2C%22package_license_set%22%3A%5B%5D%2C%22package_maintainers_set%22%3A%5B%5D%2C%22package_platforms%22%3A%5B%5D%7D&sort=relevance&type=packages&query=cudatoolkit number of different versions]. Please use the latest major version. You can see where they're defined in nixpkgs [https://github.com/NixOS/nixpkgs/blob/4d5b1d6b273fc4acd5dce966d2e9c0ca197b6df2/pkgs/development/compilers/cudatoolkit/default.nix here].
== <code>cudatoolkit</code>, <code>cudnn</code>, and related packages ==
 
The CUDA toolkit is available in a [https://search.nixos.org/packages?channel=unstable&from=0&size=50&buckets=%7B%22package_attr_set%22%3A%5B%22cudaPackages%22%5D%2C%22package_license_set%22%3A%5B%5D%2C%22package_maintainers_set%22%3A%5B%5D%2C%22package_platforms%22%3A%5B%5D%7D&sort=relevance&type=packages&query=cudatoolkit number of different versions]. Please use the latest major version. You can see where they're defined in nixpkgs [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/cudatoolkit/releases.nix here].


Several "CUDA-X" libraries are packages as well. In particular,
Several "CUDA-X" libraries are packages as well. In particular,
* cuDNN is packaged [https://github.com/NixOS/nixpkgs/blob/634141959076a8ab69ca2cca0f266852256d79ee/pkgs/development/libraries/science/math/cudnn/default.nix here].
* cuDNN is packaged [https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/cuda-modules/cudnn here].
* cuTENSOR is packaged [https://github.com/NixOS/nixpkgs/blob/634141959076a8ab69ca2cca0f266852256d79ee/pkgs/development/libraries/science/math/cutensor/default.nix here].
* cuTENSOR is packaged [https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/cuda-modules/cutensor here].


'''Note that these examples haven't been updated in a while (as of 2022-03-12). May not be the best solution. A better resource is likely the packaging CUDA sample code [https://github.com/NixOS/nixpkgs/blob/634141959076a8ab69ca2cca0f266852256d79ee/pkgs/test/cuda/cuda-library-samples/generic.nix here].'''
{{warning|1=Note that these examples have been updated more recently (as of 2024-07-30). May not be the best solution. A better resource is likely the packaging CUDA sample code [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/cuda-library-samples/generic.nix here].}}


There some possible ways to setup a development environment using CUDA on NixOS. This can accomplished in the following ways:
There are some possible ways to setup a development environment using CUDA on NixOS. This can be accomplished in the following ways:


* By making a FHS user env
* By making a FHS user env


{{file|cuda-fsh.nix|nix|<nowiki>
{{file|cuda-fhs.nix|nix|<nowiki>
{ pkgs ? import <nixpkgs> {} }:
# Run with `nix-shell cuda-fhs.nix`
 
{ pkgs ? import </nowiki><nixpkgs><nowiki> {} }:
let
(pkgs.buildFHSUserEnv {
  fhs = pkgs.buildFHSUserEnv {
  name = "cuda-env";
    name = "cuda-env";
  targetPkgs = pkgs: with pkgs; [  
    targetPkgs = pkgs: with pkgs; [  
    git
      git
    gitRepo
      gitRepo
    gnupg
      gnupg
    autoconf
      autoconf
    curl
      curl
    procps
      procps
    gnumake
      gnumake
    util-linux
      utillinux
    m4
      m4
    gperf
      gperf
    unzip
      unzip
    cudatoolkit
      cudatoolkit
    linuxPackages.nvidia_x11
      linuxPackages.nvidia_x11
    libGLU libGL
      libGLU libGL
    xorg.libXi xorg.libXmu freeglut
      xorg.libXi xorg.libXmu freeglut
    xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib  
      xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib  
    ncurses5
      ncurses5
    stdenv.cc
      stdenv.cc
    binutils
      binutils
  ];
    ];
  multiPkgs = pkgs: with pkgs; [ zlib ];
    multiPkgs = pkgs: with pkgs; [ zlib ];
  runScript = "bash";
    runScript = "bash";
  profile = ''
    profile = ''
    export CUDA_PATH=${pkgs.cudatoolkit}
      export CUDA_PATH=${pkgs.cudatoolkit}
    # export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib
      # export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib
    export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
      export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
    export EXTRA_CCFLAGS="-I/usr/include"
      export EXTRA_CCFLAGS="-I/usr/include"
  '';
    '';
}).env
  };
in pkgs.stdenv.mkDerivation {
  name = "cuda-env-shell";
  nativeBuildInputs = [ fhs ];
  shellHook = "exec cuda-env";
}
</nowiki>}}
</nowiki>}}


Line 63: Line 59:
* By making a nix-shell
* By making a nix-shell
{{file|cuda-shell.nix|nix|<nowiki>
{{file|cuda-shell.nix|nix|<nowiki>
{ pkgs ? import <nixpkgs> {} }:
# Run with `nix-shell cuda-shell.nix`
 
{ pkgs ? import </nowiki><nixpkgs><nowiki> {} }:
pkgs.stdenv.mkDerivation {
pkgs.mkShell {
   name = "cuda-env-shell";
   name = "cuda-env-shell";
   buildInputs = with pkgs; [
   buildInputs = with pkgs; [
     git gitRepo gnupg autoconf curl
     git gitRepo gnupg autoconf curl
     procps gnumake utillinux m4 gperf unzip
     procps gnumake util-linux m4 gperf unzip
     cudatoolkit linuxPackages.nvidia_x11
     cudatoolkit linuxPackages.nvidia_x11
     libGLU libGL
     libGLU libGL
Line 83: Line 79:
   '';           
   '';           
}
}
</nowiki>}}
== Setting up CUDA Binary Cache ==
The [https://app.cachix.org/cache/cuda-maintainers cuda-maintainers] cache contains pre-built CUDA packages. By adding it to your system, Nix will fetch these packages instead of building them, saving valuable time and processing power.
For more information, refer to the [[Binary Cache#Using a binary cache Using a binary cache|Using a binary cache]] page.
{{warning|1=You need to rebuild your system at least once after adding the cache, before it can be used.}}
=== NixOS ===
Add the cache to <code>substituters</code> and <code>trusted-public-keys</code> inside your system configuration:
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
nix.settings = {
  substituters = [
    "https://cuda-maintainers.cachix.org"
  ];
  trusted-public-keys = [
    "cuda-maintainers.cachix.org-1:0dq3bujKpuEPMCX6U4WylrUDZ9JyUG0VpVZa7CNfq5E="
  ];
};
</nowiki>}}
=== Non-NixOS ===
If you have [https://www.cachix.org cachix] installed and set up, all you need to do is run:
<syntaxHighlight lang="console">
$ cachix use cuda-maintainers
</syntaxHighlight>
Else, you have to add <code>substituters</code> and <code>trusted-public-keys</code> to <code>/etc/nix/nix.conf</code>:
{{file|/etc/nix/nix.conf|nix|<nowiki>
trusted-public-keys = cuda-maintainers.cachix.org-1:0dq3bujKpuEPMCX6U4WylrUDZ9JyUG0VpVZa7CNfq5E=
trusted-substituters = https://cuda-maintainers.cachix.org
trusted-users = root @wheel
</nowiki>}}
If your user is in <code>trusted-users</code>, you can also add the cache in your home directory:
{{file|~/.config/nix/nix.conf|nix|<nowiki>
substituters = https://cuda-maintainers.cachix.org
</nowiki>}}
== Some things to keep in mind when setting up CUDA in NixOS ==
* Some GPUs, like Tesla K80, don't work with the latest drivers, so you must specify them in the option <code>hardware.nvidia.package</code> getting the value from your selected kernel, for example, <code>config.boot.kernelPackages.nvidia_x11_legacy470</code>. You can check which driver version your GPU supports by visiting the  [https://www.nvidia.com/Download/index.aspx nvidia site] and checking the driver version.
* Even with the drivers correctly installed, some software, like Blender, may not see the CUDA GPU. Make sure your system configuration has the option <code>hardware.opengl.enable</code> enabled.
* By default, software packaged in source code form has CUDA support disabled, because of the unfree license. To solve this, you can enable builds with CUDA support with a nixpkgs wide configuration, or use binary packaged CUDA compatible software such as [https://github.com/edolstra/nix-warez/tree/master/blender blender-bin].
== CUDA under WSL ==
This (surprisingly) works just fine using nixpkgs 23.05 provided that you prefix the <code>LD_LIBRARY_PATH</code> in your interactive environment with the WSL library directory. For nix shell this looks like:
{{file|cuda-shell.nix|nix|<nowiki>
  shellHook = ''
      export CUDA_PATH=${pkgs.cudatoolkit}
      export LD_LIBRARY_PATH=/usr/lib/wsl/lib:${pkgs.linuxPackages.nvidia_x11}/lib:${pkgs.ncurses5}/lib
      export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
      export EXTRA_CCFLAGS="-I/usr/include"
  '';         
</nowiki>}}
</nowiki>}}


Line 91: Line 150:
* [https://discourse.nixos.org/t/cuda-setup-on-nixos/1118 CUDA setup on NixOS]
* [https://discourse.nixos.org/t/cuda-setup-on-nixos/1118 CUDA setup on NixOS]
* [https://github.com/NixOS/nixpkgs/issues/131608 eGPU with nvidia-docker on intel-xserver]
* [https://github.com/NixOS/nixpkgs/issues/131608 eGPU with nvidia-docker on intel-xserver]
* [https://discourse.nixos.org/t/cuda-in-nixos-on-gcp-for-a-tesla-k80/ Tesla K80 based CUDA setup with Terraform on GCP]
[[Category:Server]]

Latest revision as of 13:01, 30 July 2024

NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud instances. These use cases are supported by the @NixOS/cuda-maintainers team on GitHub. If you have an issue using your NVIDIA GPU for computing purposes open an issue on GitHub and tag @NixOS/cuda-maintainers.

cudatoolkit, cudnn, and related packages

The CUDA toolkit is available in a number of different versions. Please use the latest major version. You can see where they're defined in nixpkgs here.

Several "CUDA-X" libraries are packages as well. In particular,

  • cuDNN is packaged here.
  • cuTENSOR is packaged here.
Warning: Note that these examples have been updated more recently (as of 2024-07-30). May not be the best solution. A better resource is likely the packaging CUDA sample code here.

There are some possible ways to setup a development environment using CUDA on NixOS. This can be accomplished in the following ways:

  • By making a FHS user env
cuda-fhs.nix
# Run with `nix-shell cuda-fhs.nix`
{ pkgs ? import <nixpkgs> {} }:
(pkgs.buildFHSUserEnv {
  name = "cuda-env";
  targetPkgs = pkgs: with pkgs; [ 
    git
    gitRepo
    gnupg
    autoconf
    curl
    procps
    gnumake
    util-linux
    m4
    gperf
    unzip
    cudatoolkit
    linuxPackages.nvidia_x11
    libGLU libGL
    xorg.libXi xorg.libXmu freeglut
    xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib 
    ncurses5
    stdenv.cc
    binutils
  ];
  multiPkgs = pkgs: with pkgs; [ zlib ];
  runScript = "bash";
  profile = ''
    export CUDA_PATH=${pkgs.cudatoolkit}
    # export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib
    export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
    export EXTRA_CCFLAGS="-I/usr/include"
  '';
}).env


  • By making a nix-shell
cuda-shell.nix
# Run with `nix-shell cuda-shell.nix`
{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
   name = "cuda-env-shell";
   buildInputs = with pkgs; [
     git gitRepo gnupg autoconf curl
     procps gnumake util-linux m4 gperf unzip
     cudatoolkit linuxPackages.nvidia_x11
     libGLU libGL
     xorg.libXi xorg.libXmu freeglut
     xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib 
     ncurses5 stdenv.cc binutils
   ];
   shellHook = ''
      export CUDA_PATH=${pkgs.cudatoolkit}
      # export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib:${pkgs.ncurses5}/lib
      export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
      export EXTRA_CCFLAGS="-I/usr/include"
   '';          
}

Setting up CUDA Binary Cache

The cuda-maintainers cache contains pre-built CUDA packages. By adding it to your system, Nix will fetch these packages instead of building them, saving valuable time and processing power.

For more information, refer to the Using a binary cache page.

Warning: You need to rebuild your system at least once after adding the cache, before it can be used.

NixOS

Add the cache to substituters and trusted-public-keys inside your system configuration:

/etc/nixos/configuration.nix
nix.settings = {
  substituters = [
    "https://cuda-maintainers.cachix.org"
  ];
  trusted-public-keys = [
    "cuda-maintainers.cachix.org-1:0dq3bujKpuEPMCX6U4WylrUDZ9JyUG0VpVZa7CNfq5E="
  ];
};

Non-NixOS

If you have cachix installed and set up, all you need to do is run:

$ cachix use cuda-maintainers

Else, you have to add substituters and trusted-public-keys to /etc/nix/nix.conf:

/etc/nix/nix.conf
trusted-public-keys = cuda-maintainers.cachix.org-1:0dq3bujKpuEPMCX6U4WylrUDZ9JyUG0VpVZa7CNfq5E=
trusted-substituters = https://cuda-maintainers.cachix.org
trusted-users = root @wheel

If your user is in trusted-users, you can also add the cache in your home directory:

~/.config/nix/nix.conf
substituters = https://cuda-maintainers.cachix.org

Some things to keep in mind when setting up CUDA in NixOS

  • Some GPUs, like Tesla K80, don't work with the latest drivers, so you must specify them in the option hardware.nvidia.package getting the value from your selected kernel, for example, config.boot.kernelPackages.nvidia_x11_legacy470. You can check which driver version your GPU supports by visiting the nvidia site and checking the driver version.
  • Even with the drivers correctly installed, some software, like Blender, may not see the CUDA GPU. Make sure your system configuration has the option hardware.opengl.enable enabled.
  • By default, software packaged in source code form has CUDA support disabled, because of the unfree license. To solve this, you can enable builds with CUDA support with a nixpkgs wide configuration, or use binary packaged CUDA compatible software such as blender-bin.

CUDA under WSL

This (surprisingly) works just fine using nixpkgs 23.05 provided that you prefix the LD_LIBRARY_PATH in your interactive environment with the WSL library directory. For nix shell this looks like:

cuda-shell.nix
   shellHook = ''
      export CUDA_PATH=${pkgs.cudatoolkit}
      export LD_LIBRARY_PATH=/usr/lib/wsl/lib:${pkgs.linuxPackages.nvidia_x11}/lib:${pkgs.ncurses5}/lib
      export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
      export EXTRA_CCFLAGS="-I/usr/include"
   '';

See also