Official NixOS Wiki

CUDA: Difference between revisions

← Older edit

Barstoolbluz (talk | contribs)

1 edit
VisualWikitext
Bachp (talk | contribs)
7 edits
mNo edit summary
No edit summary
 
(16 intermediate revisions by 5 users not shown)
Line 1: Line 1:
NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud instances. These use cases are supported by the [https://github.com/orgs/NixOS/teams/cuda-maintainers @NixOS/cuda-maintainers team] on GitHub ([https://github.com/orgs/NixOS/projects/27 project board]). If you have an issue using your NVIDIA GPU for computing purposes [https://github.com/NixOS/nixpkgs/issues/new/choose open an issue] on GitHub and tag <code>@NixOS/cuda-maintainers</code>.
NixOS supports using NVIDIA GPUs for pure computing purposes, not just for graphics. For example, many users rely on NixOS for machine learning both locally and on cloud instances. These use cases are supported by the [https://github.com/orgs/NixOS/teams/cuda-maintainers @NixOS/cuda-maintainers team] on GitHub ([https://github.com/orgs/NixOS/projects/27 project board]). If you have an issue using your NVIDIA GPU for computing purposes [https://github.com/NixOS/nixpkgs/issues/new/choose open an issue] on GitHub and tag <code>@NixOS/cuda-maintainers</code>.


{{tip|1='''Cache''': Using the [https://app.cachix.org/cache/nix-community nix-community cache] is recommended! It will save you valuable time and electrons. Getting set up should be as simple as <code>cachix use nix-community</code>. Click [[#Setting up CUDA Binary Cache|here]] for more details.}}
{{tip|1='''Cache''': Using the binary cache is recommended! It will save you valuable time and electrons. Click [[#Setting up CUDA Binary Cache|here]] for more details.}}


{{tip|1='''Data center GPUs''': Note that you may need to adjust your driver version to use "data center" GPUs like V100/A100s. See [https://discourse.nixos.org/t/how-to-use-nvidia-v100-a100-gpus/17754 this thread] for more info.}}
{{tip|1='''Data center GPUs''': Note that you may need to adjust your driver version to use "data center" GPUs like V100/A100s. See [https://discourse.nixos.org/t/how-to-use-nvidia-v100-a100-gpus/17754 this thread] for more info.}}
== Driver Installation ==
Assuming you've followed the [[NVIDIA]] page correctly, and have a CUDA compatible GPU, you shouldn't need to do any further configuration. You can confirm your CUDA version by running the following command in your terminal.
<syntaxhighlight lang=console>
$ nvidia-smi | grep CUDA
</syntaxhighlight>


== <code>cudatoolkit</code>, <code>cudnn</code>, and related packages ==
== <code>cudatoolkit</code>, <code>cudnn</code>, and related packages ==
{{outdated|scope=section|date=July 2024|reason=Note that these examples have been updated more recently (as of 2024-07-30). May not be the best solution. A better resource is likely the packaging CUDA sample code [https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/cuda-modules/cutensor here].}}
{{outdated|scope=section|date=July 2024|reason=Note that these examples have been updated more recently (as of 2024-07-30). May not be the best solution. A better resource is likely the packaging CUDA sample code [https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/cuda-modules/cutensor here].}}
The CUDA toolkit is available in a [https://search.nixos.org/packages?channel=unstable&from=0&size=50&buckets=%7B%22package_attr_set%22%3A%5B%22cudaPackages%22%5D%2C%22package_license_set%22%3A%5B%5D%2C%22package_maintainers_set%22%3A%5B%5D%2C%22package_platforms%22%3A%5B%5D%7D&sort=relevance&type=packages&query=cudatoolkit number of different versions]. Please use the latest major version. You can see where they're defined in nixpkgs [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/cudatoolkit/releases.nix here].
The CUDA toolkit is available in a [https://search.nixos.org/packages?channel=unstable&from=0&size=50&buckets=%7B%22package_attr_set%22%3A%5B%22cudaPackages%22%5D%2C%22package_license_set%22%3A%5B%5D%2C%22package_maintainers_set%22%3A%5B%5D%2C%22package_platforms%22%3A%5B%5D%7D&sort=relevance&type=packages&query=cudatoolkit number of different versions]. Please use the latest major version. You can see where they're defined in nixpkgs [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/cudatoolkit/releases.nix here].


Line 17: Line 27:
* By making a FHS user env
* By making a FHS user env


<syntaxhighlight lang="nix" line="1" start="1"># flake.nix, run with `nix develop`
<syntaxhighlight lang="nix">
# flake.nix, run with `nix develop`
# Run with `nix-shell cuda-fhs.nix`
# Run with `nix-shell cuda-fhs.nix`
{ pkgs ? import </nowiki><nixpkgs><nowiki> {} }:
{ pkgs ? import <nixpkgs> {} }:
let
let
   # Change according to the driver used: stable, beta
   # Change according to the driver used: stable, beta
Line 26: Line 37:
(pkgs.buildFHSEnv {
(pkgs.buildFHSEnv {
   name = "cuda-env";
   name = "cuda-env";
   targetPkgs = pkgs: with pkgs; [  
   targetPkgs = pkgs: with pkgs; [
     git
     git
     gitRepo
     gitRepo
Line 42: Line 53:
     libGLU libGL
     libGLU libGL
     xorg.libXi xorg.libXmu freeglut
     xorg.libXi xorg.libXmu freeglut
     xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib  
     xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
     ncurses5
     ncurses5
     stdenv.cc
     stdenv.cc
Line 58: Line 69:
</syntaxhighlight>
</syntaxhighlight>


* By making a nix-shell


* By making a nix-shell
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix" line="1" start="1">
# flake.nix, run with `nix develop`
# flake.nix, run with `nix develop`# Run with `nix-shell cuda-shell.nix`
# Run with `nix-shell cuda-shell.nix`
{ pkgs ? import </nowiki><nixpkgs><nowiki> {} }:
{ pkgs ? import <nixpkgs> {} }:
let
let
   nvidiaPackage = pkgs.linuxPackages.nvidiaPackages.stable;
   nvidiaPackage = pkgs.linuxPackages.nvidiaPackages.stable;
Line 74: Line 86:
     libGLU libGL
     libGLU libGL
     xorg.libXi xorg.libXmu freeglut
     xorg.libXi xorg.libXmu freeglut
     xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib  
     xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
     ncurses5 stdenv.cc binutils
     ncurses5 stdenv.cc binutils
   ];
   ];
Line 82: Line 94:
       export EXTRA_LDFLAGS="-L/lib -L${nvidiaPackage}/lib"
       export EXTRA_LDFLAGS="-L/lib -L${nvidiaPackage}/lib"
       export EXTRA_CCFLAGS="-I/usr/include"
       export EXTRA_CCFLAGS="-I/usr/include"
   '';        
   '';
}
}
</syntaxhighlight>
</syntaxhighlight>


* By making a flake.nix
* By making a flake.nix
<syntaxhighlight lang="nix" line="1" start="1"># flake.nix, run with `nix develop`
 
<syntaxhighlight lang="nix"># flake.nix, run with `nix develop`
{
{
   description = "CUDA development environment";
   description = "CUDA development environment";
Line 144: Line 157:
== Setting up CUDA Binary Cache ==
== Setting up CUDA Binary Cache ==


The [https://nix-community.org/cache/ Nix-community cache] contains pre-built CUDA packages. By adding it to your system, Nix will fetch these packages instead of building them, saving valuable time and processing power.
CUDA binary caches provide pre-built CUDA packages. When a matching store path is available, Nix can download it instead of building the package locally, saving time and processing power.


For more information, refer to the [[Binary Cache#Using a binary cache Using a binary cache|Using a binary cache]] page.
For more information, refer to the [[Binary Cache#Using a binary cache Using a binary cache|Using a binary cache]] page.


{{warning|1=You need to rebuild your system at least once after adding the cache, before it can be used.}}
{{info|1=You need to rebuild your system at least once after adding the cache, before it can be used.}}
 
=== NixOS CUDA cache ===
 
==== NixOS ====


=== NixOS ===
Add the cache to <code>substituters</code> and <code>trusted-public-keys</code> inside your system configuration:
Add the cache to <code>substituters</code> and <code>trusted-public-keys</code> inside your system configuration:


{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{{file|3=<nowiki>
nix.settings = {
  substituters = [
    "https://cache.nixos-cuda.org"
  ];
  trusted-public-keys = [
    "cache.nixos-cuda.org:74DUi4Ye579gUqzH4ziL9IyiJBlDpMRn9MBN8oNan9M="
  ];
};
</nowiki>|name=/etc/nixos/configuration.nix|lang=nix}}
 
==== Non-NixOS ====
 
You have to add <code>substituters</code> and <code>trusted-public-keys</code> to <code>/etc/nix/nix.conf</code>:
 
{{file|3=<nowiki>
trusted-public-keys = cache.nixos-cuda.org:74DUi4Ye579gUqzH4ziL9IyiJBlDpMRn9MBN8oNan9M=
trusted-substituters = https://cache.nixos-cuda.org
trusted-users = root @wheel
</nowiki>|name=/etc/nix/nix.conf|lang=nix}}
 
If your user is in <code>trusted-users</code>, you can also add the cache in your home directory:
 
{{file|3=<nowiki>
trusted-public-keys = cache.nixos-cuda.org:74DUi4Ye579gUqzH4ziL9IyiJBlDpMRn9MBN8oNan9M=
trusted-substituters = https://cache.nixos-cuda.org
</nowiki>|name=~/.config/nix/nix.conf|lang=nix}}
 
=== Flox binary cache ===
 
NVIDIA CUDA packages are generally [[Unfree software|unfree]], so <code>cache.nixos.org</code> does not normally provide pre-built store paths for them. The same constraint applies to CUDA-enabled packages whose closures depend on proprietary NVIDIA software. This includes packages such as PyTorch and MAGMA, which, because of their large transitive closures, typically require considerable time and resources to build.
 
Flox is [https://discourse.nixos.org/t/nix-flox-nvidia-opening-up-cuda-redistribution-on-nix/69189 one of the providers] that NVIDIA works with to redistribute CUDA libraries and runtime packages. Flox also distributes PyTorch, MAGMA, and other CUDA-enabled packages whose Nix store closures include proprietary NVIDIA software. This agreement allows Flox to publish pre-built store paths that are not normally available from <code>cache.nixos.org</code>.
 
Flox builds the [https://github.com/nixos/nixpkgs/blob/master/pkgs/top-level/release-cuda.nix release-cuda jobset] and pushes the store paths to its public binary cache.
 
{| class="wikitable"
|-
! Cache URL
| <code>https://cache.flox.dev</code>
|-
! Public key
| <code>flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs=</code>
|}
 
Which store paths are available in the cache depends on which Nixpkgs revisions Flox builds against. Flox currently evaluates and builds CUDA packages against revisions from its own Nixpkgs fork at [https://github.com/flox/nixpkgs github.com/flox/nixpkgs]. This repo tracks the upstream <code>nixos-unstable</code> branch across four Flox-maintained branches (<code>unstable</code>, <code>staging</code>, <code>stable</code>, and <code>lts</code>). These branch names describe ''Flox's'' Nixpkgs update schedules, ''not'' the stability of the software they reference. In general, Flox's <code>unstable</code> branch tracks upstream <code>nixos-unstable</code> most closely, sampling and updating on a daily schedule.
 
Notably, Flox does not currently build against Nixpkgs [[Channel branches|release channels]], so users who pin to revisions from these channels are unlikely to get a cache hit from the Flox binary cache. Because different Nixpkgs revisions can produce different store paths for the same package, using a Nixpkgs revision that Flox builds against significantly increases the likelihood of a cache hit.
 
==== NixOS ====
 
{{file|3=<nowiki>
nix.settings = {
nix.settings = {
   substituters = [
   substituters = [
     "https://nix-community.cachix.org"
     "https://cache.flox.dev"
   ];
   ];
   trusted-public-keys = [
   trusted-public-keys = [
     "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
     "flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs="
   ];
   ];
};
};
</nowiki>}}
</nowiki>|name=/etc/nixos/configuration.nix|lang=nix}}
 
==== Non-NixOS ====
 
{{file|3=<nowiki>
extra-substituters = https://cache.flox.dev
extra-trusted-public-keys = flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs=
</nowiki>|name=/etc/nix/nix.conf|lang=nix}}
 
For more on configuring Nix to use the Flox binary cache, maximizing the likelihood of CUDA cache hits, and controlling which builds query the cache, see [https://flox.dev/docs/concepts/cuda-catalog-and-cache CUDA packages in the Flox Catalog and binary cache] in Flox's documentation.
 
{{note|The pre-built CUDA software in the Flox binary cache is based entirely on the work of the [https://nixos.org/community/teams/cuda/ Nixpkgs CUDA maintainers]. The CUDA team packages NVIDIA CUDA software for Nixpkgs and tests CUDA-enabled packages against CUDA dependencies to verify that they build and link correctly. CUDA-on-Nix would not be a reality without this team.}}
 
== Enabling CUDA In Packages ==
 
By default, software packaged in source code form has CUDA support disabled, because of the unfree license. There are multiple options to solve this.
 
You can enable builds with CUDA support with a nixpkgs wide configuration.
<syntaxhighlight lang="nix">
nixpkgs.config.cudaSupport = true;
</syntaxhighlight>


=== Non-NixOS ===
Or you can override individual packages.
<syntaxhighlight lang="nix">
environment.systemPackages = with pkgs; [
  (mlt.override {config.cudaSupport=true;})
];
</syntaxhighlight>


If you have [https://www.cachix.org cachix] installed and set up, all you need to do is run:
Or you can use binary-packaged versions of CUDA compatible software, such as [https://github.com/edolstra/nix-warez/tree/master/blender blender-bin] for Blender.


<syntaxHighlight lang="console">
{{info|If you will be using <code>cudaSupport</code> in packages, it is recommended you utilize a [[#Setting up CUDA Binary Cache|CUDA binary cache]].}}
$ cachix use nix-community
</syntaxHighlight>


Else, you have to add <code>substituters</code> and <code>trusted-public-keys</code> to <code>/etc/nix/nix.conf</code>:
Without a [[#Setting up CUDA Binary Cache|CUDA cache]], any CUDA compatible package installed with <code>cudaSupport</code> will be compiled from source. This is because NixOS Foundation does not build (and therefore [https://cache.nixos.org/ cache.nixos.org] does not cache) CUDA packages.


{{file|/etc/nix/nix.conf|nix|<nowiki>
For larger programs like Blender, that process can be very resource-intensive. If you are installing large CUDA-enabled package(s) that either are not cached or you are not using a cache, then (especially on older or weaker hardware) it is recommended to reduce the number of cores and/or jobs that the process will take, to prevent a system freeze from resource limits. This can be done with the <code>--max-jobs</code> / <code>-j</code> and <code>--cores</code> flags, for more details see the [https://github.com/NixOS/nix/blob/master/doc/manual/source/advanced-topics/cores-vs-jobs.md Tuning Cores & Jobs] manual page.
trusted-public-keys = nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
trusted-substituters = https://nix-community.cachix.org
trusted-users = root @wheel
</nowiki>}}


If your user is in <code>trusted-users</code>, you can also add the cache in your home directory:
If you don't want to deal with the increased time that compilation will take when <code>--max-jobs</code> / <code>-j</code> and <code>--cores</code> are set below maximum, you can also try simply closing other running processes to see if that frees up enough resources for compilation to be successful.


{{file|~/.config/nix/nix.conf|nix|<nowiki>
&rarr; For specifics on setting up Blender with CUDA (and OptiX) see: [[Blender#CUDA & OptiX]].
substituters = https://nix-community.cachix.org
</nowiki>}}


== Some things to keep in mind when setting up CUDA in NixOS ==
== Some things to keep in mind when setting up CUDA in NixOS ==
* Some GPUs, like Tesla K80, don't work with the latest drivers, so you must specify them in the option <code>hardware.nvidia.package</code> getting the value from your selected kernel, for example, <code>config.boot.kernelPackages.nvidia_x11_legacy470</code>. You can check which driver version your GPU supports by visiting the [https://www.nvidia.com/Download/index.aspx nvidia site] and checking the driver version.
 
* Even with the drivers correctly installed, some software, like Blender, may not see the CUDA GPU. Make sure your system configuration has the option <code>hardware.opengl.enable</code> enabled.
* Some GPUs, like Tesla K80, don't work with the latest drivers, so you must specify them in the option <code>hardware.nvidia.package</code> getting the value from your selected kernel, for example, <code>config.boot.kernelPackages.nvidia_x11_legacy470</code>. You can check which driver version your GPU supports by visiting the [https://www.nvidia.com/Download/index.aspx nvidia site] and checking the driver version.
* By default, software packaged in source code form has CUDA support disabled, because of the unfree license. To solve this, you can enable builds with CUDA support with a nixpkgs wide configuration, or use binary packaged CUDA compatible software such as [https://github.com/edolstra/nix-warez/tree/master/blender blender-bin].
* Even with the drivers correctly installed, some software, like Blender, may not see the CUDA GPU. Make sure your system configuration has the option <code>hardware.graphics.enable</code> enabled.


== CUDA under WSL ==
== CUDA under WSL ==
Line 196: Line 280:


{{file|cuda-shell.nix|nix|<nowiki>
{{file|cuda-shell.nix|nix|<nowiki>
  shellHook = ''
shellHook = ''
      export CUDA_PATH=${pkgs.cudatoolkit}
  export CUDA_PATH=${pkgs.cudatoolkit}
      export LD_LIBRARY_PATH=/usr/lib/wsl/lib:${pkgs.linuxPackages.nvidia_x11}/lib:${pkgs.ncurses5}/lib
  export LD_LIBRARY_PATH=/usr/lib/wsl/lib:${pkgs.linuxPackages.nvidia_x11}/lib:${pkgs.ncurses5}/lib
      export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
  export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
      export EXTRA_CCFLAGS="-I/usr/include"
  export EXTRA_CCFLAGS="-I/usr/include"
  '';        
'';
</nowiki>}}
</nowiki>}}


Retrieved from "https://wiki.nixos.org/wiki/CUDA"