Revision as of 11:24, 2 May 2024

Tensorflow

Quick shell ^[1]

nix-shell --arg config '{ cudaSupport = true; allowUnfree = true; }' -p 'python3.withPackages (ps: [ ps.tensorflow ])'

This starts a shell with tensorflow and it has GPU support.

Shell config

A basic shell config would look something like this:

# shell.nix
{ pkgs ? import <nixpkgs> {}, config ? {} }:
let
  pythonPackages = pkgs.python3.withPackages (ps: [ ps.tensorflow ]);
in
pkgs.mkShell {
  name = "tf";
  buildInputs = [
    (pythonPackages)
  ];
  shellHook = ''
    export PYTHONPATH="${pythonPackages}:${PYTHONPATH:-}"
  '';
}

To make this configuration work, edit ~/.config/nxpkgs/config.nix:

# ~/.config/nxpkgs/config.nix
{ cudaSupport = true; allowUnfree = true;  }

References

^[1]

↑ ^1.0 ^1.1 https://discourse.nixos.org/t/cuda-tensorflow-my-setup-is-really-hacky-would-appreciate-help-unhackying-it/43912/2

[:0-1] 1.0 ^1.1 https://discourse.nixos.org/t/cuda-tensorflow-my-setup-is-really-hacky-would-appreciate-help-unhackying-it/43912/2

[1]

@@ Line 1: / Line 1: @@
 = Tensorflow =
-There several possible ways to install tensorflow.
-== Nix-native packages (strongly recommended) ==
+=== Quick shell <ref name=":0">https://discourse.nixos.org/t/cuda-tensorflow-my-setup-is-really-hacky-would-appreciate-help-unhackying-it/43912/2</ref> ===
-Use <code>python3Packages.tensorflow</code> or <code>python3Packages.tensorflowWithCuda</code>.
+ nix-shell --arg config '{ cudaSupport = true; allowUnfree = true; }' -p 'python3.withPackages (ps: [ ps.tensorflow ]<code>)'</code>
+This starts a shell with tensorflow and it has GPU support.
-'''Cache''': Using the [https://app.cachix.org/cache/cuda-maintainers#pull cuda-maintainers cache] is recommended! It will save you valuable time and electrons. Getting set up should be as simple as <code>cachix use cuda-maintainers</code>. See the [[CUDA]] wiki page for more info.
+=== Shell config ===
+A basic shell config would look something like this:
+ <code># shell.nix
+ { pkgs ? import <nixpkgs> {}, config ? {} }:
+ let
+ <nowiki> </nowiki> pythonPackages = pkgs.python3.withPackages (ps: [ ps.tensorflow ]);
+ in
+ pkgs.mkShell {
+ <nowiki> </nowiki> name = "tf";
+ <nowiki> </nowiki> buildInputs = [
+ <nowiki> </nowiki>   (pythonPackages)
+ <nowiki> </nowiki> ];
+ <nowiki> </nowiki> shellHook = <nowiki>''</nowiki>
+ <nowiki> </nowiki>   export PYTHONPATH="${pythonPackages}:${PYTHONPATH:-}"
+ <nowiki> </nowiki> <nowiki>''</nowiki>;
+ }</code>
+To make this configuration work, edit <code>~/.config/nxpkgs/config.nix</code>:
+ <code># ~/.config/nxpkgs/config.nix
+ { cudaSupport = true; allowUnfree = true;  }</code>
-== pip install in a nix-shell ==
+== References ==
-Nixpkgs provides multiple versions, however, it is often desirable to be able to install the latest nightly from pip. This can accomplished in the following ways:
+<ref name=":0" />
-* By making a nix-shell
-<syntaxhighlight lang="nix">
-with import <nixpkgs> {};
-mkShell {
-  name = "tensorflow-cuda-shell";
-  buildInputs = with python3.pkgs; [
-    pip
-    numpy
-    setuptools
-  ];
-  shellHook = ''
-    export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:${pkgs.cudatoolkit_10_1}/lib:${pkgs.cudnn_cudatoolkit_10_1}/lib:${pkgs.cudatoolkit_10_1.lib}/lib:$LD_LIBRARY_PATH
-    alias pip="PIP_PREFIX='$(pwd)/_build/pip_packages' TMPDIR='$HOME' \pip"
-    export PYTHONPATH="$(pwd)/_build/pip_packages/lib/python3.7/site-packages:$PYTHONPATH"
-    export PATH="$(pwd)/_build/pip_packages/bin:$PATH"
-    unset SOURCE_DATE_EPOCH
-  '';
-}
-</syntaxhighlight>
-Within this shell, <code>pip install tf-nightly</code> should work and provide GPU support. The cuda toolkit version (and the version of Python) can be changed to correspond with the matching tensorflow version.
-Note: On NixOS 20.03 and above LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib:/run/opengl-driver-32/lib by default, preventing tensorflow from discovering the Cuda libraries. This can be solved by manually (or using nixGL[https://github.com/guibou/nixGL] ) appending your LD_LIBRARY_PATH in the shellHook, or by reverting to the pre-20.03 behavior by setting setLdLibraryPath to true in your hardware opengl configuration under configuration.nix.
- hardware.opengl.setLdLibraryPath = true;
-== See also ==
-* [https://discourse.nixos.org/t/installing-python-modules-from-pip-buildfhsuserenv-for-python-tutorial/5704/13]

Revision as of 11:24, 2 May 2024

Tensorflow

Quick shell [1]

Shell config

References

Quick shell ^[1]