Tensorflow: Difference between revisions

From NixOS Wiki
imported>Samuela
No edit summary
imported>Samuela
No edit summary
Line 1: Line 1:
== Tensorflow ==
= Tensorflow =
There several possible ways to install tensorflow.  
There several possible ways to install tensorflow.  


=== Nix-native packages (recommended) ===
== Nix-native packages (strongly recommended) ==
Use <code>python3Packages.tensorflow</code> or <code>python3Packages.tensorflowWithCuda</code>.
Use <code>python3Packages.tensorflow</code> or <code>python3Packages.tensorflowWithCuda</code>.


=== pip install in a nix-shell ===
'''Cache''': Using the [https://app.cachix.org/cache/cuda-maintainers#pull cuda-maintainers cache] is recommended! It will save you valuable time and electrons. Getting set up should be as simple as <code>cachix use cuda-maintainers</code>. See the [[CUDA]] wiki page for more info.
 
== pip install in a nix-shell ==
Nixpkgs provides multiple versions, however, it is often desirable to be able to install the latest nightly from pip. This can accomplished in the following ways:
Nixpkgs provides multiple versions, however, it is often desirable to be able to install the latest nightly from pip. This can accomplished in the following ways:


Line 32: Line 34:


Note: On NixOS 20.03 and above LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib:/run/opengl-driver-32/lib by default, preventing tensorflow from discovering the Cuda libraries. This can be solved by manually (or using nixGL[https://github.com/guibou/nixGL] ) appending your LD_LIBRARY_PATH in the shellHook, or by reverting to the pre-20.03 behavior by setting setLdLibraryPath to true in your hardware opengl configuration under configuration.nix.
Note: On NixOS 20.03 and above LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib:/run/opengl-driver-32/lib by default, preventing tensorflow from discovering the Cuda libraries. This can be solved by manually (or using nixGL[https://github.com/guibou/nixGL] ) appending your LD_LIBRARY_PATH in the shellHook, or by reverting to the pre-20.03 behavior by setting setLdLibraryPath to true in your hardware opengl configuration under configuration.nix.


  hardware.opengl.setLdLibraryPath = true;
  hardware.opengl.setLdLibraryPath = true;


== See also ==
== See also ==


* [https://discourse.nixos.org/t/installing-python-modules-from-pip-buildfhsuserenv-for-python-tutorial/5704/13]
* [https://discourse.nixos.org/t/installing-python-modules-from-pip-buildfhsuserenv-for-python-tutorial/5704/13]

Revision as of 19:28, 31 March 2022

Tensorflow

There several possible ways to install tensorflow.

Nix-native packages (strongly recommended)

Use python3Packages.tensorflow or python3Packages.tensorflowWithCuda.

Cache: Using the cuda-maintainers cache is recommended! It will save you valuable time and electrons. Getting set up should be as simple as cachix use cuda-maintainers. See the CUDA wiki page for more info.

pip install in a nix-shell

Nixpkgs provides multiple versions, however, it is often desirable to be able to install the latest nightly from pip. This can accomplished in the following ways:

  • By making a nix-shell
with import <nixpkgs> {};
mkShell {
  name = "tensorflow-cuda-shell";
  buildInputs = with python3.pkgs; [
    pip
    numpy
    setuptools
  ];
  shellHook = ''
    export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:${pkgs.cudatoolkit_10_1}/lib:${pkgs.cudnn_cudatoolkit_10_1}/lib:${pkgs.cudatoolkit_10_1.lib}/lib:$LD_LIBRARY_PATH
    alias pip="PIP_PREFIX='$(pwd)/_build/pip_packages' TMPDIR='$HOME' \pip"
    export PYTHONPATH="$(pwd)/_build/pip_packages/lib/python3.7/site-packages:$PYTHONPATH"
    export PATH="$(pwd)/_build/pip_packages/bin:$PATH"
    unset SOURCE_DATE_EPOCH
  '';
}

Within this shell, pip install tf-nightly should work and provide GPU support. The cuda toolkit version (and the version of Python) can be changed to correspond with the matching tensorflow version.

Note: On NixOS 20.03 and above LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib:/run/opengl-driver-32/lib by default, preventing tensorflow from discovering the Cuda libraries. This can be solved by manually (or using nixGL[1] ) appending your LD_LIBRARY_PATH in the shellHook, or by reverting to the pre-20.03 behavior by setting setLdLibraryPath to true in your hardware opengl configuration under configuration.nix.

hardware.opengl.setLdLibraryPath = true;

See also