Tensorflow: Difference between revisions

From NixOS Wiki
imported>Biggs
m Note about changing python version to be up to date
Klinger (talk | contribs)
m added Category:Python, link, description
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Tensorflow ==
[https://www.tensorflow.org/ TensorFlow] is a free and open-source software library for machine learning and artificial intelligence.
= Tensorflow =


There several possible ways to install tensorflow. Nixpkgs provides multiple versions, however, it is often desirable to be able to install the latest nightly from pip. This can accomplished in the following ways:
=== Quick shell <ref name=":0">https://discourse.nixos.org/t/cuda-tensorflow-my-setup-is-really-hacky-would-appreciate-help-unhackying-it/43912/2</ref> ===
nix-shell --arg config '{ cudaSupport = true; allowUnfree = true; }' -p 'python3.withPackages (ps: [ ps.tensorflow ]<code>)'</code>
This starts a shell with tensorflow and it has GPU support.


* By making a nix-shell
=== Shell config ===
A basic shell config would look something like this:
<code># shell.nix
{ pkgs ? import <nixpkgs> {}, config ? {} }:
let
<nowiki> </nowiki> pythonPackages = pkgs.python3.withPackages (ps: [ ps.tensorflow ]);
in
pkgs.mkShell {
<nowiki> </nowiki> name = "tf";
<nowiki> </nowiki> buildInputs = [
<nowiki> </nowiki>   (pythonPackages)
<nowiki> </nowiki> ];
<nowiki> </nowiki> shellHook = <nowiki>''</nowiki>
<nowiki> </nowiki>   export PYTHONPATH="${pythonPackages}:${PYTHONPATH:-}"
<nowiki> </nowiki> <nowiki>''</nowiki>;
}</code>
To make this configuration work, edit <code>~/.config/nxpkgs/config.nix</code>:
<code># ~/.config/nxpkgs/config.nix
{ cudaSupport = true; allowUnfree = true;  }</code>


<syntaxhighlight lang="nix">
== References ==
with import <nixpkgs> {};
<ref name=":0" />
mkShell {
  name = "tensorflow-cuda-shell";
  buildInputs = with python3.pkgs; [
    pip
    numpy
    setuptools
  ];
  shellHook = ''
    export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:${pkgs.cudatoolkit_10_1}/lib:${pkgs.cudnn_cudatoolkit_10_1}/lib:${pkgs.cudatoolkit_10_1.lib}/lib:$LD_LIBRARY_PATH
    alias pip="PIP_PREFIX='$(pwd)/_build/pip_packages' TMPDIR='$HOME' \pip"
    export PYTHONPATH="$(pwd)/_build/pip_packages/lib/python3.7/site-packages:$PYTHONPATH"
    export PATH="$(pwd)/_build/pip_packages/bin:$PATH"
    unset SOURCE_DATE_EPOCH
  '';
}
</syntaxhighlight>


Within this shell, pip install tf-nightly should work and provide GPU support. The cuda toolkit version (and the version of Python) can be changed to correspond with the matching tensorflow version.
[[Category:Python]]
 
Note: On NixOS 20.03 and above LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib:/run/opengl-driver-32/lib by default, preventing tensorflow from discovering the Cuda libraries. This can be solved by manually (or using nixGL[https://github.com/guibou/nixGL] ) appending your LD_LIBRARY_PATH in the shellHook, or by reverting to the pre-20.03 behavior by setting setLdLibraryPath to true in your hardware opengl configuration under configuration.nix.
 
 
hardware.opengl.setLdLibraryPath = true;
 
 
 
== See also ==
 
* [https://discourse.nixos.org/t/installing-python-modules-from-pip-buildfhsuserenv-for-python-tutorial/5704/13]

Latest revision as of 18:07, 25 August 2024

TensorFlow is a free and open-source software library for machine learning and artificial intelligence.

Tensorflow

Quick shell [1]

nix-shell --arg config '{ cudaSupport = true; allowUnfree = true; }' -p 'python3.withPackages (ps: [ ps.tensorflow ])'

This starts a shell with tensorflow and it has GPU support.

Shell config

A basic shell config would look something like this:

# shell.nix
{ pkgs ? import <nixpkgs> {}, config ? {} }:
let
  pythonPackages = pkgs.python3.withPackages (ps: [ ps.tensorflow ]);
in
pkgs.mkShell {
  name = "tf";
  buildInputs = [
    (pythonPackages)
  ];
  shellHook = ''
    export PYTHONPATH="${pythonPackages}:${PYTHONPATH:-}"
  '';
}

To make this configuration work, edit ~/.config/nxpkgs/config.nix:

# ~/.config/nxpkgs/config.nix
{ cudaSupport = true; allowUnfree = true;  }

References

[1]