Packaging/Python: Difference between revisions

From NixOS Wiki
imported>Mic92
link to language helpers instead of out-dated list of tools
imported>Milahu
+ link to fetchPypi impl
Line 14: Line 14:
</syntaxHighlight>
</syntaxHighlight>
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>fetchPypi { ... }</code>.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>[https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/fetchpypi.nix fetchPypi] { ... }</code>.


== Pip and Virtualenv enabled nix-shell ==
== Pip and Virtualenv enabled nix-shell ==

Revision as of 20:21, 12 October 2021

See Language-specific package helpers for a list of tools to package python packages.

Prepare Packaging

When you want to package a new software from a local checkout with the inputs coming from nixpkgs (and not virtualenv+pip) you can use the following shell.nix [1]:

with import <nixpkgs> {};
with pkgs.python3Packages;

buildPythonPackage rec {
  name = "mypackage";
  src = ./path/to/source;
  propagatedBuildInputs = [ pytest numpy pkgs.libsndfile ];
}

You can now run nix-shell and it will drop you in a shell similar to the python setup.py develop mode which uses the local code in ./path/to/source as input. propagatedBuildInputs will contain the packages you need in your project. After you've finished developing you can replace the relative path with fetchFromGitHub { ... } or fetchPypi { ... }.

Pip and Virtualenv enabled nix-shell

It might be the case that you simply need to prototype fast small projects with pip and virtualenv without the need for relying on the dependencies being already packaged in nixpkgs. {{Notice|Keep in mind that the virtualenv symlinks will be invalidated if you update your system!||

For a local working python environment you can use the following shell.nix[2].

with import <nixpkgs> {};

stdenv.mkDerivation {
  name = "pip-env";
  buildInputs = [
    # System requirements.
    readline

    # Python requirements (enough to get a virtualenv going).
    python27Full
    python27Packages.virtualenv
    python27Packages.pip
  ];
  src = null;
  shellHook = ''
    # Allow the use of wheels.
    SOURCE_DATE_EPOCH=$(date +%s)

    # Augment the dynamic linker path
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${R}/lib/R/lib:${readline}/lib
  '';
}

When invoked with nix-shell, this environment gives you a readline-enabled Python, plus virtualenv and pip, from which you can create a virtual environment and then proceed to fill it with pip-installed packages from requirements.txt or any other source of packages.

And the only other thing you need to do is figure out which non-Python packages your pip-installable packages will need, and include them in buildInputs.

Caveats

ModuleNotFoundError: No module named 'pkg_resources'

If you see the pkg_resources issues at runetime:

  File "/nix/store/czdpbzpv9csghfs0clw10i758mpxixbc-python3.7-acronym-1.6.0/lib/python3.7/site-packages/acronym/acronym.py", line 17, in <module>
    from pkg_resources import resource_filename
ModuleNotFoundError: No module named 'pkg_resources'

add

  ...
  propagatedBuildInputs = [
    ...
    setuptools
  ];
}

to your derivation.

References