Official NixOS Wiki

Packaging/Python: Difference between revisions

← Older edit
VisualWikitext
imported>Adisbladis
Abus (talk | contribs)
1 edit
m Fixed formatting error with note
 
(21 intermediate revisions by 11 users not shown)
Line 14: Line 14:
</syntaxHighlight>
</syntaxHighlight>
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>[https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/fetchpypi.nix fetchPypi] { ... }</code>.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>[https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchpypi/default.nix fetchPypi] { ... }</code>.
 
== Testing out a module in a Python shell ==
Once you have your derivation written, you can create a [[Flakes|flake]] to give yourself a shell and easily test out the module you wrote the derivation for.
 
Create a <code>flake.nix</code> next to your derivation:<syntaxhighlight lang="nix">
{
  description = "Dev shell using external derivation";
 
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
    flake-utils.url = "github:numtide/flake-utils";
  };
 
  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        # replace ./derivation.nix with the path (relative to this file) to your derivation
        mypackage = pkgs.python3Packages.callPackage ./derivation.nix {
          lib = pkgs.lib;
        };
      in {
        packages.default = pkgs.python3.withPackages(_: [ mypackage ]);
      });
}
</syntaxhighlight>Replace "mypackage" with your package name. Then, run the flake to get into a python REPL with your package, ready to be imported:<syntaxhighlight lang="shell-session">
$ nix run
Python 3.11.10 (main, Sep  7 2024, 01:03:31) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mypackage
>>> # test things out...
</syntaxhighlight>


== Pip and Virtualenv enabled nix-shell ==
== Pip and Virtualenv enabled nix-shell ==
It might be the case that you simply need to prototype fast small projects with <code>pip</code> and <code>virtualenv</code> without the need for relying on the dependencies being already packaged in nixpkgs.  
It might be the case that you simply need to prototype fast small projects with <code>pip</code> and <code>virtualenv</code> without the need for relying on the dependencies being already packaged in nixpkgs.
{{Notice|Keep in mind that the virtualenv symlinks will be invalidated if you update your system!||
 
{{Note|Keep in mind that the virtualenv symlinks will be invalidated if you update your system!}}
   
   
For a local working python environment you can use the following <code>shell.nix</code><ref>https://groups.google.com/forum/#!topic/nix-devel/3qPfwCAV3GE</ref>.
For a local working python environment you can use the following <code>shell.nix</code><ref>https://groups.google.com/forum/#!topic/nix-devel/3qPfwCAV3GE</ref>.
Line 144: Line 177:


... or use <code>fetchurl</code> to download the <code>*.whl</code> file directly.<br>
... or use <code>fetchurl</code> to download the <code>*.whl</code> file directly.<br>
examples: [https://grep.app/search?filter%5Brepo%5D%5B0%5D=NixOS/nixpkgs&filter%5Blang%5D%5B0%5D=Nix&filter%5Bpath%5D%5B0%5D=pkgs/development/python-modules/&q=.whl grep for ".whl" in nixpkgs/pkgs/development/python-modules/]


reference: [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/fetchpypi.nix fetchPypi implementation]
reference: [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchpypi/default.nix fetchPypi implementation]


== setup.py ==
== Fix Missing <code>setup.py</code> ==


The <code>setup.py</code> file is required for <code>buildPythonPackage</code>,
The <code>setup.py</code> file is required for <code>buildPythonPackage</code>,
but it's missing in some packages
but it's missing in some packages.
If you get the following error, you need to one of the workarounds below.


<pre>
<pre>
Line 166: Line 199:


If both <code>setup.py</code> and <code>pyproject.toml</code> are missing,
If both <code>setup.py</code> and <code>pyproject.toml</code> are missing,
you have to add one of these files, for example:
you have to add one of these files.
 
'''Note:''' sometimes you will be able to find <code>pyproject.toml</code> in the source for a package despite it not being present in a <code>.whl</code> file. You can inspect the contents of a <code>.whl</code> file by downloading it from PyPi and then extracting it with <code>nix-shell -p python311Packages.wheel --command wheel unpack path/to/package.whl</code>.
 
For example, you can create the <code>setup.py</code> in the <code>preBuild</code> phase.


<syntaxHighlight lang=nix>
<syntaxHighlight lang=nix>
buildPythonPackage {
buildPythonPackage {
   preBuild = ''
   preBuild = ''
     cat >setup.py <<'EOF'
     cat > setup.py << EOF
    from setuptools import setup
    setup(
      name='someprogram',
      # ...
    )
    EOF
  '';
}
</syntaxHighlight>
 
setup.py example:
 
<syntaxHighlight lang=python>
from setuptools import setup
from setuptools import setup


Line 205: Line 229:
   },
   },
)
)
    EOF
  '';
}
</syntaxHighlight>
</syntaxHighlight>
More info about the <code>setup.py</code> can be found [https://docs.python.org/3.11/distutils/setupscript.html here]. (<b>note:</b> from python 3.12 onwards, distutils is deprecated see https://docs.python.org/3.11/distutils/index.html)


<code>scripts</code> is useful for self-contained python scripts with no local imports.
<code>scripts</code> is useful for self-contained python scripts with no local imports.
Line 239: Line 268:


== Automatic packaging ==
== Automatic packaging ==
 
{| class="wikitable"
TODO https://github.com/nix-community/pip2nix
|+
 
!
TODO https://github.com/nix-community/poetry2nix
!Project
 
!URL
TODO https://pypi.org/project/pypi2nix/
!Stars
!Status
|-
|TODO
|poetry2nix
|https://github.com/nix-community/poetry2nix
|884+
|unmaintained
|-
|TODO
|pip2nix
|https://github.com/nix-community/pip2nix
|175+
|
|-
|TODO
|<s>pypi2nix</s>
|https://github.com/nix-community/pypi2nix
|194
|archived
|}


== Testing via this command is deprecated ==
== Testing via this command is deprecated ==
Line 264: Line 313:
   checkPhase = ''
   checkPhase = ''
     runHook preCheck
     runHook preCheck
     ${python.interpreter} -m unittest
     ${python3.interpreter} -m unittest
     runHook postCheck
     runHook postCheck
   '';
   '';
Line 272: Line 321:


* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/mk-python-derivation.nix buildPythonPackage implementation]
* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/mk-python-derivation.nix buildPythonPackage implementation]
* [https://ryantm.github.io/nixpkgs/languages-frameworks/python/ Python] in the nixpkgs manual
* [https://nixos.org/manual/nixpkgs/#python Python] in the nixpkgs manual
* [https://github.com/on-nix/python Python on Nix] is an "Extensive collection of Python projects from PyPI"
* [https://github.com/on-nix/python Python on Nix] is an "Extensive collection of Python projects from PyPI"
* [https://ryantm.github.io/nixpkgs/languages-frameworks/rust/#examples Rust section of Nixpkgs manual] - build Rust code in Python projects
* [https://nixos.org/manual/nixpkgs/stable/#examples Rust section of Nixpkgs manual] - build Rust code in Python projects


== References ==
== References ==
[[Category:Python]]
Retrieved from "https://wiki.nixos.org/wiki/Packaging/Python"