Official NixOS Wiki

Packaging/Python: Difference between revisions

← Older edit
QuBe (talk | contribs)
9 edits
VisualWikitext
imported>Abhillman
info regarding pyproject.toml in src
QuBe (talk | contribs)
9 edits
m Add uv2nix to the table
 
(15 intermediate revisions by 8 users not shown)
Line 15: Line 15:
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
You can now run <code>nix-shell</code> and it will drop you in a shell similar to the <code>python setup.py develop</code> mode which uses the local code in <tt>./path/to/source</tt> as input. <code>propagatedBuildInputs</code> will contain the packages you  need in your project.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>[https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchpypi/default.nix fetchPypi] { ... }</code>.
After you've finished developing you can replace the relative path with <code>fetchFromGitHub { ... }</code> or <code>[https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchpypi/default.nix fetchPypi] { ... }</code>.
== Testing out a module in a Python shell ==
Once you have your derivation written, you can create a [[Flakes|flake]] to give yourself a shell and easily test out the module you wrote the derivation for.
Create a <code>flake.nix</code> next to your derivation:<syntaxhighlight lang="nix">
{
  description = "Dev shell using external derivation";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        # replace ./derivation.nix with the path (relative to this file) to your derivation
        mypackage = pkgs.python3Packages.callPackage ./derivation.nix {
          lib = pkgs.lib;
        };
      in {
        packages.default = pkgs.python3.withPackages(_: [ mypackage ]);
      });
}
</syntaxhighlight>Replace "mypackage" with your package name. Then, run the flake to get into a python REPL with your package, ready to be imported:<syntaxhighlight lang="shell-session">
$ nix run
Python 3.11.10 (main, Sep  7 2024, 01:03:31) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mypackage
>>> # test things out...
</syntaxhighlight>


== Pip and Virtualenv enabled nix-shell ==
== Pip and Virtualenv enabled nix-shell ==
It might be the case that you simply need to prototype fast small projects with <code>pip</code> and <code>virtualenv</code> without the need for relying on the dependencies being already packaged in nixpkgs.  
It might be the case that you simply need to prototype fast small projects with <code>pip</code> and <code>virtualenv</code> without the need for relying on the dependencies being already packaged in nixpkgs.
{{Notice|Keep in mind that the virtualenv symlinks will be invalidated if you update your system!||
 
{{Note|Keep in mind that the virtualenv symlinks will be invalidated if you update your system!}}
   
   
For a local working python environment you can use the following <code>shell.nix</code><ref>https://groups.google.com/forum/#!topic/nix-devel/3qPfwCAV3GE</ref>.
For a local working python environment you can use the following <code>shell.nix</code><ref>https://groups.google.com/forum/#!topic/nix-devel/3qPfwCAV3GE</ref>.
Line 152: Line 185:
but it's missing in some packages.
but it's missing in some packages.
If you get the following error, you need to one of the workarounds below.
If you get the following error, you need to one of the workarounds below.
'''Note: sometimes you will be able to find <code>pyproject.toml</code> in the source for a package despite it not being present in a <code>.whl</code> file.''' You can inspect the contents of a <code>.whl</code> file by downloading it from PyPi and then extracting it with <code>nix shell -p python311Packages.wheel --command wheel unpack path/to/package.whl</code>.


<pre>
<pre>
Line 169: Line 200:
If both <code>setup.py</code> and <code>pyproject.toml</code> are missing,
If both <code>setup.py</code> and <code>pyproject.toml</code> are missing,
you have to add one of these files.
you have to add one of these files.
'''Note:''' sometimes you will be able to find <code>pyproject.toml</code> in the source for a package despite it not being present in a <code>.whl</code> file. You can inspect the contents of a <code>.whl</code> file by downloading it from PyPi and then extracting it with <code>nix-shell -p python311Packages.wheel --command wheel unpack path/to/package.whl</code>.
For example, you can create the <code>setup.py</code> in the <code>preBuild</code> phase.
For example, you can create the <code>setup.py</code> in the <code>preBuild</code> phase.


Line 200: Line 234:
</syntaxHighlight>
</syntaxHighlight>


More info about the <code>setup.py</code> can be found [https://docs.python.org/3/distutils/setupscript.html here].
More info about the <code>setup.py</code> can be found [https://docs.python.org/3.11/distutils/setupscript.html here]. (<b>note:</b> from python 3.12 onwards, distutils is deprecated see https://docs.python.org/3.11/distutils/index.html)


<code>scripts</code> is useful for self-contained python scripts with no local imports.
<code>scripts</code> is useful for self-contained python scripts with no local imports.
Line 234: Line 268:


== Automatic packaging ==
== Automatic packaging ==
 
{| class="wikitable"
TODO https://github.com/nix-community/poetry2nix - 400 stars
|+
 
!
TODO https://github.com/nix-community/pip2nix - 100 stars
!Project
 
!URL
TODO https://github.com/nix-community/pypi2nix - 200 stars - archived
!Stars
!Status
|-
|TODO
|poetry2nix
|https://github.com/nix-community/poetry2nix
|884+
|unmaintained
|-
|TODO
|uv2nix
|https://github.com/pyproject-nix/uv2nix
|675+
|
|-
|TODO
|pip2nix
|https://github.com/nix-community/pip2nix
|175+
|
|-
|TODO
|<s>pypi2nix</s>
|https://github.com/nix-community/pypi2nix
|194
|archived
|}


== Testing via this command is deprecated ==
== Testing via this command is deprecated ==
Line 267: Line 327:


* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/mk-python-derivation.nix buildPythonPackage implementation]
* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/mk-python-derivation.nix buildPythonPackage implementation]
* [https://ryantm.github.io/nixpkgs/languages-frameworks/python/ Python] in the nixpkgs manual
* [https://nixos.org/manual/nixpkgs/#python Python] in the nixpkgs manual
* [https://github.com/on-nix/python Python on Nix] is an "Extensive collection of Python projects from PyPI"
* [https://github.com/on-nix/python Python on Nix] is an "Extensive collection of Python projects from PyPI"
* [https://ryantm.github.io/nixpkgs/languages-frameworks/rust/#examples Rust section of Nixpkgs manual] - build Rust code in Python projects
* [https://nixos.org/manual/nixpkgs/stable/#examples Rust section of Nixpkgs manual] - build Rust code in Python projects


== References ==
== References ==
[[Category:Python]]
Retrieved from "https://wiki.nixos.org/wiki/Packaging/Python"