Python: Difference between revisions

Line 1:

== ~~Installation~~ ==

== Use a Python environment with Nix ==

Python environment allow you to run Python code with the <code>python</code> command. It may also have Python packages installed as well.

Python ~~is a development package~~, ~~and '''not meant to go in your system or home configuration'''~~.

Nix allows to create many independent Python environment, much like <code>venv</code>.

~~If you need access~~ to ~~python for development,~~ create a ~~<code>shell~~.~~nix</code> for the specific project, along with any libraries needed:~~

There are many ways to create a Python environment with Nix.

<~~syntaxHighlight~~ lang="nix">

=== Using the Nixpkgs Python infrastructure ===

Nixpkgs has the few last Python versions packaged, as well as a consequent set of Python packages packaged that you can use to quickly create a Python environment.

Create a file <code>shell.nix</code> in the project directory, with the following template:

# shell.nix

let

pkgs = import <nixpkgs> {};

Line 12:

Line 19:

packages = [

(pkgs.python3.withPackages (python-pkgs: [

# select Python packages here

python-pkgs.pandas

python-pkgs.requests

Line 17:

Line 25:

];

}

</~~syntaxHighlight~~>

</syntaxhighlight>

This example will make Nix create a Python environment with Python packages "pandas" and "requests". You can remove them, or add more.

You can find Python packages that are available in Nixpkgs using [https://search.nixos.org/packages search.nixos.org]. For instance, type a Python package name (for example <code>numpy</code>) in the search bar and click on the search button on the right. Then on the left in the "Package sets" section click on "python311Packages" to list only Python packages. for Python (version 3.11).

Note that in the snippet above, on line 8 and 9, each package to install is in the form <code>python-pkgs.<name></code>. The package name must correspond to the one found in [https://search.nixos.org/packages search.nixos.org] as explained above. To understand why it is prefixed by <code>python-pkgs</code>, read the [https://nix.dev/tutorials/nix-language.html Nix language basics].

Then, once you have picked the Python packages you want, run <code>nix-shell</code> (or <code>nix develop -f shell.nix</code>) to make Nix create the Python environment and enter it. If it succeeds, Python should be available in your PATH, try using <code>python --version</code> at this point.

As a word of warning, using <code>pkgs = import <nixpkgs> {};</code> as in the snippet above (line 3) will result in a non-reproducible environment. Use Nixpkgs pinning or Nix Flakes to make your environment reproducible.

==== Using a Python packages not in Nixpkgs ====

Python packages in Nixpkgs are created and updated by Nixpkgs maintainers. Although the community invests a great effort to keep a complete and up-to-date package set, it isn't uncommon for packages you want to be missing, out of date or broken. To use your own packages in a Nix environment, you may package it yourself.

~~Then run <code>nix~~-~~shell<~~/~~code> to use~~ the ~~shell~~.

The following is a high-level overview. For a complete explanation, see [https://nixos.org/manual/nixpkgs/unstable/#developing-with-python Developing with Python] in the Nixpkgs Manual.

== ~~Using alternative packages~~ ==

Generally, you may create a file that looks like this:<syntaxhighlight lang="nix" line="1">

# toolz.nix

{ lib

, buildPythonPackage

, fetchPypi

, setuptools

, wheel

}:

buildPythonPackage rec {

pname = "toolz";

version = "0.10.0";

src = fetchPypi {

inherit pname version;

hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";

};

We saw above how to install Python packages using nixpkgs. Since these are written by hand by nixpkgs maintainers, it isn't uncommon for packages you want to be missing or out of date. To create a custom Python environment with your own package(s), first create a derivation for each python package (look at examples in the <code>python-modules</code> subfolder in Nixpkgs). Then, use those derivations with <code>callPackage</code> as follows:

# do not run tests

doCheck = false;

# specific to buildPythonPackage, see its reference

~~with pkgs;~~

pyproject = true;

build-system = [

setuptools

wheel

];

}

</syntaxhighlight>Given the file above is named <code>toolz.nix</code> and is the same directory as the previous <code>shell.nix</code> , you can edit <code>shell.nix</code> to use the package <code>toolz</code> above like so: <syntaxhighlight lang="nix" line="1">

# shell.nix

let

~~my-python-package~~ = ~~ps: ps.callPackage ./my-package.nix~~ {};

pkgs = import <nixpkgs> {};

~~python-with-my-~~packages = python3.withPackages(ps: ~~with ps;~~ [

in pkgs.mkShell {

~~(my~~-python-~~package ps~~)

packages = [

]);

(pkgs.python3.withPackages (python-pkgs: [

~~in ...~~

# select Python packages here

</syntaxhighlight>

python-pkgs.pandas

python-pkgs.requests

(pkgs.callPackage ./toolz.nix)

]))

];

}

</syntaxhighlight>Note that the parenthesis (line 10) are required.

~~=== Package~~ and ~~development shell for a python project ===~~

Next time you enter the shell specified by this file, Nix will build and include the Python package you have written.

== Package a Python application ==

It is possible to use <code>buildPythonApplication</code> to package python applications. As explained in the nixpkgs manual, it uses the widely used `setup.py` file in order to package properly the application. We now show how to package a simple python application: a basic flask web server.

Line 355:

Line 407:

== See also ==

* [https://nixos.org/manual/nixpkgs/unstable/#python "Python" in Nixpkgs Manual]

* [[Packaging/Python]]

* [https://ryantm.github.io/nixpkgs/languages-frameworks/python/#python Python user guide in nixpkgs manual]

* [https://github.com/cachix/nixpkgs-python Nixpgs-python: A repo providing every version of python as flake ]

[[Category:Languages]]

@@ Line 1: / Line 1: @@
-== Installation ==
+== Use a Python environment with Nix ==
+Python environment allow you to run Python code with the <code>python</code> command. It may also have Python packages installed as well.
-Python is a development package, and '''not meant to go in your system or home configuration'''.
+Nix allows to create many independent Python environment, much like <code>venv</code>.
-If you need access to python for development, create a <code>shell.nix</code> for the specific project, along with any libraries needed:
+There are many ways to create a Python environment with Nix.
-<syntaxHighlight lang="nix">
+=== Using the Nixpkgs Python infrastructure ===
+Nixpkgs has the few last Python versions packaged, as well as a consequent set of Python packages packaged that you can use to quickly create a Python environment.
+Create a file <code>shell.nix</code> in the project directory, with the following template:
+<syntaxhighlight lang="nix" line="1">
+# shell.nix
 let
    pkgs = import <nixpkgs> {};
@@ Line 12: / Line 19: @@
    packages = [
      (pkgs.python3.withPackages (python-pkgs: [
+      # select Python packages here
        python-pkgs.pandas
        python-pkgs.requests
@@ Line 17: / Line 25: @@
    ];
 }
-</syntaxHighlight>
+</syntaxhighlight>
+This example will make Nix create a Python environment with Python packages "pandas" and "requests". You can remove them, or add more.
+You can find Python packages that are available in Nixpkgs using [https://search.nixos.org/packages search.nixos.org]. For instance, type a Python package name (for example <code>numpy</code>) in the search bar and click on the search button on the right. Then on the left in the "Package sets" section click on "python311Packages" to list only Python packages. for Python (version 3.11).
+Note that in the snippet above, on line 8 and 9, each package to install is in the form <code>python-pkgs.<name></code>. The package name must correspond to the one found in [https://search.nixos.org/packages search.nixos.org] as explained above. To understand why it is prefixed by <code>python-pkgs</code>, read the [https://nix.dev/tutorials/nix-language.html Nix language basics].
+Then, once you have picked the Python packages you want, run <code>nix-shell</code> (or <code>nix develop -f shell.nix</code>) to make Nix create the Python environment and enter it. If it succeeds, Python should be available in your PATH, try using <code>python --version</code> at this point.
+As a word of warning, using <code>pkgs = import <nixpkgs> {};</code> as in the snippet above (line 3) will result in a non-reproducible environment. Use Nixpkgs pinning or Nix Flakes to make your environment reproducible.
+==== Using a Python packages not in Nixpkgs ====
+Python packages in Nixpkgs are created and updated by Nixpkgs maintainers. Although the community invests a great effort to keep a complete and up-to-date package set, it isn't uncommon for packages you want to be missing, out of date or broken. To use your own packages in a Nix environment, you may package it yourself.
-Then run <code>nix-shell</code> to use the shell.
+The following is a high-level overview. For a complete explanation, see [https://nixos.org/manual/nixpkgs/unstable/#developing-with-python Developing with Python] in the Nixpkgs Manual.
-== Using alternative packages ==
+Generally, you may create a file that looks like this:<syntaxhighlight lang="nix" line="1">
+# toolz.nix
+{ lib
+, buildPythonPackage
+, fetchPypi
+, setuptools
+, wheel
+}:
+buildPythonPackage rec {
+  pname = "toolz";
+  version = "0.10.0";
+  src = fetchPypi {
+    inherit pname version;
+    hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
+  };
-We saw above how to install Python packages using nixpkgs. Since these are written by hand by nixpkgs maintainers, it isn't uncommon for packages you want to be missing or out of date. To create a custom Python environment with your own package(s), first create a derivation for each python package (look at examples in the <code>python-modules</code> subfolder in Nixpkgs). Then, use those derivations with <code>callPackage</code> as follows:
+  # do not run tests
+  doCheck = false;
-<syntaxhighlight lang="nix">
+  # specific to buildPythonPackage, see its reference
-with pkgs;
+  pyproject = true;
+  build-system = [
+    setuptools
+    wheel
+  ];
+}
+</syntaxhighlight>Given the file above is named <code>toolz.nix</code> and is the same directory as the previous <code>shell.nix</code> , you can edit <code>shell.nix</code> to use the package <code>toolz</code> above like so: <syntaxhighlight lang="nix" line="1">
+# shell.nix
 let
-   my-python-package = ps: ps.callPackage ./my-package.nix {};
+   pkgs = import <nixpkgs> {};
-   python-with-my-packages = python3.withPackages(ps: with ps; [
+in pkgs.mkShell {
-    (my-python-package ps)
+   packages = [
-   ]);
+    (pkgs.python3.withPackages (python-pkgs: [
-in ...
+      # select Python packages here
-</syntaxhighlight>
+      python-pkgs.pandas
+      python-pkgs.requests
+      (pkgs.callPackage ./toolz.nix)
+    ]))
+   ];
+}
+</syntaxhighlight>Note that the parenthesis (line 10) are required.
-=== Package and development shell for a python project ===
+Next time you enter the shell specified by this file, Nix will build and include the Python package you have written.
+== Package a Python application ==
 It is possible to use <code>buildPythonApplication</code> to package python applications. As explained in the nixpkgs manual, it uses the widely used `setup.py` file in order to package properly the application. We now show how to package a simple python application: a basic flask web server.
@@ Line 355: / Line 407: @@
 == See also ==
+* [https://nixos.org/manual/nixpkgs/unstable/#python "Python" in Nixpkgs Manual]
 * [[Packaging/Python]]
-* [https://ryantm.github.io/nixpkgs/languages-frameworks/python/#python Python user guide in nixpkgs manual]
 * [https://github.com/cachix/nixpkgs-python Nixpgs-python: A repo providing every version of python as flake ]
 [[Category:Languages]]