Python: Difference between revisions
m Poetry doesn't have an input named `python` https://github.com/NixOS/nixpkgs/blob/7c503a8a9d182df45a4e4fc2f836bcc7316fed2c/pkgs/tools/package-management/poetry/default.nix#L1 |
Added a sample python / nix-ld patch to allow compiled libraries to run with poetry or venv. |
||
Line 177: | Line 177: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
=== | === Running compiled libraries === | ||
If you want to run some compiled libraries as for example <code>grpcio</code><ref>https://pypi.org/project/grpcio/</ref>, you may encounter the following error :<syntaxhighlight lang="shell-session"> | |||
$ python -c 'import grpc' | |||
$ | Traceback (most recent call last): | ||
< | File "<string>", line 1, in <module> | ||
File "/.../grpc/__init__.py", line 22, in <module> | |||
from grpc import _compression | |||
File "/.../grpc/_compression.py", line 20, in <module> | |||
from grpc._cython import cygrpc | |||
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory | |||
</syntaxhighlight>This means that the library use compiled dynamically linked binaries that your NixOs environment fail to resolve. | |||
On NixOS, installing packages that need to compile code or use C libraries from outside of the <code>nix</code> package manager may fail if dependencies are not found in the expected locations. | |||
There are multiple ways to make it work: | There are multiple ways to make it work: | ||
* Use [https://github.com/GuillaumeDesforges/fix-python/ fix-python], this is most suited for beginners. | * Use [https://github.com/GuillaumeDesforges/fix-python/ fix-python], this is most suited for beginners. | ||
* Create a FHS user env with <code>buildFHSUserEnv</code>. | * Create a FHS user env with <code>buildFHSUserEnv</code>. | ||
* Setup <code>nix-ld</code> in your NixOS configuration. | * Setup <code>nix-ld</code><ref name=":0">https://github.com/Mic92/nix-ld</ref> in your NixOS configuration. | ||
==== Setup nix-ld ==== | |||
nix-ld<ref name=":0" /> allow you to <code>Run unpatched dynamic binaries on NixOS.</code> | |||
The following configuration automatically fix the dependencies :<syntaxhighlight lang="nixos" line="1"> | |||
let | |||
python = pkgs.python311; # or another version | |||
patchedpython = (python.overrideAttrs ( | |||
previousAttrs: { | |||
# Add the nix-ld libraries to the LD_LIBRARY_PATH. | |||
# creating a new library path from all desired libraries | |||
# https://github.com/NixOS/nixpkgs/blob/c339c066b893e5683830ba870b1ccd3bbea88ece/nixos/modules/programs/nix-ld.nix#L44 | |||
# > We currently take all libraries from systemd and nix as the default. | |||
postInstall = previousAttrs.postInstall + '' | |||
mv "$out/bin/python3.11" "$out/bin/unpatched_python3.11" | |||
cat << EOF >> "$out/bin/python3.11" | |||
#!/run/current-system/sw/bin/bash | |||
export LD_LIBRARY_PATH="${lib.makeLibraryPath (with pkgs; [ | |||
zlib | |||
zstd | |||
stdenv.cc.cc | |||
curl | |||
openssl | |||
attr | |||
libssh | |||
bzip2 | |||
libxml2 | |||
acl | |||
libsodium | |||
util-linux | |||
xz | |||
systemd | |||
])}" | |||
exec "$out/bin/unpatched_python3.11" "\$@" | |||
EOF | |||
chmod +x "$out/bin/python3.11" | |||
''; | |||
} | |||
)); | |||
in | |||
{ | |||
# Some other config... | |||
environment.systemPackages = with pkgs; [ | |||
patchedpython | |||
# or maybe if you want poetry | |||
(poetry.override { python3 = patchedpython; }) | |||
]; | |||
} | |||
</syntaxhighlight>This configuration set the <code>LD_LIBRARY_PATH</code> environment variable before running python using the <code>overrideAttrs</code><ref>https://nixos.org/manual/nixpkgs/stable/#sec-pkg-overrideAttrs</ref> function to override the <code>postInstall</code> script of cpython <code>mkDerivation</code><ref>https://github.com/NixOS/nixpkgs/blob/24.05/pkgs/development/interpreters/python/cpython/default.nix</ref>. | |||
After this step, you should be able to install compiled libraries using venv, poetry, conda or other packages managers... | |||
=== Using <code>venv</code> === | |||
To create a Python virtual environment with <code>venv</code>:<syntaxhighlight lang="console"> | |||
$ nix-shell -p python3 --command "python -m venv .venv --copies" | |||
</syntaxhighlight>You can then activate and use the Python virtual environment as usual and install dependencies with <code>pip</code> and similar. | |||
=== Using poetry === | === Using poetry === |