Packaging/Tutorial: Difference between revisions
imported>Fadenb m →NOTE3: If you definitely have lots of time, then read Nixpkgs manual: syntaxhighlight |
imported>Jmarmstrong1207 Add tip about the "Create and debug packages" page existing, and that there is a discussion whether to deprecate this one. |
||
(10 intermediate revisions by 8 users not shown) | |||
Line 1: | Line 1: | ||
{{ outdated | This page has duplicate info from [[Create and debug nix packages|with this other wiki page here]]. That page has much more info that might be of more help for you. This page currently has a discussion if it should be deprecated, but at the meantime it still might be useful until a decision is made to migrate over. }} | |||
===== | If you are a new Nix adept and want to package some new software, you will benefit from reading existing Nix package descriptions. The more you read, the faster you'll be able to build your package. I've even based a "Generic Algorithm on Doing Packaging Using Existing Knowledge" on this fact. | ||
The nixpkgs manual describes the [https://nixos.org/nixpkgs/manual/#chap-quick-start standard workflow to adding packages to nixpkgs]. Be sure to read this one first. | |||
===== NOTE 0: If you have time, read [https://unix.stackexchange.com/questions/717168/how-to-package-my-software-in-nix-or-write-my-own-package-derivation-for-nixpkgs this tutorial] first ===== | |||
===== NOTE1: If you have time, [https://nixos.org/nixos/nix-pills/ read Nix Pills] first ===== | |||
===== NOTE2: If you have even more time, [https://nixos.org/nix/manual/#chap-writing-nix-expressions read Nix manual] to get familiar with Nix syntax ===== | ===== NOTE2: If you have even more time, [https://nixos.org/nix/manual/#chap-writing-nix-expressions read Nix manual] to get familiar with Nix syntax ===== | ||
Line 8: | Line 14: | ||
<ol style="list-style-type: decimal;"> | <ol style="list-style-type: decimal;"> | ||
<li><p>Detect package building category. It may be something simple, that | <li><p>Detect package building category. It may be something simple, that requires <code>gcc</code> and <code>make</code> only, or it may be a python library, or it may be a python application, or it may be [https://nixos.org/nixpkgs/manual/#chap-language-support any other LANGUAGE library/application], or it may be Qt applications, or it may be a Gnome application, or it does uses Mono/wine under a hood, or it vendors it's dependencies, or it is closed source/binary redistributable, or it uses CMake/any other alternative build tool, or it actually is a compiler, ... There are lots of such build categories, and it often isn't reflected at hierarchy level in Nixpkgs source tree.</p></li> | ||
<li><p>Try <code>grep</code>ping for building category keywords in Nixpkgs source. Find some example packages, review them and copy one of them as a new expression for your package. Too bad if you didn't find anything, then you should read Nix Pills first and skim through several package expressions in Nixpkgs source to be able to write own | <li><p>Try <code>grep</code>ping for building category keywords in Nixpkgs source. Find some example packages, review them and copy one of them as a new expression for your package. Too bad if you didn't find anything, then you should read Nix Pills first and skim through several package expressions in Nixpkgs source to be able to write your own.</p></li> | ||
<li><p>Build it without changes, just to test it builds</p></li> | <li><p>Build it without changes, just to test if it builds</p></li> | ||
<li><p>Modify source URL and sha256. Don't forget about that sha256 stuff! What I do, I change last symbol of sha256, let the package fail, and then paste correct sha256 again into expression. There are also various prefetch techniques</p></li> | <li><p>Modify source URL and sha256. Don't forget about that sha256 stuff! What I do, I change the last symbol of the sha256, let the package fail, and then paste the correct sha256 again into expression. There are also various prefetch techniques</p></li> | ||
<li><p>Run <code>nix-shell</code> on this expression (for example: <syntaxhighlight lang="bash" inline>nix-shell '<nixpkgs>' -A hello</syntaxhighlight>). Then run in order:</p> | <li><p>Run <code>nix-shell</code> on this expression (for example: <syntaxhighlight lang="bash" inline>nix-shell '<nixpkgs>' -A hello</syntaxhighlight>). Then run in order:</p> | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
$ unpackPhase | $ unpackPhase | ||
$ cd | $ cd $sourceRoot # $sourceRoot is created in unpack phase, see https://nixos.org/manual/nixpkgs/stable/#ssec-unpack-phase for details. | ||
$ patchPhase | $ patchPhase | ||
$ configurePhase | $ configurePhase | ||
$ buildPhase</syntaxhighlight> | $ buildPhase</syntaxhighlight> | ||
<p>(visit [https://nixos.org/nixpkgs/manual/#sec-stdenv-phases stdenv-phases] chapter to learn more about phases)</p></ | <p>(visit [https://nixos.org/nixpkgs/manual/#sec-stdenv-phases stdenv-phases] chapter to learn more about phases)</p> | ||
<p>You may need to run <syntaxhighlight lang="console" inline>$ eval "$configurePhase"</syntaxhighlight> sometimes if the build expression overrides that phase.</p> | |||
<p>It most likely will fail.</p> | |||
<p>NOTE: You can use the [https://nixos.org/manual/nixpkgs/stable/#breakpointhook breakpointHook] to attach into the build environment when an error happens.</p> | |||
</li></ol> | |||
<ol start="6" style="list-style-type: decimal;"> | <ol start="6" style="list-style-type: decimal;"> | ||
<li>Try to find failure category. It may be headers not found, library not found, executable not found, compiler not found, wrong build system, empty result, and many many more... But solutions to these problems are a bit more controllable: | <li>Try to find the failure category. It may be headers not found, library not found, executable not found, compiler not found, wrong build system, empty result, and many many more... But solutions to these problems are a bit more controllable: | ||
<ol style="list-style-type: lower-alpha;"> | |||
<li><code>buildInput</code> dependency missing</li> | |||
<li><code>buildInput</code> dependency missing</li | <li>source requires patching</li> | ||
<li>you should enter a subdir first</li> | |||
<li>you should specify correct build parameters</li> | |||
<li>source requires patching | <li><code>patchShebangs</code> should be run first</li> | ||
</li | <li>....</li> | ||
</ol> | |||
<li>you should enter a subdir first | |||
</li | |||
<li>you should specify correct build parameters | |||
</li | |||
<li><code>patchShebangs</code> should be run first | |||
</li> | |||
<li>....</li></ | |||
Don't hesitate to learn from other package manager expressions, like AUR. | Don't hesitate to learn from other package manager expressions, like AUR. | ||
</li> | |||
<li><p>Apply a fix, [https://unix.stackexchange.com/questions/191885/quickly-reload-nix-expression-in-nix-shell exit nix-shell and enter it again]. Do steps 6-7 while there are problems.</p></li> | <li><p>Apply a fix, [https://unix.stackexchange.com/questions/191885/quickly-reload-nix-expression-in-nix-shell exit nix-shell and enter it again]. Do steps 6-7 while there are problems.</p></li> | ||
<li><p>After you | <li><p>After you get the <code>buildPhase</code> to pass and produce correct binaries/libraries, it's time for the<code>installPhase</code>. There are two ways: either you do package development under root and have write access to Nix store inside nix-shell, or you simply run <code>nix-build</code> on you expression, so Nix will perform it's <code>installPhase</code>.</p> | ||
In first case, you can run | <p>In the first case, you can run</p> | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
$ installPhase | $ installPhase | ||
</syntaxhighlight> | </syntaxhighlight> | ||
directly and examine using <code>tree $out</code> your package files. If something is missing or too many, fix the <code>postInstallPhase</code> or <code>preInstallPhase</code> or <code>installPhase</code> in your expression. | |||
<p>directly and examine using <code>tree $out</code> in your package files. If something is missing or too many, fix the <code>postInstallPhase</code> or <code>preInstallPhase</code> or <code>installPhase</code> in your expression.</p> | |||
</li></ol> | |||
<ol start="9" style="list-style-type: decimal;"> | <ol start="9" style="list-style-type: decimal;"> | ||
<li><p>At this point you should have something that is both buildable and packageable. It still may not be runnable. Run the binaries to find | <li><p>At this point you should have something that is both buildable and packageable. It still may not be runnable. Run the binaries to find any hidden dependencies, or patches required, or wrapper required, or patchelf required, or ... In general, I use Google and search in Nixpkgs issue tracker related problems.</p></li> | ||
<li><p>After all the modifications, package should be buildable, runnable and should be self-sufficient. You may add <code>meta</code> section, but it isn't strictly required if you don't plan to publish your result.</p></li></ol> | <li><p>After all the modifications, package should be buildable, runnable and should be self-sufficient. You may add <code>meta</code> section, but it isn't strictly required if you don't plan to publish your result.</p></li></ol> |
Revision as of 21:54, 28 February 2024
If you are a new Nix adept and want to package some new software, you will benefit from reading existing Nix package descriptions. The more you read, the faster you'll be able to build your package. I've even based a "Generic Algorithm on Doing Packaging Using Existing Knowledge" on this fact.
The nixpkgs manual describes the standard workflow to adding packages to nixpkgs. Be sure to read this one first.
NOTE 0: If you have time, read this tutorial first
NOTE1: If you have time, read Nix Pills first
NOTE2: If you have even more time, read Nix manual to get familiar with Nix syntax
NOTE3: If you definitely have lots of time, then read Nixpkgs manual
Detect package building category. It may be something simple, that requires
gcc
andmake
only, or it may be a python library, or it may be a python application, or it may be any other LANGUAGE library/application, or it may be Qt applications, or it may be a Gnome application, or it does uses Mono/wine under a hood, or it vendors it's dependencies, or it is closed source/binary redistributable, or it uses CMake/any other alternative build tool, or it actually is a compiler, ... There are lots of such build categories, and it often isn't reflected at hierarchy level in Nixpkgs source tree.Try
grep
ping for building category keywords in Nixpkgs source. Find some example packages, review them and copy one of them as a new expression for your package. Too bad if you didn't find anything, then you should read Nix Pills first and skim through several package expressions in Nixpkgs source to be able to write your own.Build it without changes, just to test if it builds
Modify source URL and sha256. Don't forget about that sha256 stuff! What I do, I change the last symbol of the sha256, let the package fail, and then paste the correct sha256 again into expression. There are also various prefetch techniques
Run
nix-shell
on this expression (for example:nix-shell '<nixpkgs>' -A hello
). Then run in order:$ unpackPhase $ cd $sourceRoot # $sourceRoot is created in unpack phase, see https://nixos.org/manual/nixpkgs/stable/#ssec-unpack-phase for details. $ patchPhase $ configurePhase $ buildPhase
(visit stdenv-phases chapter to learn more about phases)
You may need to run
$ eval "$configurePhase"
sometimes if the build expression overrides that phase.It most likely will fail.
NOTE: You can use the breakpointHook to attach into the build environment when an error happens.
- Try to find the failure category. It may be headers not found, library not found, executable not found, compiler not found, wrong build system, empty result, and many many more... But solutions to these problems are a bit more controllable:
buildInput
dependency missing- source requires patching
- you should enter a subdir first
- you should specify correct build parameters
patchShebangs
should be run first- ....
Don't hesitate to learn from other package manager expressions, like AUR.
Apply a fix, exit nix-shell and enter it again. Do steps 6-7 while there are problems.
After you get the
buildPhase
to pass and produce correct binaries/libraries, it's time for theinstallPhase
. There are two ways: either you do package development under root and have write access to Nix store inside nix-shell, or you simply runnix-build
on you expression, so Nix will perform it'sinstallPhase
.In the first case, you can run
$ installPhase
directly and examine using
tree $out
in your package files. If something is missing or too many, fix thepostInstallPhase
orpreInstallPhase
orinstallPhase
in your expression.
At this point you should have something that is both buildable and packageable. It still may not be runnable. Run the binaries to find any hidden dependencies, or patches required, or wrapper required, or patchelf required, or ... In general, I use Google and search in Nixpkgs issue tracker related problems.
After all the modifications, package should be buildable, runnable and should be self-sufficient. You may add
meta
section, but it isn't strictly required if you don't plan to publish your result.