Documentation Gaps: Difference between revisions

(14 intermediate revisions by 9 users not shown)

Line 1:

~~= Documentation Gaps =~~

{{Merge|FAQ|This is quite literally an FAQ outside of the FAQ page.}}

~~It's possible this is just me, but I find that there's several concepts implied by the Nix manuals that aren't explained well there. I'd like to contribute~~ documentation ~~for those myself, but there are cases where I don't understand it myself. (Or else I wouldn't want there to be documentation.)~~

This page details some known documentation gaps in the official NixOS/Nixpkgs documentation.

~~There's no small danger that as I learn the answers to my questions, I'll forget the question~~ in the ~~first place, so I'd like to keep a little list of them here, by way of note taking.~~

I've noticed that several members of the community seem to know answers to these questions off the top of their heads - I hope they don't take the existence of this page personally: I'd love to bug them less on IRC.

== What are setup hooks? ==

Line 13:

Line 9:

<blockquote>'''Answer:'''

Mostly yes. Some of them [https://github.com/NixOS/nixpkgs/blob/c3b2c5bf77f9d437b4656f0244bcdf2ab8d0102c/pkgs/stdenv/generic/default.nix#L101-L111 are automatically enabled by default]. Others you can enable on-demand, by adding them to your <code>buildInputs</code> — e.g. <code>buildInputs = [ makeWrapper ];</code> (to find the ~~"~~proper~~"~~ name for this use, you must go [https://search.nix.gsc.io/?q=makeWrapper%20%3D&i=nope&files=&repos= top-level hunting]). Then they will either auto-inject some steps into build phases (e.g. [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/development/tools/misc/pkgconfig/default.nix#L8 <code>~~pkgconfig~~</code>] or autoconf), or allow you to use some useful scripts during build easily (e.g. <code>makeWrapper</code> — see [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L130 wrapProgram] and [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L1 makeWrapper] docs for details, then [https://~~search~~.~~nix.gsc.io~~/?q=~~wrapProgram~~&~~i=nope&files=&repos~~= grep in nixpkgs] for numerous usage examples).

Mostly yes. Some of them [https://github.com/NixOS/nixpkgs/blob/c3b2c5bf77f9d437b4656f0244bcdf2ab8d0102c/pkgs/stdenv/generic/default.nix#L101-L111 are automatically enabled by default]. Others you can enable on-demand, by adding them to your <code>buildInputs</code> — e.g. <code>buildInputs = [ makeWrapper ];</code> (to find the "proper" name for this use, you must go [https://search.nix.gsc.io/?q=makeWrapper%20%3D&i=nope&files=&repos= top-level hunting]). Then they will either auto-inject some steps into build phases (e.g. [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/development/tools/misc/pkgconfig/default.nix#L8 <code>pkg-config</code>] or autoconf), or allow you to use some useful scripts during build easily (e.g. <code>makeWrapper</code> — see [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L130 wrapProgram] and [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L1 makeWrapper] docs for details, then [https://github.com/search?q=repo%3ANixOS%2Fnixpkgs%20wrapProgram&type=code grep in nixpkgs] for numerous usage examples).

See also [https://github.com/NixOS/nixpkgs/tree/master/pkgs/build-support/setup-hooks <nixpkgs>/pkgs/build-support/setup-hooks] for a list of some common setup hooks.

</blockquote>

== What's the relationship between <code>nix-env</code> and <code>buildEnv</code> ==

AFAICT, <code>nix-env</code> somehow assembles a <code>buildEnv</code> expression and the realization of that expression becomes a profile, somehow.

# [https://github.com/NixOS/nix/blob/~~215b70f51e5abd350c9b7db656aedac9d96d0046~~/src/nix-env/user-env.cc#~~L108~~ '''nix'''/user-env/]

# [https://github.com/NixOS/nix/blob/2c42a9dbaa805f4f29561d9a1c10b41dfe98dcfa/src/nix-env/user-env.cc#L123 '''nix'''/user-env/]

# [https://github.com/NixOS/nix/blob/master/~~corepkgs~~/buildenv.nix '''nix'''/~~corepkgs~~/buildenv.nix]

# [https://github.com/NixOS/nix/blob/master/src/nix-env/buildenv.nix '''nix'''/src/nix-env/buildenv.nix]

# [https://github.com/NixOS/nix/~~tree~~/master/src/buildenv '''nix'''/src/buildenv/]

# [https://github.com/NixOS/nix/blob/master/src/libstore/builtins/buildenv.cc '''nix'''/src/libstore/builtins/buildenv.cc]

'''#1''' loads the nix file in '''#2''' and uses it to merge everything together, and '''#2''' uses a copy of '''#3''' that was compiled when nix got built

'''#1''' loads the nix file in '''#2''' and uses it to merge everything together, and '''#2''' uses a copy of '''#3''' that was compiled when nix got built.

Have a look also at [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/buildenv/default.nix '''nixpkgs'''/pkgs/build-support/buildenv/default.nix]

== How to add a new cross-compilation target/platform? ==

Line 37:

Line 36:

In nixpkgs there is [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/replace-dependency.nix replace-dependency] which provides this functionality but it is neither used nor explicitly documented as of right now (2017-06-22).

<blockquote>'''Answer:''' There is an open [https://github.com/NixOS/nixpkgs/pull/10851/ pull request] and [https://theshortlog.com/~~2016/10/01~~/~~Nix~~-security-updates-update/ blog article] to implement security updates

<blockquote>'''Answer:''' There is an open [https://github.com/NixOS/nixpkgs/pull/10851/ pull request] and [https://www.theshortlog.com/posts/nix-security-updates-update/ blog article] to implement security updates

</blockquote>

== Tips for using nix-shell for development ==

mkShell is like stdenv.mkDerivationto dedicated for shell.nix. It adds one more feature where multiple package build inputs can all be merged together. This is useful when developing in a repo that has multiple packages (micro-services).

</blockquote>

== The structure and placement of nixpkgs ==

Basically something that documents what all the directories mean and where things are placed and how they are structured and why.

== A list of all the trivial builders and build support ==

And an explanation of what they do and what they are used for.

== How do manpages work? Or: <code>environment.pathsToLink</code> and <code>buildEnv</code> ==

I'm trying to understand how manpages work in Nix/NixOS.

There is a pacakge called [https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/misc/man-db/default.nix#L29 <code>man-db</code>], which seems to be looking in <code>/nix/var/nix/profiles/default/share/man</code> for manpages. It can build a cache at <code>/var/cache/man/nixpkgs</code>.

Under NixOS (on my machine, running NixOS 20.09), these paths do not exist. <code>man-db</code> is installed as a NixOS module, the configuration is at [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/misc/documentation.nix#L104 <code>nixos/modules/misc/documentation.nix</code>] in the nixpkgs repository. Manpages are in <code>/run/current-system/sw/share/man</code>, and the cache resides in <code>/var/cache/man/nixos</code>.

What I don't understand is: How are these paths populated? In the module, if <code>man</code> is enabled, <code>environment.pathsToLink</code> is set to <code>[ "/share/man" ]</code>. The documentation describes this option as "List of directories to be symlinked in /run/current-system/sw". Does this mean that for every package in <code>environment.systemPackages</code>, it does something like <code>for f in ${pkg}/share/man/* ; do ln -s $f /run/current-system/sw/share/man/$fname ; done</code>? I could only find that <code>environment.systemPackages</code> is later used to make a <code>buildEnv</code>, but what <code>buildEnv</code> does is a mystery to me.

So, to summarize my ramblings, I guess I would find a thorough documentation of buildEnv very helpful.

[[Category:Community]]

@@ Line 1: / Line 1: @@
-= Documentation Gaps =
+{{Merge|FAQ|This is quite literally an FAQ outside of the FAQ page.}}
-It's possible this is just me, but I find that there's several concepts implied by the Nix manuals that aren't explained well there. I'd like to contribute documentation for those myself, but there are cases where I don't understand it myself. (Or else I wouldn't want there to be documentation.)
+This page details some known documentation gaps in the official NixOS/Nixpkgs documentation.
-There's no small danger that as I learn the answers to my questions, I'll forget the question in the first place, so I'd like to keep a little list of them here, by way of note taking.
-I've noticed that several members of the community seem to know answers to these questions off the top of their heads - I hope they don't take the existence of this page personally: I'd love to bug them less on IRC.
 == What are setup hooks? ==
@@ Line 13: / Line 9: @@
 <blockquote>'''Answer:'''
-Mostly yes. Some of them [https://github.com/NixOS/nixpkgs/blob/c3b2c5bf77f9d437b4656f0244bcdf2ab8d0102c/pkgs/stdenv/generic/default.nix#L101-L111 are automatically enabled by default]. Others you can enable on-demand, by adding them to your <code>buildInputs</code> — e.g. <code>buildInputs = [ makeWrapper ];</code> (to find the &quot;proper&quot; name for this use, you must go [https://search.nix.gsc.io/?q=makeWrapper%20%3D&i=nope&files=&repos= top-level hunting]). Then they will either auto-inject some steps into build phases (e.g. [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/development/tools/misc/pkgconfig/default.nix#L8 <code>pkgconfig</code>] or autoconf), or allow you to use some useful scripts during build easily (e.g. <code>makeWrapper</code> — see [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L130 wrapProgram] and [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L1 makeWrapper] docs for details, then [https://search.nix.gsc.io/?q=wrapProgram&i=nope&files=&repos= grep in nixpkgs] for numerous usage examples).
+Mostly yes. Some of them [https://github.com/NixOS/nixpkgs/blob/c3b2c5bf77f9d437b4656f0244bcdf2ab8d0102c/pkgs/stdenv/generic/default.nix#L101-L111 are automatically enabled by default]. Others you can enable on-demand, by adding them to your <code>buildInputs</code> — e.g. <code>buildInputs = [ makeWrapper ];</code> (to find the "proper" name for this use, you must go [https://search.nix.gsc.io/?q=makeWrapper%20%3D&i=nope&files=&repos= top-level hunting]). Then they will either auto-inject some steps into build phases (e.g. [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/development/tools/misc/pkgconfig/default.nix#L8 <code>pkg-config</code>] or autoconf), or allow you to use some useful scripts during build easily (e.g. <code>makeWrapper</code> — see [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L130 wrapProgram] and [https://github.com/NixOS/nixpkgs/blob/0e9e77750818f40303c72ad658b3dca299591e4f/pkgs/build-support/setup-hooks/make-wrapper.sh#L1 makeWrapper] docs for details, then [https://github.com/search?q=repo%3ANixOS%2Fnixpkgs%20wrapProgram&type=code grep in nixpkgs] for numerous usage examples).
 See also [https://github.com/NixOS/nixpkgs/tree/master/pkgs/build-support/setup-hooks <nixpkgs>/pkgs/build-support/setup-hooks] for a list of some common setup hooks.
 </blockquote>
 == What's the relationship between <code>nix-env</code> and <code>buildEnv</code> ==
 AFAICT, <code>nix-env</code> somehow assembles a <code>buildEnv</code> expression and the realization of that expression becomes a profile, somehow.
-# [https://github.com/NixOS/nix/blob/215b70f51e5abd350c9b7db656aedac9d96d0046/src/nix-env/user-env.cc#L108 '''nix'''/user-env/]
+# [https://github.com/NixOS/nix/blob/2c42a9dbaa805f4f29561d9a1c10b41dfe98dcfa/src/nix-env/user-env.cc#L123 '''nix'''/user-env/]
-# [https://github.com/NixOS/nix/blob/master/corepkgs/buildenv.nix '''nix'''/corepkgs/buildenv.nix]
+# [https://github.com/NixOS/nix/blob/master/src/nix-env/buildenv.nix '''nix'''/src/nix-env/buildenv.nix]
-# [https://github.com/NixOS/nix/tree/master/src/buildenv '''nix'''/src/buildenv/]
+# [https://github.com/NixOS/nix/blob/master/src/libstore/builtins/buildenv.cc '''nix'''/src/libstore/builtins/buildenv.cc]
-'''#1''' loads the nix file in '''#2''' and uses it to merge everything together, and '''#2''' uses a copy of '''#3''' that was compiled when nix got built
+'''#1''' loads the nix file in '''#2''' and uses it to merge everything together, and '''#2''' uses a copy of '''#3''' that was compiled when nix got built.
+Have a look also at [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/buildenv/default.nix '''nixpkgs'''/pkgs/build-support/buildenv/default.nix]
 == How to add a new cross-compilation target/platform? ==
@@ Line 37: / Line 36: @@
 In nixpkgs there is [https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/replace-dependency.nix replace-dependency] which provides this functionality but it is neither used nor explicitly documented as of right now (2017-06-22).
-<blockquote>'''Answer:''' There is an open [https://github.com/NixOS/nixpkgs/pull/10851/ pull request] and [https://theshortlog.com/2016/10/01/Nix-security-updates-update/ blog article] to implement security updates
+<blockquote>'''Answer:''' There is an open [https://github.com/NixOS/nixpkgs/pull/10851/ pull request] and [https://www.theshortlog.com/posts/nix-security-updates-update/ blog article] to implement security updates
 </blockquote>
+== Tips for using nix-shell for development ==
+<blockquote>
+mkShell is like stdenv.mkDerivationto dedicated for shell.nix. It adds one more feature where multiple package build inputs can all be merged together. This is useful when developing in a repo that has multiple packages (micro-services).
+</blockquote>
+== The structure and placement of nixpkgs ==
+Basically something that documents what all the directories mean and where things are placed and how they are structured and why.
+== A list of all the trivial builders and build support ==
+And an explanation of what they do and what they are used for.
+== How do manpages work? Or: <code>environment.pathsToLink</code> and <code>buildEnv</code> ==
+I'm trying to understand how manpages work in Nix/NixOS.
+There is a pacakge called [https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/misc/man-db/default.nix#L29 <code>man-db</code>], which seems to be looking in <code>/nix/var/nix/profiles/default/share/man</code> for manpages. It can build a cache at <code>/var/cache/man/nixpkgs</code>.
+Under NixOS (on my machine, running NixOS 20.09), these paths do not exist. <code>man-db</code> is installed as a NixOS module, the configuration is at [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/misc/documentation.nix#L104 <code>nixos/modules/misc/documentation.nix</code>] in the nixpkgs repository. Manpages are in <code>/run/current-system/sw/share/man</code>, and the cache resides in <code>/var/cache/man/nixos</code>.
+What I don't understand is: How are these paths populated? In the module, if <code>man</code> is enabled, <code>environment.pathsToLink</code> is set to <code>[ "/share/man" ]</code>. The documentation describes this option as "List of directories to be symlinked in /run/current-system/sw". Does this mean that for every package in <code>environment.systemPackages</code>, it does something like <code>for f in ${pkg}/share/man/* ; do ln -s $f /run/current-system/sw/share/man/$fname ; done</code>? I could only find that <code>environment.systemPackages</code> is later used to make a <code>buildEnv</code>, but what <code>buildEnv</code> does is a mystery to me.
+So, to summarize my ramblings, I guess I would find a thorough documentation of buildEnv very helpful.
+[[Category:Community]]