Nixpkgs/Create and debug packages: Difference between revisions

imported>Bencoman
imported>Bencoman
Undo revision 1221 by Bencoman (talk)
Line 1: Line 1:
This article describes how to work with the nix related repositories to add new packages, edit and debug existing packages. For details on the NixOS module system see [[NixOS:Modules]]. [[NixOS:extend_NixOS]] explains how to write, test and debug your own modules.
There is a chapter about hacking packages and modules in the NixOS manual: http://nixos.org/nixos/manual/index.html#ch-development
Writing packages is covered in http://nixos.org/nixpkgs/manual and writing modules in http://nixos.org/nixos/manual
The nix repositories are hosted here: https://github.com/nixos
== Basics ==
The code for nix packages is managed in the nixpkgs repository. NixOS services, and other system configuration options are managed in the nixos sub-directory of the nixpkgs repository.
The steps to take for your first change should look something like this:
# Fork the repo (e.g. click the fork button on https://github.com/nixos/nixpkgs).
# Clone your fork <code>git clone git@github.com:YOUR-GITHUB-ACCOUNT-NAME/nixpkgs.git</code>
# Hack hack hack
# Push your changes to your fork
# Open a pull request
# Profit!
This is pretty much the standard way to use github, so if you have trouble using git or github any general guide on these should get you going, or just ask on the NixOS IRC channel. The rest of this guide deals with the "Hack hack hack" step :)
== How to install from the local repository ==
For expediency just for this article, we'll shallow clone direct from the distribution repo and set an environment variable pointing to it.
<syntaxhighlight lang="bash">
$ mkdir -p ~/tmpdev && cd ~/tmpdev
$ git clone --depth=1 https://github.com/nixos/nixpkgs
$ export NIXPKGS=~/tmpdev/nixpkgs
$ ls $NIXPKGS
</syntaxhighlight>
make some changes ...
'''example: list all available software''' from the local repository
$NIXPKGS
<syntaxhighlight lang="console">
$ nix-env -f $NIXPKGS -qaP '*'
</syntaxhighlight>
'''example: install software from local repository'''
<syntaxhighlight lang="console">
$ nix-env -f $NIXPKGS -i python-urlgrabber
</syntaxhighlight>
'''example: update the system''' based on your local '''$NIXPKGS'''
<syntaxhighlight lang="console">
$ nixos-rebuild -I nixos=$NIXPKGS/nixos -I nixpkgs=$NIXPKGS switch
</syntaxhighlight>
'''example: build an expression and put the output in to `pwd`/results'''
<syntaxhighlight lang="console">
$ nix-build $NIXPKGS -A irssi
</syntaxhighlight>
'''example: get an environment which is used to build irssi (also see nix-shell)'''
<syntaxhighlight lang="console">
$ nix-build $NIXPKGS --run-env -A irssi
</syntaxhighlight>
'''example: get a persistent environment which is used to build irssi'''
<syntaxhighlight lang="console">
$ nix-build $NIXPKGS --run-env -A irssi --add-root
</syntaxhighlight>
== Tracking upstream changes and avoiding extra rebuilding ==
You have forked the relevant nix repository, but you will want to track changes in the upstream nix repo too. You can add a remote, and a corresponding branch for this.
<syntaxhighlight lang="console">
$ git remote add upstream https://github.com/NixOS/nixpkgs.git
</syntaxhighlight>
You can create a branch to track the upstream master branch:
<syntaxhighlight lang="console">
$ git checkout -b upstream-master upstream/master
$ git pull
</syntaxhighlight>
This will put you into a branch with all the latest changes. Hydra, the build farm, regularly creates binaries, but, since people are constantly contributing to the nix repositories, it is usually the case that there are changes in the master branch which have not yet made it into the binary channel. To take advantage of available binaries you can switch to the revision which produced the binaries in your current system and apply your changes from there. You can use `nixos-version` to see the relevant short revision hash:
<syntaxhighlight lang="console">
$ nixos-version
14.11pre52727.5d97886 (Caterpillar)
${NixOS release}.${nixpkgs revision}
(since the git-repo called nixos was merged into nixpkgs)
</syntaxhighlight>
 
<syntaxhighlight lang="console">
$ nixos-version
13.07pre4871_18de9f6-3c35dae (Aardvark)
${NixOS release}_${NixOS revision}-${nixpkgs revision}
</syntaxhighlight>
This string shows the Nixos release number (13.07pre4871) followed by the nixos revision used to produce your current system (18de9f6) followed by the nixpkgs revision (3c35dae).
<syntaxhighlight lang="console">
$ git branch
upstream-master
$ git checkout -b nixpkgs-channel 3c35dae
Switched to a new branch 'nixpkgs-channel'
$ git checkout -b my-new-pkg
Switched to a new branch 'my-new-pkg'
</syntaxhighlight>
After making some changes you can commit them into your local repo:
<syntaxhighlight lang="console">
$ git add foo
$ git commit
</syntaxhighlight>
Then you push your changes to your fork:
<syntaxhighlight lang="console">
$ git push origin my-new-pkg
</syntaxhighlight>
You can use this to open a pull request on github.
If some time has passed since you have created your fork, you will want to merge your changes with upstream and test that it still works.
<syntaxhighlight lang="console">
$ git fetch upstream
$ git merge upstream
</syntaxhighlight>
If your merge then fails because someone else has made the same change (for example, someone else also packaged a library you have just packed for the program you want to get into nixpkgs), then you can do this:
<syntaxhighlight lang="console">
$ git rebase -i HEAD~10
</syntaxhighlight>
there select the edit mode for your commit and remove the your code which added the library. **Warning: only use 'git rebase' on your commits, which have not been pushed and nobody else is working with already!**
Next you have to test if your program works with the library packaged from someone else, then do:
<syntaxhighlight lang="console">
$ git checkout master
$ git log --stat
</syntaxhighlight>
and pick the commit where the library was added. Finally cherry-pick that commit into your branch:
<syntaxhighlight lang="console">
$ git checkout my-new-pkg
$ git cherry-pick 5d97886a6a545fb20495e0837cc50fa63d2a80e1
</syntaxhighlight>
Afterwards do your usual tests and if needed also make modifications to the library but keep in mind that this might break the other use-case of that library and if in doubt check that as well.
This article describes how to work with the nix related repositories to add new packages, edit and debug existing packages. For details on the NixOS module system see [[NixOS:Modules]]. [[NixOS:extend_NixOS]] explains how to write, test and debug your own modules.
This article describes how to work with the nix related repositories to add new packages, edit and debug existing packages. For details on the NixOS module system see [[NixOS:Modules]]. [[NixOS:extend_NixOS]] explains how to write, test and debug your own modules.


Line 449: Line 301:


Tip: A git repository can be used for snapshotting attempts at building the package. This also makes it easy to generate patches, should you need to.
Tip: A git repository can be used for snapshotting attempts at building the package. This also makes it easy to generate patches, should you need to.
== nix channels ==
nix channels can be used in parallel with your new local repositories, see its [[install/remove software#nix-channels| nix-channel-documentation]]
== Testing Package Updates with Nox ==
If you are updating a package's version, you can use nox to make sure all packages that depend on the updated package still compile correctly.
First make sure it is in your environment:
<syntaxhighlight lang="bash">
nix-env -i nox
</syntaxhighlight>
You can run nox against uncommited changes to a nixpkgs repository:
<syntaxhighlight lang="bash">
cd ~/.nix-defexpr
nox-review wip
</syntaxhighlight>
If you have already commited your changes and created a pull request, you can use the pr command:
<syntaxhighlight lang="bash">
nox-review pr 5341
</syntaxhighlight>
== See also ==
* [[Generic Algorithm on Doing Packaging]]
[[Category:Development]]
[[Category:NixOS]]
[[Category:Nixpkgs]]
[[Category:Guide]]


== nix channels ==
== nix channels ==