Channel branches: Difference between revisions

From NixOS Wiki
imported>Asymmetric
m →‎Channel release process: Remove redundant section (already in main body)
Klinger (talk | contribs)
 
(44 intermediate revisions by 30 users not shown)
Line 1: Line 1:
''Nix Channels'' are mechanisms for distributing Nix expressions as well as the associated binaries for them; a Nix channel corresponds to a repository in a conventional package management system. Official Nix channels are automatically updated once tests are passed in Nixpkgs' [[Hydra]] instance. It is also possible to create one's own Nix channels, but here we focus the official channels.
Nixpkgs is the Git repository containing all package recipes and NixOS module declarations. Installing packages directly from the <code>master</code> branch of the Nixpkgs repository is possible but risky, since Git commits are merged into <code>master</code> before being heavily tested. That is where channel branches are useful.
 
A "channel branch" is Git branch with the "verified" Git commits in Nixpkgs that is exposed at [https://channels.nixos.org channels.nixos.org]. There is also a [https://status.nixos.org channel status webpage] that tracks the age of channel branches.
 
Each channel branch follows a corresponding development branch to which new commits are first added. These new commits are then "verified" using the [[Hydra]] continuous integration system, where each channel branch corresponds to building any new or updated packages for that branch and perform the associated tests. A channel branch is updated once its builds succeeds for a new commit. Contrary to users of the development branches, channel branch users will benefit from both "verified" commits and pre-built packages from the [https://cache.nixos.org public binary cache].
 
There are several types of channel branches, each with its own use case and verification phase.


== The official channels ==
== The official channels ==


Packages for Nix, as well as modules for NixOS are distributed through a number of channels. These channels can be broadly categorized into ''stable'' and ''unstable'' channels, and ''large'' and ''small'' channels:
Channels can be broadly categorized into ''stable'' and ''unstable'' channels, and ''large'' and ''small'' channels:


* Stable/unstable:
* Stable/unstable:
** Stable channels (<code>nixos-17.09</code>) provide conservative updates for fixing bugs and security vulnerabilities, but do not receive major updates after initial release. New stable channels are released every six months.
** Stable channels (<code>nixos-24.05</code>) only provide conservative updates for fixing bugs and security vulnerabilities, but do not receive major updates after the initial release. New stable channels are released every six months.
** Unstable channels (<code>nixos-unstable</code>, <code>nixos-unstable-small</code>) correspond to the main development branch of Nixpkgs, delivering the latest tested updates on a rolling basis.
** Unstable channels (<code>nixos-unstable</code>, <code>nixpkgs-unstable</code>) follow the <code>master</code> branch of Nixpkgs, delivering the latest tested updates on a rolling basis.
 
* Large/small:
* Large/small:
** Large channels (<code>nixos-17.09</code>, <code>nixos-unstable</code>) provide binary builds for the full breadth of Nixpkgs.
** Large channels (<code>nixos-24.05</code>, <code>nixos-unstable</code>) are updated only after Hydra has finished building the full breadth of Nixpkgs.
** Small channels (<code>nixos-17.09-small</code>, <code>nixos-unstable-small</code>) are identical to large channels, but contain fewer binaries. This means they update faster, but require more to be built from source.
** Small channels (<code>nixos-24.05-small</code>, <code>nixos-unstable-small</code>) are identical to large channels, but are updated as soon as Hydra has finished building a defined set of commonly-used packages. Thus, users following these channels will get faster updates but may need to build any packages they use from outside the defined set themselves. These channels are intended to be used for server setups, for example.
 
Most users will want the stable/large channel, currently <code>nixos-24.05</code>.
 
NixOS uses [https://nixos.org/manual/nix/stable/command-ref/files/profiles Nix profiles] for the <code>root</code> user to refer different versions of the system-wide configuration. Profiles set for other users only control the user environment for that user.
 
== Channel update process ==
 
The channel update process begins when anyone with commit access pushes changes to either <code>master</code> or one of the <code>release-XX.XX</code> branches.
 
=== Hydra Build ===
 
Then, for each '''unstable''' channel (see above), a particular job at [https://hydra.nixos.org hydra.nixos.org] is started which must succeed:
 
* For NixOS: the [http://hydra.nixos.org/job/nixos/trunk-combined/tested trunk-combined/tested] job, which includes some automated NixOS tests.
*  For nixos-small: the [http://hydra.nixos.org/job/nixos/unstable-small/tested unstable-small/tested] job.
*  For nixpkgs: the [http://hydra.nixos.org/job/nixpkgs/trunk/unstable trunk/unstable] job, which contains some critical release packages.
 
=== Success Conditions ===
 
For a channel update to succeed, two conditions need to be satisfied:
 
* Particular jobset evaluation needs to be completely built ie. no more queued jobs, even if some jobs may fail
* Particular jobset evaluation's tested/unstable job needs to be built succesfully
 
The nixos.org server has a cronjob for which nixos-channel-scripts are executed and poll for the newest jobset that satisfies the above two conditions and trigger a channel update.


https://nixos.org/channels/ hosts the full list of the available official channels is available. Most users will want the stable/large channel, currently <code>nixos-17.09</code>.
=== Channel Update ===


Like packages installed via <code>nix-env</code>, channels are managed at user-level. NixOS uses the channels set for the <code>root</code> user to update the system-wide configuration; channels set for other users control only the user environment for that user. If you wish to change the channel used by the system-level configuration (<code>/etc/nixos/configuration.nix</code>), ensure you run the correct <code>nix-channel</code> command as root:
Once the job succeeds at a particular nixpkgs commit, cache.nixos.org will download binaries from [https://hydra.nixos.org hydra.nixos.org]. When the download completes, the channel updates.


{| class="wikitable"
For the <code>NixOS</code> channel command-not-found index is generated, which can take some time since it has to fetch all packages. <code>nixpkgs</code> is quickly updated since none of the above needs to happen once a channel update is triggered.
|+ Common nix-channel commands
|-
|Listing current channels
|<code>nix-channel --list</code>
|-
| Adding a primary channel
|<code><nowiki>nix-channel --add https://nixos.org/channels/channel-name nixos</nowiki></code>
|-
| Adding other channels
|<code><nowiki>nix-channel --add https://some.channel/url my-alias</nowiki></code>
|-
| Remove a channel
|<code>nix-channel --remove channel-alias</code>
|-
| Updating a channel
|<code>nix-channel --update channel-alias</code>
|-
| Updating all channels
|<code>nix-channel --update</code>
|}


Note that updating channels won't cause a rebuild in itself; if you want to update channels and rebuild, you can run <code>nixos rebuild --upgrade</code> to do both in one step.
Updates for the -unstable channels typically take a few days after commits land in the master branch.


== Channel release process ==
To find out when a channel was last updated, check [https://status.nixos.org/ https://status.nixos.org/]. The progress of a particular pull request can be tracked via the (third-party) [https://nixpk.gs/pr-tracker.html Nixpkgs Pull Request Tracker].


=== Unstable ===
== When unstable lags behind master ==


The unstable channels (<code>nixpkgs/unstable</code> and <code>nixos/unstable</code>) are built off of the <code>master</code> branch.
As https://status.nixos.org shows, a downside of nixos-unstable is that when the channel is blocked due to hydra failures, other (security) fixes will also not get in. While of course we try to keep hydra green, it is expected that this happens every once in a while. When you want to upgrade or downgrade a single package while leaving the rest of your system on nixos-unstable, you could use [[User:Raboof#using_a_fork_of_a_packaged_project|this approach]].


Once a PR gets merged, a job is triggered in [[Hydra]]. [https://hydra.nixos.org/job/nixpkgs/trunk/unstable#tabs-constituents This page] lists the 15 most recent builds in the <code>nixpkgs/unstable</code> channel, with the left-most column being the most recent.
== Check build status ==
[https://github.com/nix-community/hydra-check hydra-check]
<syntaxhighlight lang="sh">
$ hydra-check --channel unstable bash
Build Status for nixpkgs.bash.x86_64-linux on unstable
✔ bash-4.4-p23 from 2021-05-23 - https://hydra.nixos.org/build/143785213
</syntaxhighlight>
also useful for finding build logs


A build is considered successful when both of the following two conditions are true:
== See also ==


* All of the jobs in a column are green (meaning, the tests have succeeded)
* [https://nix.dev/concepts/faq#which-channel-branch-should-i-use nix.dev FAQ: Which channel branch should I use?]
* All of the jobs in the corresponding evaluation have finished (with a success or a failure)
* [https://samuel.dionne-riel.com/blog/2024/05/07/its-not-flakes-vs-channels.html It's not about “Flakes vs. Channels” by samueldr]


The evaluation for a build can be found by clicking on the column heading, and then following the the "Part of" link in the "Summary" tab.
[[Category:Nix]]
[[Category:NixOS]]
[[Category:Hydra]]
[[Category:Software]]

Latest revision as of 20:48, 1 July 2024

Nixpkgs is the Git repository containing all package recipes and NixOS module declarations. Installing packages directly from the master branch of the Nixpkgs repository is possible but risky, since Git commits are merged into master before being heavily tested. That is where channel branches are useful.

A "channel branch" is Git branch with the "verified" Git commits in Nixpkgs that is exposed at channels.nixos.org. There is also a channel status webpage that tracks the age of channel branches.

Each channel branch follows a corresponding development branch to which new commits are first added. These new commits are then "verified" using the Hydra continuous integration system, where each channel branch corresponds to building any new or updated packages for that branch and perform the associated tests. A channel branch is updated once its builds succeeds for a new commit. Contrary to users of the development branches, channel branch users will benefit from both "verified" commits and pre-built packages from the public binary cache.

There are several types of channel branches, each with its own use case and verification phase.

The official channels

Channels can be broadly categorized into stable and unstable channels, and large and small channels:

  • Stable/unstable:
    • Stable channels (nixos-24.05) only provide conservative updates for fixing bugs and security vulnerabilities, but do not receive major updates after the initial release. New stable channels are released every six months.
    • Unstable channels (nixos-unstable, nixpkgs-unstable) follow the master branch of Nixpkgs, delivering the latest tested updates on a rolling basis.
  • Large/small:
    • Large channels (nixos-24.05, nixos-unstable) are updated only after Hydra has finished building the full breadth of Nixpkgs.
    • Small channels (nixos-24.05-small, nixos-unstable-small) are identical to large channels, but are updated as soon as Hydra has finished building a defined set of commonly-used packages. Thus, users following these channels will get faster updates but may need to build any packages they use from outside the defined set themselves. These channels are intended to be used for server setups, for example.

Most users will want the stable/large channel, currently nixos-24.05.

NixOS uses Nix profiles for the root user to refer different versions of the system-wide configuration. Profiles set for other users only control the user environment for that user.

Channel update process

The channel update process begins when anyone with commit access pushes changes to either master or one of the release-XX.XX branches.

Hydra Build

Then, for each unstable channel (see above), a particular job at hydra.nixos.org is started which must succeed:

Success Conditions

For a channel update to succeed, two conditions need to be satisfied:

  • Particular jobset evaluation needs to be completely built ie. no more queued jobs, even if some jobs may fail
  • Particular jobset evaluation's tested/unstable job needs to be built succesfully

The nixos.org server has a cronjob for which nixos-channel-scripts are executed and poll for the newest jobset that satisfies the above two conditions and trigger a channel update.

Channel Update

Once the job succeeds at a particular nixpkgs commit, cache.nixos.org will download binaries from hydra.nixos.org. When the download completes, the channel updates.

For the NixOS channel command-not-found index is generated, which can take some time since it has to fetch all packages. nixpkgs is quickly updated since none of the above needs to happen once a channel update is triggered.

Updates for the -unstable channels typically take a few days after commits land in the master branch.

To find out when a channel was last updated, check https://status.nixos.org/. The progress of a particular pull request can be tracked via the (third-party) Nixpkgs Pull Request Tracker.

When unstable lags behind master

As https://status.nixos.org shows, a downside of nixos-unstable is that when the channel is blocked due to hydra failures, other (security) fixes will also not get in. While of course we try to keep hydra green, it is expected that this happens every once in a while. When you want to upgrade or downgrade a single package while leaving the rest of your system on nixos-unstable, you could use this approach.

Check build status

hydra-check

$ hydra-check --channel unstable bash
Build Status for nixpkgs.bash.x86_64-linux on unstable
✔ bash-4.4-p23 from 2021-05-23 - https://hydra.nixos.org/build/143785213

also useful for finding build logs

See also