Continuous Integration (CI): Difference between revisions
imported>Srid Self-hosted runner |
Added a section about caching binary results in CI |
||
| (3 intermediate revisions by 3 users not shown) | |||
| Line 7: | Line 7: | ||
<code>nix-build</code> will always ensure the built store path is put in the local store, be it by building or by downloading from a substituter. On CI, we often only want to check whether we can build the derivation, without using or running the output. | <code>nix-build</code> will always ensure the built store path is put in the local store, be it by building or by downloading from a substituter. On CI, we often only want to check whether we can build the derivation, without using or running the output. | ||
This can be achieved by using | This can be achieved by using <code>--dry-run</code> to check whether the result would be fetched, and only building it it has to be built. | ||
[https://gist.github.com/Profpatsch/fbbc9a0006e246076f11efca0387d293 nix-build-if-changed.py] implements this in a relatively straightforward (but naïve) Python script. | [https://gist.github.com/Profpatsch/fbbc9a0006e246076f11efca0387d293 nix-build-if-changed.py] implements this in a relatively straightforward (but naïve) Python script. | ||
[https://github.com/Mic92/nix-build-uncached/ nix-build-uncached] implements it in a slightly more elaborate manner, and is available on nixpkgs (as the <code>nix-build-uncached</code> package). | [https://github.com/Mic92/nix-build-uncached/ nix-build-uncached] implements it in a slightly more elaborate manner, and is available on nixpkgs (as the <code>nix-build-uncached</code> package). | ||
== Caching built results == | |||
After building your project you might want to cache the results. The cache server could be a [https://cachix.org/ Cachix] cache, a self-hosted [https://github.com/zhaofengli/attic Attic] cache or even your own nix machine. | |||
[https://github.com/Mic92/nix-fast-build nix-fast-build] uses <code>nix-eval-jobs</code> in parallel to speed-up the evaluation and building process. It's useful for building flakes that have multiple outputs. It also supports uploading to [https://cachix.org/ Cachix] and [https://github.com/zhaofengli/attic Attic]. | |||
You can also use <code>nix-copy-closure</code> to directly upload to a remote /nix/store through a SSH connection. As it's already built in Nix, It's the simplest way to cache the results. Albeit, in my experience it's slower.[citation needed] | |||
== Instructions for specific CI Providers == | == Instructions for specific CI Providers == | ||
| Line 26: | Line 34: | ||
==== Self-hosted runners ==== | ==== Self-hosted runners ==== | ||
NixOS has a | NixOS has a few unofficial modules for running self-hosted GitHub runners. see [https://nix-community.github.io/srvos/github_actions_runner srvos], [https://github.com/juspay/github-nix-ci juspay/github-nix-ci] | ||
=== Build kite === | === Build kite === | ||
| Line 52: | Line 60: | ||
[https://sourcehut.org Sourcehut] provides [https://man.sr.ht/builds.sr.ht/compatibility.md#nixos an official NixOS image] | [https://sourcehut.org Sourcehut] provides [https://man.sr.ht/builds.sr.ht/compatibility.md#nixos an official NixOS image] | ||
[[Category:Server]] | |||
[[Category:Applications]] | |||
Latest revision as of 01:50, 15 March 2025
Nix nicely integrates into your CI.
TODO: some general tips & tricks
Avoid downloading already built results
nix-build will always ensure the built store path is put in the local store, be it by building or by downloading from a substituter. On CI, we often only want to check whether we can build the derivation, without using or running the output.
This can be achieved by using --dry-run to check whether the result would be fetched, and only building it it has to be built.
nix-build-if-changed.py implements this in a relatively straightforward (but naïve) Python script.
nix-build-uncached implements it in a slightly more elaborate manner, and is available on nixpkgs (as the nix-build-uncached package).
Caching built results
After building your project you might want to cache the results. The cache server could be a Cachix cache, a self-hosted Attic cache or even your own nix machine.
nix-fast-build uses nix-eval-jobs in parallel to speed-up the evaluation and building process. It's useful for building flakes that have multiple outputs. It also supports uploading to Cachix and Attic.
You can also use nix-copy-closure to directly upload to a remote /nix/store through a SSH connection. As it's already built in Nix, It's the simplest way to cache the results. Albeit, in my experience it's slower.[citation needed]
Instructions for specific CI Providers
Travis CI
See Nix_on_Travis article
Github actions
See install-nix-action to install nix in Linux/macOS actions. There is also one action to setup cachix, a hosted binary cache.
Self-hosted runners
NixOS has a few unofficial modules for running self-hosted GitHub runners. see srvos, juspay/github-nix-ci
Build kite
See the Buildkite article
Drone
There is no official NixOS module however both drone and drone-cli are packaged in nixpkgs.
Mic92 has the following custom module in his repository. An example public project can be found in cntr and this project for an example using flakes Mic92's dotfiles.
Jenkins
Gitlab
NixOS has a module for Gitlab runner
Garnix
Garnix is a nix-specific CI provider that also provides a cache. it currently only works with flake-enabled repos.
Sourcehut
Sourcehut provides an official NixOS image