Continuous Integration (CI): Difference between revisions

(19 intermediate revisions by 10 users not shown)

Line 7:

<code>nix-build</code> will always ensure the built store path is put in the local store, be it by building or by downloading from a substituter. On CI, we often only want to check whether we can build the derivation, without using or running the output.

This can be achieved by using `--dry-run` to check whether the result would be fetched, and only building it it has to be built.

This can be achieved by using <code>--dry-run</code> to check whether the result would be fetched, and only building it it has to be built.

[https://gist.github.com/Profpatsch/fbbc9a0006e246076f11efca0387d293 nix-build-if-changed.py] implements this in a relatively straightforward (but naïve) Python script.

[https://github.com/Mic92/nix-build-uncached/ nix-build-uncached] implements it in a slightly more elaborate manner, and is available on nixpkgs (as the <code>nix-build-uncached</code> package).

== Caching built results ==

After building your project you might want to cache the results. The cache server could be a [https://cachix.org/ Cachix] cache, a self-hosted [https://github.com/zhaofengli/attic Attic] cache or even your own nix machine.

[https://github.com/Mic92/nix-fast-build nix-fast-build] uses <code>nix-eval-jobs</code> in parallel to speed-up the evaluation and building process. It's useful for building flakes that have multiple outputs. It also supports uploading to [https://cachix.org/ Cachix] and [https://github.com/zhaofengli/attic Attic].

You can also use <code>nix-copy-closure</code> to directly upload to a remote /nix/store through a SSH connection. As it's already built in Nix, It's the simplest way to cache the results. Albeit, in my experience it's slower.[citation needed]

== Instructions for specific CI Providers ==

Line 23:

Line 31:

See [https://github.com/cachix/install-nix-action install-nix-action] to install nix in Linux/macOS actions.

There is also [https://github.com/cachix/cachix-action one action] to setup [https://cachix.org/ cachix], a hosted binary cache.

==== Self-hosted runners ====

NixOS has a few unofficial modules for running self-hosted GitHub runners. see [https://nix-community.github.io/srvos/github_actions_runner srvos], [https://github.com/juspay/github-nix-ci juspay/github-nix-ci]

=== Build kite ===

NixOS ~~comes with a~~ module ~~to run~~ [https://~~buildkite~~.com ~~build-kite~~] ~~agents~~:

See the [[Buildkite]] article

=== Drone ===

There is no official NixOS module however both drone and drone-cli are packaged in nixpkgs.

Mic92 has the following [https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modules/drone/ custom module] in his repository. An example public project can be found in [https://github.com/Mic92/cntr/blob/e1b58463d55567bd795d1f3cfb4c6784f57c895a/.drone.yml cntr] and this project for an example using flakes [https://github.com/Mic92/dotfiles/blob/05daca459d96557eeeb72196258e6bffc73c1360/.drone.yml Mic92's dotfiles].

=== Jenkins ===

[[Jenkins]]

=== Gitlab ===

NixOS has a module for [[Gitlab runner]]

=== Garnix ===

[https://garnix.io/ Garnix] is a nix-specific CI provider that also provides a cache. it currently only works with flake-enabled repos.

=== Sourcehut ===

~~<syntaxHighlight lang=nix>~~

[https://sourcehut.org Sourcehut] provides [https://man.sr.ht/builds.sr.ht/compatibility.md#nixos an official NixOS image]

{

~~services.buildkite-agents.builder = {~~

~~enable = true;~~

~~# store a token provided in the buildkite webinterface in the `Agents` tab under `Agents token`~~

~~tokenPath = "~~/~~path~~/to/~~token";~~

~~privateSshKeyPath = "~~/~~path~~/to/~~ssh/key";~~

};

}

~~</syntaxHighlight>~~

[~~https~~:~~//nixos.org/nixos/options.html#services.buildkite Further NixOS options~~]

[[Category:Server]]

[[Category:Applications]]

@@ Line 7: / Line 7: @@
 <code>nix-build</code> will always ensure the built store path is put in the local store, be it by building or by downloading from a substituter. On CI, we often only want to check whether we can build the derivation, without using or running the output.
-This can be achieved by using `--dry-run` to check whether the result would be fetched, and only building it it has to be built.
+This can be achieved by using <code>--dry-run</code> to check whether the result would be fetched, and only building it it has to be built.
 [https://gist.github.com/Profpatsch/fbbc9a0006e246076f11efca0387d293 nix-build-if-changed.py] implements this in a relatively straightforward (but naïve) Python script.
 [https://github.com/Mic92/nix-build-uncached/ nix-build-uncached] implements it in a slightly more elaborate manner, and is available on nixpkgs (as the <code>nix-build-uncached</code> package).
+== Caching built results ==
+After building your project you might want to cache the results. The cache server could be a [https://cachix.org/ Cachix] cache, a self-hosted [https://github.com/zhaofengli/attic Attic] cache or even your own nix machine.
+[https://github.com/Mic92/nix-fast-build nix-fast-build] uses <code>nix-eval-jobs</code> in parallel to speed-up the evaluation and building process. It's useful for building flakes that have multiple outputs. It also supports uploading to [https://cachix.org/ Cachix] and [https://github.com/zhaofengli/attic Attic].
+You can also use <code>nix-copy-closure</code> to directly upload to a remote /nix/store through a SSH connection. As it's already built in Nix, It's the simplest way to cache the results. Albeit, in my experience it's slower.[citation needed]
 == Instructions for specific CI Providers ==
@@ Line 23: / Line 31: @@
 See [https://github.com/cachix/install-nix-action install-nix-action] to install nix in Linux/macOS actions.
 There is also [https://github.com/cachix/cachix-action one action] to setup [https://cachix.org/ cachix], a hosted binary cache.
+==== Self-hosted runners ====
+NixOS has a few unofficial modules for running self-hosted GitHub runners. see [https://nix-community.github.io/srvos/github_actions_runner srvos], [https://github.com/juspay/github-nix-ci juspay/github-nix-ci]
 === Build kite ===
-NixOS comes with a module to run [https://buildkite.com build-kite] agents:
+See the [[Buildkite]] article
+=== Drone ===
+There is no official NixOS module however both drone and drone-cli are packaged in nixpkgs.
+Mic92 has the following [https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modules/drone/ custom module] in his repository. An example public project can be found in [https://github.com/Mic92/cntr/blob/e1b58463d55567bd795d1f3cfb4c6784f57c895a/.drone.yml cntr] and this project for an example using flakes [https://github.com/Mic92/dotfiles/blob/05daca459d96557eeeb72196258e6bffc73c1360/.drone.yml Mic92's dotfiles].
+=== Jenkins ===
+[[Jenkins]]
+=== Gitlab ===
+NixOS has a module for [[Gitlab runner]]
+=== Garnix ===
+[https://garnix.io/ Garnix] is a nix-specific CI provider that also provides a cache. it currently only works with flake-enabled repos.
+=== Sourcehut ===
-<syntaxHighlight lang=nix>
+[https://sourcehut.org Sourcehut] provides [https://man.sr.ht/builds.sr.ht/compatibility.md#nixos an official NixOS image]
-{
-  services.buildkite-agents.builder = {
-    enable = true;
-    # store a token provided in the  buildkite webinterface in the `Agents` tab under `Agents token`
-    tokenPath = "/path/to/token";
-    privateSshKeyPath = "/path/to/ssh/key";
-  };
-}
-</syntaxHighlight>
-[https://nixos.org/nixos/options.html#services.buildkite Further NixOS options]
+[[Category:Server]]
+[[Category:Applications]]