Hydra/de: Difference between revisions
Created page with "Diese Konfiguration erzeugt die Datei /etc/nix/machines. Falls die Hydra-option <code>buildMachinesFiles</code> immer noch auf eine leere Liste gesetzt ist, wird diese Einstellung ignoriert. Deswegen muss diese Option wieder entfernt werden oder <code>/etc/nix/machines</code> hinzugefügt werden." |
No edit summary |
||
(24 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
<languages /> | <languages /> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
{{note| Hydra is intended to be used by Nix/NixOS package developers; it is not needed to simply use NixOS.}} | |||
</div> | |||
Hydra ist ein Werkzeug für kontinuierliche Integrationstests und Softwarefreigabe, das eine rein funktionale Sprache zur Beschreibung von Build-Jobs und deren Abhängigkeiten verwendet. Kontinuierliche Integration ist eine einfache Technik zur Verbesserung der Qualität des Softwareentwicklungsprozesses. Ein automatisiertes System prüft kontinuierlich oder periodisch den Quellcode eines Projekts, baut ihn, führt Tests durch und erstellt Berichte für die Entwickler. Auf diese Weise werden verschiedene Fehler, die versehentlich in die Codebasis aufgenommen werden könnten, automatisch erkannt. | Hydra ist ein Werkzeug für kontinuierliche Integrationstests und Softwarefreigabe, das eine rein funktionale Sprache zur Beschreibung von Build-Jobs und deren Abhängigkeiten verwendet. Kontinuierliche Integration ist eine einfache Technik zur Verbesserung der Qualität des Softwareentwicklungsprozesses. Ein automatisiertes System prüft kontinuierlich oder periodisch den Quellcode eines Projekts, baut ihn, führt Tests durch und erstellt Berichte für die Entwickler. Auf diese Weise werden verschiedene Fehler, die versehentlich in die Codebasis aufgenommen werden könnten, automatisch erkannt. | ||
Line 15: | Line 19: | ||
services.hydra = { | services.hydra = { | ||
enable = true; | enable = true; | ||
hydraURL = "http://localhost:3000"; # | hydraURL = "http://localhost:3000"; # externally visible URL | ||
notificationSender = "hydra@localhost"; # | notificationSender = "hydra@localhost"; # e-mail of Hydra service | ||
# | # a standalone Hydra will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines | ||
buildMachinesFiles = []; | buildMachinesFiles = []; | ||
# | # you will probably also want, otherwise *everything* will be built from scratch | ||
useSubstitutes = true; | useSubstitutes = true; | ||
}; | }; | ||
</syntaxHighlight> | </syntaxHighlight> | ||
Dieses Modul aktiviert PostgreSQL automatisch, | Dieses Modul aktiviert [[PostgreSQL|PostgreSQL]] automatisch, | ||
außer die <code>services.hydra.dbi</code> wird geändert. Das Datenbanklayout wird automatisch erstellt vom Hydra-Service. Allerdings ist anzumerken, das zusätzliche Daten in der Datenbank gespeichert werden, was eine vollständige deklarative Konfiguration unmöglich macht. Daher sind Backups erforderlich. | außer die <code>services.hydra.dbi</code> wird geändert. Das Datenbanklayout wird automatisch erstellt vom Hydra-Service. Allerdings ist anzumerken, das zusätzliche Daten in der Datenbank gespeichert werden, was eine vollständige deklarative Konfiguration unmöglich macht. Daher sind Backups erforderlich. | ||
* Siehe <code>nixos-option</code> oder die [https://search.nixos.org/options?query=services.hydra Nixos Optionsseite] für eine Liste aller Optionen. | * Siehe <code>nixos-option</code> oder die [https://search.nixos.org/options?query=services.hydra Nixos Optionsseite] für eine Liste aller Optionen. | ||
<span id="Web_Configuration"></span> | |||
=== Webserverkonfiguration === | === Webserverkonfiguration === | ||
Hydra | |||
[http://localhost:3000/ | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
Hydra will provide the web interface [http://localhost:3000/ at localhost] port 3000. However you need to create a new admin user (as UNIX user <code>hydra</code>) before being able to perform any changes: | |||
</div> | |||
<syntaxHighlight lang=bash> | <syntaxHighlight lang=bash> | ||
Line 40: | Line 46: | ||
</syntaxHighlight> | </syntaxHighlight> | ||
<span id="Virtual_machine"></span> | |||
=== Virtuelle Maschinen === | === Virtuelle Maschinen === | ||
Hydra benutzt localhost als Standardbuildmaschine, | Hydra benutzt localhost als Standardbuildmaschine, | ||
wenn nichts anderes konfiguriert ist. | wenn nichts anderes konfiguriert ist. | ||
Line 50: | Line 58: | ||
nix.buildMachines = [ | nix.buildMachines = [ | ||
{ hostName = "localhost"; | { hostName = "localhost"; | ||
protocol = null; | |||
system = "x86_64-linux"; | system = "x86_64-linux"; | ||
supportedFeatures = ["kvm" "nixos-test" "big-parallel" "benchmark"]; | supportedFeatures = ["kvm" "nixos-test" "big-parallel" "benchmark"]; | ||
Line 62: | Line 71: | ||
Falls die Hydra-option <code>buildMachinesFiles</code> immer noch auf eine leere Liste gesetzt ist, wird diese Einstellung ignoriert. Deswegen muss diese Option wieder entfernt werden oder <code>/etc/nix/machines</code> hinzugefügt werden. | Falls die Hydra-option <code>buildMachinesFiles</code> immer noch auf eine leere Liste gesetzt ist, wird diese Einstellung ignoriert. Deswegen muss diese Option wieder entfernt werden oder <code>/etc/nix/machines</code> hinzugefügt werden. | ||
< | <span id="Flake_jobset"></span> | ||
== Flake | == Flake-Jobset == | ||
Ein Jobset kann wie folgt konfiguriert werden. | |||
* Type: [[Flakes|Flake]] | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
* Flake URI: an URI to a repo containing a Flake like git+https://git.myserver.net/user/repo.git | * Flake URI: an URI to a repo containing a Flake like git+https://git.myserver.net/user/repo.git | ||
</div> | </div> | ||
Der Flake-Output sollte das Attribut <code>hydraJobs</code> enthalten. <code>hydraJobs</code> ist ein Attributset, das verschachtelt sein kann und auf Derivations verweist. | |||
</ | |||
Ein Beispiel für ein Flake-Output, das Hydra veranlasst, alle Pakete zu bauen, könnte wie folgt aussehen: | |||
<syntaxHighlight lang=nix> | <syntaxHighlight lang=nix> | ||
{ | { | ||
Line 93: | Line 95: | ||
... | ... | ||
}; | }; | ||
hydraJobs = { | |||
hydraJobs = { | |||
inherit (self) | inherit (self) | ||
packages; | packages; | ||
Line 103: | Line 103: | ||
} | } | ||
</syntaxHighlight> | </syntaxHighlight> | ||
< | <span id="Restricted_Mode"></span> | ||
=== Restricted | === Eingeschränkter Modus (Restricted mode) === | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
Line 113: | Line 111: | ||
</div> | </div> | ||
<syntaxHighlight lang=nix> | <syntaxHighlight lang=nix> | ||
nix.settings.allowed-uris = [ | nix.settings.allowed-uris = [ | ||
Line 121: | Line 118: | ||
]; | ]; | ||
</syntaxHighlight> | </syntaxHighlight> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
== Build a single Package from nixpkgs == | |||
</div> | </div> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
Right now it is not possible to build a single package from nixpkgs with just that input. You will need to provide a supplementary repository which defines what to build. For examples you can check the [https://github.com/makefu/hydra-example hydra-example by makefu] and in the [https://nixos.org/hydra/manual/#idm140737315920320 Hydra Manual]. | Right now it is not possible to build a single package from nixpkgs with just that input. You will need to provide a supplementary repository which defines what to build. For examples you can check the [https://github.com/makefu/hydra-example hydra-example by makefu] and in the [https://nixos.org/hydra/manual/#idm140737315920320 Hydra Manual]. | ||
</div> | </div> | ||
Line 130: | Line 129: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
=== Imperative Building === | === Imperative Building === | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
These steps are required to build the <code>hello</code> package. | These steps are required to build the <code>hello</code> package. | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
# log into Hydra after creating a user with <code>hydra-create-user</code> | # log into Hydra after creating a user with <code>hydra-create-user</code> | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
# create new project | # create new project | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* identifier: example-hello | * identifier: example-hello | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* display name: example-hello | * display name: example-hello | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
# Actions -> Create jobset | # Actions -> Create jobset | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* identifier: hello | * identifier: hello | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* Nix expression: <code>release.nix</code> in <code>hydra-example</code> -> will evaluate the file release.nix in the given input | * Nix expression: <code>release.nix</code> in <code>hydra-example</code> -> will evaluate the file release.nix in the given input | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* check interval: 60 | * check interval: 60 | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* scheduling shares: 1 | * scheduling shares: 1 | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* Inputs: | * Inputs: | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
Line 157: | Line 192: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
=== Declarative Building === | === Declarative Building === | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
Since 2016, Hydra supports declarative creation of jobsets. Check out the [https://github.com/shlevy/declarative-hydra-example example repository and description by Shea Levy]. | Since 2016, Hydra supports declarative creation of jobsets. Check out the [https://github.com/shlevy/declarative-hydra-example example repository and description by Shea Levy]. | ||
</div> | </div> | ||
Line 162: | Line 200: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
== Hydra Internals == | == Hydra Internals == | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
=== Definitions === | === Definitions === | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
This subsection provides an overview of the Hydra-specific definitions and how to configure them. | This subsection provides an overview of the Hydra-specific definitions and how to configure them. | ||
</div> | </div> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
==== Project ==== | ==== Project ==== | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
A cluster of Jobs which are all coming from a single input (like a git checkout), the first thing you will need to create. Every Job should be able to be built independently from another. Most of the time the project maps to a single repository like <code>nixpkgs</code>. It is comparable to the project definition in Jenkins. | A cluster of Jobs which are all coming from a single input (like a git checkout), the first thing you will need to create. Every Job should be able to be built independently from another. Most of the time the project maps to a single repository like <code>nixpkgs</code>. It is comparable to the project definition in Jenkins. | ||
</div> | </div> | ||
< | <span id="Jobsets"></span> | ||
==== Jobset ==== | ==== Jobset ==== | ||
Eine Liste von Jobs, die ausgeführt werden sollen. Oft passt ein Jobset zu einem bestimmten Branch (master, staging, stable). Ein Jobset wird durch seine Eingaben definiert und wird ausgelöst, wenn sich diese Eingaben ändern, z.B. wenn ein neuer Commit zu einem Branch hinzugefügt wird. Jobsets können voneinander abhängen. | |||
==== Job ==== | ==== Job ==== | ||
Ein Closure, welches als Teil eines Jobsatzes erstellt wird (wie ein einzelnes Paket, ISO-Image oder Tarball) | |||
Ein Closure, welches als Teil eines Jobsatzes erstellt wird (wie ein einzelnes Paket, ISO-Image oder Tarball). | |||
==== Release Set ==== | ==== Release Set ==== | ||
Definiert alle Jobs, die in Ihrem Release beschrieben sind. Konventionell wird eine Datei namens <code>release.nix</code> verwendet. Eine ausführliche Beschreibung der Struktur finden Sie im [https://nixos.org/hydra/manual/#idm140737315920320 Hydra Handbuch für Build Recipes]. | Definiert alle Jobs, die in Ihrem Release beschrieben sind. Konventionell wird eine Datei namens <code>release.nix</code> verwendet. Eine ausführliche Beschreibung der Struktur finden Sie im [https://nixos.org/hydra/manual/#idm140737315920320 Hydra Handbuch für Build Recipes]. | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
==== Evaluation ==== | ==== Evaluation ==== | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
The process of interpreting nix code into a list of <code>.drv files</code>. These files are the build recipes for all related outputs. You can introspect these files by running <code>nix show-derivation nixpkgs.hello</code>. | The process of interpreting nix code into a list of <code>.drv files</code>. These files are the build recipes for all related outputs. You can introspect these files by running <code>nix show-derivation nixpkgs.hello</code>. | ||
</div> | </div> | ||
Line 191: | Line 241: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
==== Build ==== | ==== Build ==== | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
Instantiation of a Job which is being triggered by being part of the release set. | Instantiation of a Job which is being triggered by being part of the release set. | ||
</div> | </div> | ||
Line 196: | Line 249: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
== Known Issues == | == Known Issues == | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* hydra-queue-runner sometimes gets stuck even with builds are in the queue, and the builds are not scheduled. The issue is being tracked [https://github.com/NixOS/hydra/issues/366 here]. In the meantime, a workaround is to add a cron job that regularly restarts the hydra-queue-runner systemd service. Possible fix: [https://github.com/NixOS/hydra/commit/73ca325d1c0f7914640a63764c9a6d448fde5bd0] | * hydra-queue-runner sometimes gets stuck even with builds are in the queue, and the builds are not scheduled. The issue is being tracked [https://github.com/NixOS/hydra/issues/366 here]. In the meantime, a workaround is to add a cron job that regularly restarts the hydra-queue-runner systemd service. Possible fix: [https://github.com/NixOS/hydra/commit/73ca325d1c0f7914640a63764c9a6d448fde5bd0] | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* If you see <code>error: unexpected end-of-file</code> it can mean multiple things, some of them are: | * If you see <code>error: unexpected end-of-file</code> it can mean multiple things, some of them are: | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
# You have a miss-match between nix versions on the Hydra server and the builder | # You have a miss-match between nix versions on the Hydra server and the builder | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
# It can also mean that <code>hydra-queue-runner</code> needs privileges on the build server. Reference: [https://github.com/NixOS/nix/issues/2789] | # It can also mean that <code>hydra-queue-runner</code> needs privileges on the build server. Reference: [https://github.com/NixOS/nix/issues/2789] | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* The default timeout for git operations is 600 seconds [https://github.com/NixOS/hydra/issues/1181], which might cause fetches of large repositories like [https://github.com/NixOS/nixpkgs nixos/nixpkgs] to fail: <code>error fetching latest change from git repo at `https://github.com/nixos/nixpkgs': timeout</code>. The timeout can be increased with the following configuration.nix snippet: | * The default timeout for git operations is 600 seconds [https://github.com/NixOS/hydra/issues/1181], which might cause fetches of large repositories like [https://github.com/NixOS/nixpkgs nixos/nixpkgs] to fail: <code>error fetching latest change from git repo at `https://github.com/nixos/nixpkgs': timeout</code>. The timeout can be increased with the following configuration.nix snippet: | ||
</div> | </div> | ||
<syntaxHighlight lang=nix> | <syntaxHighlight lang=nix> | ||
{ | { | ||
Line 213: | Line 280: | ||
} | } | ||
</syntaxHighlight> | </syntaxHighlight> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
== Hydra for NixOS releases == | |||
</div> | </div> | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
Hydra is used for managing official Nix project releases. The project Hydra server: https://hydra.nixos.org/ | Hydra is used for managing official Nix project releases. The project Hydra server: https://hydra.nixos.org/ | ||
</div> | </div> | ||
Line 222: | Line 291: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
Some Hydra trackers for Nix projects: | Some Hydra trackers for Nix projects: | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* [https://hydra.nixos.org/project/nixpkgs Nixpkgs] | * [https://hydra.nixos.org/project/nixpkgs Nixpkgs] | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* [https://hydra.nixos.org/project/nixos NixOS] | * [https://hydra.nixos.org/project/nixos NixOS] | ||
</div> | </div> | ||
Line 228: | Line 303: | ||
<div lang="en" dir="ltr" class="mw-content-ltr"> | <div lang="en" dir="ltr" class="mw-content-ltr"> | ||
== Resources == | == Resources == | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* [https://www.youtube.com/watch?v=RXV0Y5Bn-QQ Video: Setting up a Hydra Build Farm by Peter Simons (2016)] | * [https://www.youtube.com/watch?v=RXV0Y5Bn-QQ Video: Setting up a Hydra Build Farm by Peter Simons (2016)] | ||
</div> | |||
<div lang="en" dir="ltr" class="mw-content-ltr"> | |||
* [https://gist.github.com/joepie91/c26f01a787af87a96f967219234a8723 Hydra Caveats by Joepie91] | * [https://gist.github.com/joepie91/c26f01a787af87a96f967219234a8723 Hydra Caveats by Joepie91] | ||
</div> | </div> |