Hydra/en - Revision history

FuzzyBot: Updating to match new version of source page

2025-10-07T21:48:50Z

Updating to match new version of source page

← Older revision		Revision as of 21:48, 7 October 2025
Line 4:		Line 4:
	Hydra is a tool for continuous integration testing and software release that uses a purely functional language to describe build jobs and their dependencies. Continuous integration is a simple technique to improve the quality of the software development process. An automated system continuously or periodically checks out the source code of a project, builds it, runs tests, and produces reports for the developers. Thus, various errors that might accidentally be committed into the code base are automatically caught.		Hydra is a tool for continuous integration testing and software release that uses a purely functional language to describe build jobs and their dependencies. Continuous integration is a simple technique to improve the quality of the software development process. An automated system continuously or periodically checks out the source code of a project, builds it, runs tests, and produces reports for the developers. Thus, various errors that might accidentally be committed into the code base are automatically caught.

	The official Hydra servers provide pre-built binary packages to speed up the update time for ~~Nixpgs~~: Users do not have to compile them on their own computers.		The official Hydra servers provide pre-built binary packages to speed up the update time for Nixpkgs: Users do not have to compile them on their own computers.

	The [https://nixos.org/hydra/manual/ Hydra manual] provides an overview of the functionality and features of Hydra, as well as an up-to-date installation guide.		The [https://nixos.org/hydra/manual/ Hydra manual] provides an overview of the functionality and features of Hydra, as well as an up-to-date installation guide.
Line 12:		Line 12:
	A full deployment can be enabled as easy as:		A full deployment can be enabled as easy as:

	<~~syntaxHighlight~~ lang=nix>		<syntaxhighlight lang="nix">
	services.hydra = {		services.hydra = {
	enable = true;		enable = true;
Line 19:		Line 19:
	# a standalone Hydra will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines		# a standalone Hydra will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines
	buildMachinesFiles = [];		buildMachinesFiles = [];
	# you will probably also want, otherwise everything will be built from scratch		# you will probably also want this, otherwise everything will be built from scratch
	useSubstitutes = true;		useSubstitutes = true;
	};		};
	</~~syntaxHighlight~~>		</syntaxhighlight>

	The module will automatically enable [[PostgreSQL\|postgresql]] if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the Hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.		The module will automatically enable [[PostgreSQL\|postgresql]] if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the Hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.

FuzzyBot: Updating to match new version of source page

2025-08-26T15:55:07Z

Updating to match new version of source page

← Older revision		Revision as of 15:55, 26 August 2025
Line 1:		Line 1:
	<languages />		<languages />
			{{note\| Hydra is intended to be used by Nix/NixOS package developers; it is not needed to simply use NixOS.}}

	Hydra is a tool for continuous integration testing and software release that uses a purely functional language to describe build jobs and their dependencies. Continuous integration is a simple technique to improve the quality of the software development process. An automated system continuously or periodically checks out the source code of a project, builds it, runs tests, and produces reports for the developers. Thus, various errors that might accidentally be committed into the code base are automatically caught.		Hydra is a tool for continuous integration testing and software release that uses a purely functional language to describe build jobs and their dependencies. Continuous integration is a simple technique to improve the quality of the software development process. An automated system continuously or periodically checks out the source code of a project, builds it, runs tests, and produces reports for the developers. Thus, various errors that might accidentally be committed into the code base are automatically caught.

Line 22:		Line 24:
	</syntaxHighlight>		</syntaxHighlight>

	The module will automatically enable postgresql if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the Hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.		The module will automatically enable [[PostgreSQL\|postgresql]] if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the Hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.

	* See nixos-option or the [https://search.nixos.org/options?query=services.hydra Nixos Options page] for all options		* See nixos-option or the [https://search.nixos.org/options?query=services.hydra Nixos Options page] for all options
Line 44:		Line 46:
	nix.buildMachines = [		nix.buildMachines = [
	{ hostName = "localhost";		{ hostName = "localhost";
			protocol = null;
	system = "x86_64-linux";		system = "x86_64-linux";
	supportedFeatures = ["kvm" "nixos-test" "big-parallel" "benchmark"];		supportedFeatures = ["kvm" "nixos-test" "big-parallel" "benchmark"];
Line 58:		Line 61:
	Configure jobset to the following:		Configure jobset to the following:

	* Type: Flake		* Type: [[Flakes\|Flake]]

	* Flake URI: an URI to a repo containing a Flake like git+https://git.myserver.net/user/repo.git		* Flake URI: an URI to a repo containing a Flake like git+https://git.myserver.net/user/repo.git

FuzzyBot: Updating to match new version of source page

2024-07-30T12:51:34Z

Updating to match new version of source page

Show changes

FuzzyBot: Updating to match new version of source page

2024-07-09T07:18:00Z

Updating to match new version of source page

← Older revision		Revision as of 07:18, 9 July 2024
Line 4:		Line 4:
	The official Hydra servers provide pre-built binary packages to speed up the update time for Nixpgs: Users do not have to compile them on their own computers.		The official Hydra servers provide pre-built binary packages to speed up the update time for Nixpgs: Users do not have to compile them on their own computers.

	The [https://nixos.org/hydra/manual/ Hydra manual] provides an overview of the functionality and features of ~~hydra~~, as well as an up-to-date installation guide.		The [https://nixos.org/hydra/manual/ Hydra manual] provides an overview of the functionality and features of Hydra, as well as an up-to-date installation guide.

	== Installation ==		== Installation ==
	~~Since 2017, hydra is available as a NixOS module and therefore a~~ full deployment can be enabled as easy as
			A full deployment can be enabled as easy as:

	<syntaxHighlight lang=nix>		<syntaxHighlight lang=nix>
	services.hydra = {		services.hydra = {
	enable = true;		enable = true;
	hydraURL = "http://localhost:3000"; # externally visible URL		hydraURL = "http://localhost:3000"; # externally visible URL
	notificationSender = "hydra@localhost"; # e-mail of ~~hydra~~ service		notificationSender = "hydra@localhost"; # e-mail of Hydra service
	# a standalone ~~hydra~~ will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines		# a standalone Hydra will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines
	buildMachinesFiles = [];		buildMachinesFiles = [];
	# you will probably also want, otherwise everything will be built from scratch		# you will probably also want, otherwise everything will be built from scratch
Line 19:		Line 21:
	};		};
	</syntaxHighlight>		</syntaxHighlight>
	The module will automatically enable postgresql if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the ~~hydra~~ service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.
			The module will automatically enable postgresql if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the Hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.
	* See nixos-option or the [https://search.nixos.org/options?query=services.hydra Nixos Options page] for all options		* See nixos-option or the [https://search.nixos.org/options?query=services.hydra Nixos Options page] for all options

	=== Web Configuration ===		=== Web Configuration ===
	Hydra will provide the web interface [http://localhost:3000/ at localhost] port 3000. However you need to create a new admin user (as ~~unix~~ user <code>hydra</code>) before being able to perform any changes:		Hydra will provide the web interface [http://localhost:3000/ at localhost] port 3000. However you need to create a new admin user (as UNIX user <code>hydra</code>) before being able to perform any changes:
	<syntaxHighlight lang=bash>		<syntaxHighlight lang=bash>
	# su - hydra		# su - hydra
Line 44:		Line 47:
	</syntaxHighlight>		</syntaxHighlight>

	This option leads to the file /etc/nix/machines being created. If the hydra service config is still set to buildMachinesFiles = [], then it will be ignored, so remove this option again or add /etc/nix/machines to it.		This option leads to the file /etc/nix/machines being created. If the hydra service config is still set to buildMachinesFiles = [], then it will be ignored, so remove this option again or add <code>/etc/nix/machines</code> to it.


	== Flake jobset ==		== Flake jobset ==

	Configure jobset the following:		Configure jobset to the following:

	* Type: Flake		* Type: Flake
Line 93:		Line 95:
	=== Imperative Building ===		=== Imperative Building ===
	These steps are required to build the <code>hello</code> package.		These steps are required to build the <code>hello</code> package.
	# log into ~~hydra~~ after creating a user with <code>hydra-create-user</code>		# log into Hydra after creating a user with <code>hydra-create-user</code>
	# create new project		# create new project
	* identifier: example-hello		* identifier: example-hello
Line 115:		Line 117:

	=== Declarative Building ===		=== Declarative Building ===
	Since 2016, ~~hydra~~ supports declarative creation of jobsets. Check out the [https://github.com/shlevy/declarative-hydra-example example repository and description by Shea Levy].		Since 2016, Hydra supports declarative creation of jobsets. Check out the [https://github.com/shlevy/declarative-hydra-example example repository and description by Shea Levy].

	== Hydra Internals ==		== Hydra Internals ==
Line 122:		Line 124:

	==== Project ====		==== Project ====
	A cluster of Jobs which are all coming from a single input (like a git checkout), the first thing you will need to create. Every Job should be able to be built independently from another. Most of the time the project maps to a single repository like <code>nixpkgs</code>. It is comparable to the project definition in Jenkins		A cluster of Jobs which are all coming from a single input (like a git checkout), the first thing you will need to create. Every Job should be able to be built independently from another. Most of the time the project maps to a single repository like <code>nixpkgs</code>. It is comparable to the project definition in Jenkins.

	==== ~~Job Set~~ ====		==== Jobsets ====
	A list of jobs which will be run. Often a ~~Jobset~~ fits to a certain branch (master, staging, stable). A ~~jobset~~ is defined by its inputs and will trigger if these inputs change~~, e~~.~~g. like~~ a new commit onto a branch is added. ~~Job sets~~ may depend on each other		A Jobset is a list of jobs which will be run. Often a jobset fits to a certain branch (master, staging, stable). A Jobset is defined by its inputs and will trigger if these inputs change. For example when a new commit onto a branch is added. Jobsets may depend on each other.

	==== Job ====		==== Job ====
	A closure which will be built as part of a ~~jobset~~ (like a single package, iso image or tarball)		A closure which will be built as part of a job set (like a single package, iso image or tarball).

	==== Release Set ====		==== Release Set ====
Line 134:		Line 136:

	==== Evaluation ====		==== Evaluation ====
	The process of interpreting nix code into a list of <code>.drv files</code>. These files are the build recipes for all related outputs. You can introspect these files by running <code>nix show-derivation nixpkgs.hello</code>		The process of interpreting nix code into a list of <code>.drv files</code>. These files are the build recipes for all related outputs. You can introspect these files by running <code>nix show-derivation nixpkgs.hello</code>.

	==== Build ====		==== Build ====
	Instantiation of a Job which is being triggered by being part of the release set		Instantiation of a Job which is being triggered by being part of the release set.

	== Known Issues ==		== Known Issues ==

FuzzyBot: Updating to match new version of source page

2024-07-08T08:17:20Z

Updating to match new version of source page

New page

<languages />
Hydra is a tool for continuous integration testing and software release that uses a purely functional language to describe build jobs and their dependencies. Continuous integration is a simple technique to improve the quality of the software development process. An automated system continuously or periodically checks out the source code of a project, builds it, runs tests, and produces reports for the developers. Thus, various errors that might accidentally be committed into the code base are automatically caught.

The official Hydra servers provide pre-built binary packages to speed up the update time for Nixpgs: Users do not have to compile them on their own computers.

The [https://nixos.org/hydra/manual/ Hydra manual] provides an overview of the functionality and features of hydra, as well as an up-to-date installation guide.

== Installation ==
Since 2017, hydra is available as a NixOS module and therefore a full deployment can be enabled as easy as
<syntaxHighlight lang=nix>
services.hydra = {
enable = true;
hydraURL = "http://localhost:3000"; # externally visible URL
notificationSender = "hydra@localhost"; # e-mail of hydra service
# a standalone hydra will require you to unset the buildMachinesFiles list to avoid using a nonexistant /etc/nix/machines
buildMachinesFiles = [];
# you will probably also want, otherwise *everything* will be built from scratch
useSubstitutes = true;
};
</syntaxHighlight>
The module will automatically enable postgresql if you do not change the <code>services.hydra.dbi</code> option. Database layout will be created automatically by the hydra service, however keep in mind that some state will be stored in the database and a complete stateless configuration is currently not possible - do your backups.
* See nixos-option or the [https://search.nixos.org/options?query=services.hydra Nixos Options page] for all options

=== Web Configuration ===
Hydra will provide the web interface [http://localhost:3000/ at localhost] port 3000. However you need to create a new admin user (as unix user <code>hydra</code>) before being able to perform any changes:
<syntaxHighlight lang=bash>
# su - hydra
$ hydra-create-user alice --full-name 'Alice Q. User' \
--email-address 'alice@example.org' --password-prompt --role admin
</syntaxHighlight>

=== Virtual machine ===
If not configured explicitly to do otherwise, Hydra will specify localhost as the default build machine. By default, system features enabling builds to be performed in virtual machines like "kvm" or "nixos-test" are not enabled. Such jobs will be queued indefinitely. Those options can be activated as follows:
<syntaxHighlight lang=nix>
{
nix.buildMachines = [
{ hostName = "localhost";
system = "x86_64-linux";
supportedFeatures = ["kvm" "nixos-test" "big-parallel" "benchmark"];
maxJobs = 8;
}
];
}
</syntaxHighlight>

This option leads to the file /etc/nix/machines being created. If the hydra service config is still set to buildMachinesFiles = [], then it will be ignored, so remove this option again or add /etc/nix/machines to it.

== Flake jobset ==

Configure jobset the following:

* Type: Flake
* Flake URI: an URI to a repo containing a Flake like git+https://git.myserver.net/user/repo.git

The Flake output should have the attribute <code>hydraJobs</code> containing an attribute set that may be nested and reference derivations.

A sample Flake output that makes Hydra build all packages could look like this:

<syntaxHighlight lang=nix>
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs, ... }: {
packages.x86_64-linux = {
...
};

hydraJobs = {
inherit (self)
packages;
};
};
}
</syntaxHighlight>

=== Restricted Mode ===

Hydra evaluates flakes in [https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval restricted mode]. This prevents access to files outside of the nix store, including those fetched as flake inputs. Update your <code>nix.settings.allowed-uris</code> to include URI prefixes from which you expect flake inputs to be fetched:

<syntaxHighlight lang=nix>
nix.settings.allowed-uris = [
"github:"
"git+https://github.com/"
"git+ssh://github.com/"
];
</syntaxHighlight>

== Build a single Package from nixpkgs ==
Right now it is not possible to build a single package from nixpkgs with just that input. You will need to provide a supplementary repository which defines what to build. For examples you can check the [https://github.com/makefu/hydra-example hydra-example by makefu] and in the [https://nixos.org/hydra/manual/#idm140737315920320 Hydra Manual].

=== Imperative Building ===
These steps are required to build the <code>hello</code> package.
# log into hydra after creating a user with <code>hydra-create-user</code>
# create new project
* identifier: example-hello
* display name: example-hello
# Actions -> Create jobset
* identifier: hello
* Nix expression: <code>release.nix</code> in <code>hydra-example</code> -> will evaluate the file release.nix in the given input
* check interval: 60
* scheduling shares: 1
* Inputs:
{| class="wikitable"
|-
! Input Name !! Type !! Value !! Note
|-
| nixpkgs || git checkout || https://github.com/nixos/nixpkgs nixos-21.11 || will check out branch nixos-21.11, will be made available to the nix expression via <code><nixpkgs></code>.
|-
| hydra-example || git checkout || https://github.com/makefu/hydra-example || hydra-example is used by the jobset as input, <code>release.nix</code> is in the root directory
|}

After creation, the jobset should be in the '''evaluation phase''' where inputs will be fetched. This phase may take some time as the complete <code>nixpkgs</code> repository needs to be downloaded before continuing. The result of the evaluation should be a single job which will get built.

=== Declarative Building ===
Since 2016, hydra supports declarative creation of jobsets. Check out the [https://github.com/shlevy/declarative-hydra-example example repository and description by Shea Levy].

== Hydra Internals ==
=== Definitions ===
This subsection provides an overview of the Hydra-specific definitions and how to configure them.

==== Project ====
A cluster of Jobs which are all coming from a single input (like a git checkout), the first thing you will need to create. Every Job should be able to be built independently from another. Most of the time the project maps to a single repository like <code>nixpkgs</code>. It is comparable to the project definition in Jenkins

==== Job Set ====
A list of jobs which will be run. Often a Jobset fits to a certain branch (master, staging, stable). A jobset is defined by its inputs and will trigger if these inputs change, e.g. like a new commit onto a branch is added. Job sets may depend on each other

==== Job ====
A closure which will be built as part of a jobset (like a single package, iso image or tarball)

==== Release Set ====
Defines all the jobs which are described in your release. By convention a file called<code>release.nix</code> is being used. See the [https://nixos.org/hydra/manual/#idm140737315920320 Hydra manual for Build Recipes] for a thorough description of the structure.

==== Evaluation ====
The process of interpreting nix code into a list of <code>.drv files</code>. These files are the build recipes for all related outputs. You can introspect these files by running <code>nix show-derivation nixpkgs.hello</code>

==== Build ====
Instantiation of a Job which is being triggered by being part of the release set

== Known Issues ==
* hydra-queue-runner sometimes gets stuck even with builds are in the queue, and the builds are not scheduled. The issue is being tracked [https://github.com/NixOS/hydra/issues/366 here]. In the meantime, a workaround is to add a cron job that regularly restarts the hydra-queue-runner systemd service. Possible fix: [https://github.com/NixOS/hydra/commit/73ca325d1c0f7914640a63764c9a6d448fde5bd0]
* If you see <code>error: unexpected end-of-file</code> it can mean multiple things, some of them are:
# You have a miss-match between nix versions on the Hydra server and the builder
# It can also mean that <code>hydra-queue-runner</code> needs privileges on the build server. Reference: [https://github.com/NixOS/nix/issues/2789]
* The default timeout for git operations is 600 seconds [https://github.com/NixOS/hydra/issues/1181], which might cause fetches of large repositories like [https://github.com/NixOS/nixpkgs nixos/nixpkgs] to fail: <code>error fetching latest change from git repo at `https://github.com/nixos/nixpkgs': timeout</code>. The timeout can be increased with the following configuration.nix snippet:

<syntaxHighlight lang=nix>
{
services.hydra.extraConfig = ''
<git-input>
timeout = 3600
</git-input>
'';
}
</syntaxHighlight>

== Hydra for NixOS releases ==
Hydra is used for managing official Nix project releases. The project Hydra server: https://hydra.nixos.org/

Some Hydra trackers for Nix projects:
* [https://hydra.nixos.org/project/nixpkgs Nixpkgs]
* [https://hydra.nixos.org/project/nixos NixOS]

== Resources ==
* [https://www.youtube.com/watch?v=RXV0Y5Bn-QQ Video: Setting up a Hydra Build Farm by Peter Simons (2016)]
* [https://gist.github.com/joepie91/c26f01a787af87a96f967219234a8723 Hydra Caveats by Joepie91]

[[Category:Pedias]]
[[Category:Hydra]]
[[Category:Nix]]
[[Category:Incomplete]]
[[Category:Software]]
[[Category:Server]]