NixOS Containers: Difference between revisions
imported>Mic92 No edit summary |
mention microVMs as an alternative |
||
(29 intermediate revisions by 16 users not shown) | |||
Line 1: | Line 1: | ||
== Native NixOS containers == | |||
It is possible to configure native [https://wiki.archlinux.org/title/systemd-nspawn systemd-nspawn] containers, which are running NixOS and are configured and managed by NixOS using the <code>containers</code> directive. | |||
=== Configuration === | |||
The following example creates a container called <code>nextcloud</code> running the web application [[Nextcloud]]. It will start automatically at boot and has its private network subnet. | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
networking.nat = { | |||
enable = true; | |||
internalInterfaces = ["ve-+"]; | |||
externalInterface = "ens3"; | |||
# Lazy IPv6 connectivity for the container | |||
enableIPv6 = true; | |||
}; | |||
containers.nextcloud = { | |||
autoStart = true; | |||
privateNetwork = true; | |||
hostAddress = "192.168.100.10"; | |||
localAddress = "192.168.100.11"; | |||
hostAddress6 = "fc00::1"; | |||
localAddress6 = "fc00::2"; | |||
config = { config, pkgs, lib, ... }: { | |||
services.nextcloud = { | |||
enable = true; | |||
package = pkgs.nextcloud28; | |||
hostName = "localhost"; | |||
config.adminpassFile = "${pkgs.writeText "adminpass" "test123"}"; # DON'T DO THIS IN PRODUCTION - the password file will be world-readable in the Nix Store! | |||
}; | |||
system.stateVersion = "23.11"; | |||
networking = { | |||
firewall = { | |||
enable = true; | |||
allowedTCPPorts = [ 80 ]; | |||
}; | |||
# Use systemd-resolved inside the container | |||
# Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686 | |||
useHostResolvConf = lib.mkForce false; | |||
}; | |||
services.resolved.enable = true; | |||
}; | |||
}; | |||
</nowiki>}} | |||
In order to reach the web application on the host system, we have to open [[Firewall]] port 80 and also configure NAT through <code>networking.nat</code>. The web service of the container will be available at http://192.168.100.11 | |||
==== Networking ==== | |||
{{expansion}} | |||
By default, if <code>privateNetwork</code> is not set, the container shares the network with the host, enabling it to bind any port on any interface. However, when <code>privateNetwork</code> is set to <code>true</code>, the container gains its private virtual <code>eth0</code> and <code>ve-<container_name></code> on the host. This isolation is beneficial when you want the container to have its dedicated networking stack. | |||
'''NAT (Network Address Translation)''' | |||
<syntaxhighlight lang="nix"> | |||
</syntaxhighlight> | |||
'''Bridge''' | |||
<syntaxhighlight lang="nix"> | |||
networking = { | |||
bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly | |||
# Get bridge-ip with DHCP | |||
useDHCP = false; | |||
interfaces."br0".useDHCP = true; | |||
# Set bridge-ip static | |||
interfaces."br0".ipv4.addresses = [{ | |||
address = "192.168.100.3"; | |||
prefixLength = 24; | |||
}]; | |||
defaultGateway = "192.168.100.1"; | |||
nameservers = [ "192.168.100.1" ]; | |||
}; | |||
containers.<name> = { | |||
privateNetwork = true; | |||
hostBridge = "br0"; # Specify the bridge name | |||
localAddress = "192.168.100.5/24"; | |||
config = { }; | |||
}; | |||
</syntaxhighlight> | |||
=== Usage === | |||
List containers | |||
<syntaxhighlight lang="console"> | |||
# machinectl list | |||
</syntaxhighlight> | |||
Checking the status of the container | |||
<syntaxhighlight lang="console"> | |||
# systemctl status container@nextcloud | |||
</syntaxhighlight> | |||
Login into the container | |||
<syntaxhighlight lang="console"> | |||
# nixos-container root-login nextcloud | |||
</syntaxhighlight> | |||
Start or stop a container | |||
<syntaxhighlight lang="console"> | |||
# nixos-container start nextcloud | |||
# nixos-container stop nextcloud | |||
</syntaxhighlight> | |||
Destroy a container including its file system | |||
<syntaxhighlight lang="console"> | |||
# nixos-container destroy nextcloud | |||
</syntaxhighlight> | |||
Further informations are available in the {{manual:nixos|sec=#ch-containers|chapter=NixOS manual}}. | |||
== Declarative OCI containers (Docker/Podman) == | |||
=== Example config === | |||
<syntaxhighlight lang="nixos"> | |||
{ config, pkgs, ... }: | |||
{ | |||
config.virtualisation.oci-containers.containers = { | |||
hackagecompare = { | |||
image = "chrissound/hackagecomparestats-webserver:latest"; | |||
ports = ["127.0.0.1:3010:3010"]; | |||
volumes = [ | |||
"/root/hackagecompare/packageStatistics.json:/root/hackagecompare/packageStatistics.json" | |||
]; | |||
cmd = [ | |||
"--base-url" | |||
"\"/hackagecompare\"" | |||
]; | |||
}; | |||
}; | |||
} | |||
</syntaxhighlight> | |||
=== Usage === | |||
NixOS uses Podman to run OCI containers. Note that these are '''user-specific''', so running commands with or without sudo can change your output. | |||
List containers | |||
<syntaxhighlight lang="console"> | |||
# podman ps | |||
</syntaxhighlight> | |||
Update image | |||
<syntaxhighlight lang="console"> | |||
# podman restart hackagecompare | |||
</syntaxhighlight> | |||
List images | |||
<syntaxhighlight lang="console"> | |||
# podman ls | |||
</syntaxhighlight>Remove container<syntaxhighlight lang="console"> | |||
# podman rm hackagecompare | |||
</syntaxhighlight> | |||
Remove image | |||
<syntaxhighlight lang="console"> | |||
# podman rmi c0d9a5f58afe | |||
</syntaxhighlight>Update image<syntaxhighlight lang="console"> | |||
# podman pull chrissound/hackagecomparestats-webserver:latest | |||
</syntaxhighlight>Run interactive shell in running container<syntaxhighlight lang="console"> | |||
# podman exec -ti $ContainerId /bin/sh | |||
</syntaxhighlight> | |||
== Troubleshooting == | == Troubleshooting == | ||
Line 15: | Line 187: | ||
== See also == | == See also == | ||
{{manual:nixos|sec=#ch-containers|chapter=Chapter | * {{manual:nixos|sec=#ch-containers|chapter=Chapter on Container Management}} | ||
* [https://blog.beardhatcode.be/2020/12/Declarative-Nixos-Containers.html Blog Article - Declarative NixOS Containers] | |||
* [https://discourse.nixos.org/t/extra-container-run-declarative-containers-without-full-system-rebuilds/511 NixOS Discourse - Extra-container: Run declarative containers without full system rebuilds] | |||
* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/virtualization/nixos-container/nixos-container.pl Nixpkgs - nixos-container.pl] | |||
* [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/nixos-containers.nix Nixpkgs - nixos-containers.nix] | |||
* [https://nixcademy.com/2023/08/29/nixos-nspawn/ nixos-nspawn] | |||
* [https://github.com/tfc/nspawn-nixos tfc/nspawn-nixos] | |||
* MicroVMs as a more isolated alternative, e.g. with https://github.com/astro/microvm.nix | |||
[[Category:Server]] | |||
[[Category:NixOS]] | |||
[[Category:Container]] |
Latest revision as of 08:15, 10 July 2024
Native NixOS containers
It is possible to configure native systemd-nspawn containers, which are running NixOS and are configured and managed by NixOS using the containers
directive.
Configuration
The following example creates a container called nextcloud
running the web application Nextcloud. It will start automatically at boot and has its private network subnet.
/etc/nixos/configuration.nix
networking.nat = {
enable = true;
internalInterfaces = ["ve-+"];
externalInterface = "ens3";
# Lazy IPv6 connectivity for the container
enableIPv6 = true;
};
containers.nextcloud = {
autoStart = true;
privateNetwork = true;
hostAddress = "192.168.100.10";
localAddress = "192.168.100.11";
hostAddress6 = "fc00::1";
localAddress6 = "fc00::2";
config = { config, pkgs, lib, ... }: {
services.nextcloud = {
enable = true;
package = pkgs.nextcloud28;
hostName = "localhost";
config.adminpassFile = "${pkgs.writeText "adminpass" "test123"}"; # DON'T DO THIS IN PRODUCTION - the password file will be world-readable in the Nix Store!
};
system.stateVersion = "23.11";
networking = {
firewall = {
enable = true;
allowedTCPPorts = [ 80 ];
};
# Use systemd-resolved inside the container
# Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
useHostResolvConf = lib.mkForce false;
};
services.resolved.enable = true;
};
};
In order to reach the web application on the host system, we have to open Firewall port 80 and also configure NAT through networking.nat
. The web service of the container will be available at http://192.168.100.11
Networking
By default, if privateNetwork
is not set, the container shares the network with the host, enabling it to bind any port on any interface. However, when privateNetwork
is set to true
, the container gains its private virtual eth0
and ve-<container_name>
on the host. This isolation is beneficial when you want the container to have its dedicated networking stack.
NAT (Network Address Translation)
Bridge
networking = {
bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly
# Get bridge-ip with DHCP
useDHCP = false;
interfaces."br0".useDHCP = true;
# Set bridge-ip static
interfaces."br0".ipv4.addresses = [{
address = "192.168.100.3";
prefixLength = 24;
}];
defaultGateway = "192.168.100.1";
nameservers = [ "192.168.100.1" ];
};
containers.<name> = {
privateNetwork = true;
hostBridge = "br0"; # Specify the bridge name
localAddress = "192.168.100.5/24";
config = { };
};
Usage
List containers
# machinectl list
Checking the status of the container
# systemctl status container@nextcloud
Login into the container
# nixos-container root-login nextcloud
Start or stop a container
# nixos-container start nextcloud
# nixos-container stop nextcloud
Destroy a container including its file system
# nixos-container destroy nextcloud
Further informations are available in the NixOS Manual, NixOS manual.
Declarative OCI containers (Docker/Podman)
Example config
{ config, pkgs, ... }:
{
config.virtualisation.oci-containers.containers = {
hackagecompare = {
image = "chrissound/hackagecomparestats-webserver:latest";
ports = ["127.0.0.1:3010:3010"];
volumes = [
"/root/hackagecompare/packageStatistics.json:/root/hackagecompare/packageStatistics.json"
];
cmd = [
"--base-url"
"\"/hackagecompare\""
];
};
};
}
Usage
NixOS uses Podman to run OCI containers. Note that these are user-specific, so running commands with or without sudo can change your output.
List containers
# podman ps
Update image
# podman restart hackagecompare
List images
# podman ls
Remove container
# podman rm hackagecompare
Remove image
# podman rmi c0d9a5f58afe
Update image
# podman pull chrissound/hackagecomparestats-webserver:latest
Run interactive shell in running container
# podman exec -ti $ContainerId /bin/sh
Troubleshooting
I have changed the host's channel and some services are no longer functional
Symptoms:
- Lost data in PostgreSQL database
- MySQL has changed its path, where it creates the database
Solution
If you did not have a system.stateVersion
option set inside your declarative container configuration, it will use the default one for the channel. Your data might be safe, if you did nothing meanwhile. Add the missing system.stateVersion
to your container, rebuild, and possibly stop/start the container.
See also
- NixOS Manual, Chapter on Container Management
- Blog Article - Declarative NixOS Containers
- NixOS Discourse - Extra-container: Run declarative containers without full system rebuilds
- Nixpkgs - nixos-container.pl
- Nixpkgs - nixos-containers.nix
- nixos-nspawn
- tfc/nspawn-nixos
- MicroVMs as a more isolated alternative, e.g. with https://github.com/astro/microvm.nix