Official NixOS Wiki

NixOS Containers: Difference between revisions

← Older edit
DHCP (talk | contribs)
104 edits
VisualWikitext
imported>Nix
add more see also references
DHCP (talk | contribs)
104 edits
m update system.stateVersion to 26.05
 
(41 intermediate revisions by 20 users not shown)
Line 1: Line 1:
There's not much to read here. Look at {{manual:nixos|sec=#ch-containers|chapter=the upstream documentation on containers}} meanwhile.
Setup native [https://wiki.archlinux.org/title/systemd-nspawn systemd-nspawn] containers, which are running NixOS and are configured and managed by NixOS using the <code>containers</code> directive.


== Declarative docker containers ==
See [[Docker]] page for OCI container (Docker, Podman) configuration.


Example config:
=== Configuration ===
   { config, pkgs, ... }:
 
  {
The following example creates a container called webserver running a httpd web server. It will start automatically at boot and has its private network subnet.
     config.virtualisation.oci-containers.containers = {
{{file|3=networking.nat = {
      hackagecompare = {
  enable = true;
        image = "chrissound/hackagecomparestats-webserver:latest";
  # Use "ve-*" when using nftables instead of iptables
        ports = ["127.0.0.1:3010:3010"];
  internalInterfaces = ["ve-+"];
        volumes = [
  externalInterface = "ens3";
          "/root/hackagecompare/packageStatistics.json:/root/hackagecompare/packageStatistics.json"
  # Lazy IPv6 connectivity for the container
        ];
  enableIPv6 = true;
        cmd = [
};
          "--base-url"
 
          "\"/hackagecompare\""
containers.webserver = {
  autoStart = true;
  privateNetwork = true;
  hostAddress = "192.168.100.10";
  localAddress = "192.168.100.11";
  hostAddress6 = "fc00::1";
  localAddress6 = "fc00::2";
   config = { config, pkgs, lib, ... }: {
 
    services.httpd = {
      enable = true;
      adminAddr = "admin@example.org";
    };
 
    networking = {
      firewall.allowedTCPPorts = [ 80 ];
 
      # Use systemd-resolved inside the container
      # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
      useHostResolvConf = lib.mkForce false;
    };
   
    services.resolved.enable = true;
 
     system.stateVersion = "26.05";
  };
};|name=/etc/nixos/configuration.nix|lang=nix}}
 
In order to reach the web application on the host system, we have to open [[Firewall]] port 80 and also configure NAT through <code>networking.nat</code>. The web service of the container will be available at http://192.168.100.11
 
==== Networking ====
 
{{expansion}}
 
By default, if <code>privateNetwork</code> is not set, the container shares the network with the host, enabling it to bind any port on any interface. However, when <code>privateNetwork</code> is set to <code>true</code>, the container gains its private virtual <code>eth0</code> and <code>ve-<container_name></code> on the host. This isolation is beneficial when you want the container to have its dedicated networking stack.
 
'''NAT (Network Address Translation)'''
 
In order to allow the container to connect to the internet, you have to configure NAT through <code>networking.nat</code>.
{{File|3=networking.nat = {
  enable = true;
  # Use "ve-*" when using nftables instead of iptables
  internalInterfaces = ["ve-+"];
  externalInterface = "ens3";
  # Lazy IPv6 connectivity for the container
  enableIPv6 = true;
};|name=/etc/nixos/configuration.nix|lang=nix}}'''Bridge'''
 
Connect a container to a bridge using Network Manager interfaces:
{{File|3=networking = {
  bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly
 
  # Get bridge-ip with DHCP
  useDHCP = false;
  interfaces."br0".useDHCP = true;
 
  # Set bridge-ip static
  interfaces."br0".ipv4.addresses = [{
    address = "192.168.100.3";
    prefixLength = 24;
  }];
  defaultGateway = "192.168.100.1";
  nameservers = [ "192.168.100.1" ];
};
 
containers.<name> = {
  privateNetwork = true;
  hostBridge = "br0"; # Specify the bridge name
  localAddress = "192.168.100.5/24";
  config = { };
};|name=/etc/nixos/configuration.nix|lang=nix}}'''Without privateNetwork (simpler)'''
 
If the service can be accessed by changing its port, the private network is not needed necessarily. Be careful to not use occupied ports. This example runs an [[Actual]] server on port 3003. It can be accessed through the host at <code>http://localhost:3003</code>. Since <code>privateNetwork</code> is not defined, it defaults to <code>false</code>.
 
{{File|3=containers.actualContainer = {
  autoStart = true;
  config = {...}: {
    services.actual = {
      enable = true;
      settings.port = 3003;
    };
  };
};|name=/etc/nixos/configuration.nix|lang=nix}}
 
=== Usage ===
 
List containers
<syntaxhighlight lang="console">
# machinectl list
</syntaxhighlight>
 
Checking the status of the container
<syntaxhighlight lang="console">
# systemctl status container@webserver
</syntaxhighlight>
 
Login into the container
<syntaxhighlight lang="console">
# nixos-container root-login webserver
</syntaxhighlight>
 
Start or stop a container
<syntaxhighlight lang="console">
# nixos-container start webserver
# nixos-container stop webserver
</syntaxhighlight>
 
Destroy a container including its file system
<syntaxhighlight lang="console">
# nixos-container destroy webserver
</syntaxhighlight>
 
View log for container
<syntaxhighlight lang="console">
# journalctl -M webserver
</syntaxhighlight>
Further informations are available in the {{manual:nixos|sec=#ch-containers|chapter=NixOS manual}}.
 
== Tips and tricks ==
 
=== Define and create nixos-container from a Flake file ===
 
We can define and create a custom container called <code>container</code> from a file stored as <code>flake.nix</code>. In this case we use the unstable branch of the nixpkgs repository as a source.
{{File|3={
  inputs.nixpkgs.url = "nixpkgs/nixos-unstable";
 
  outputs = { self, nixpkgs }: {
 
    nixosConfigurations.container = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ ({ pkgs, ... }: {
            boot.isContainer = true;
 
            networking.firewall.allowedTCPPorts = [ 80 ];
 
            services.httpd = {
              enable = true;
              adminAddr = "morty@example.org";
            };
          })
         ];
         ];
    };
  };
}|name=/etc/nixos/configuration.nix|lang=nix}}
To create and run that container, enter following commands. In this example the <code>flake.nix</code> file is in the same directory.
<syntaxhighlight lang="console">
# nixos-container create flake-test --flake .
host IP is 10.233.4.1, container IP is 10.233.4.2
# nixos-container start flake-test
</syntaxhighlight>
=== Use agenix secrets in container ===
To add <code>agenix</code> secrets to a container bind mount the <code>ssh-host.key</code> and import the <code>agenix.nixosModule</code> and set <code>age.identityPaths</code> [https://discourse.nixos.org/t/secrets-inside-nixos-containers/34403/6 Source]
{{File|3={ agenix, ... }:
{
  containers."withSecret" = {
    # pass the private key to the container for agenix to decrypt the secret
    bindMounts."/etc/ssh/ssh_host_ed25519_key".isReadOnly = true;
    config =
      {
        config,
        lib,
        pkgs,
        ...
      }:
      {
        imports = [ agenix.nixosModules.default ]; # import agenix-module into the nixos-container
        age.identityPaths = [ "/etc/ssh/ssh_host_ed25519_key" ]; # isn't set automatically when openssh is not setup
        # import the secret
        age.secrets."secret-name" = {
          file = ../secrets/secret.age;
        };
      };
  };
}|name=/etc/nixos/configuration.nix|lang=nix}}
=== Bridge together two nixos-containers ===
'''Target:'''
Create two containers, both with <code>privateNetwork = true;</code>:
* <code>containerA</code> at 192.168.100.2
** which will access <code>containerB</code>
* <code>containerB</code> at 192.168.100.3
** which runs an httpd server at http://localhost:80
They should be connected with a bridge <code>br0</code> and both should have internet address.
Assuming Network Manager is used, so the introduction of <code>systemd.network</code> should not interfere with the rest of the setup.
'''Configuration:'''
Create and configure the internet connection and the bridge:
{{File|3=# Give containers access to the internet
networking.nat = {
  enable = true;
  internalInterfaces = [ "br0" ]; # Connect the bridge to the internet
  externalInterface = "wlp5s0";  # Adjust according to your internet interface
  # Lazy IPv6 connectivity for the container
  enableIPv6 = true;
};
# Both systemd-networkd and NetworkManager can exist in parallel on the same machine,
# when they manage a distinct set of interfaces.
# If upstream connectivity is managed by NetworkManager (for example, NM handles wifi and networkd does VM networking),
# set systemd.network.wait-online.enable to false so that boot isn't blocked on connectivity that networkd will never provide.
# https://wiki.nixos.org/wiki/Systemd/networkd#When_to_use
systemd.network = {
  enable = true;
  wait-online.enable = false;
  netdevs = {
      # Create the bridge interface
      # Each interface is stored as a seperate file under /etc/systemd/network by default
      # <number>-name is required so that it is not overwritten by other configurations of the same name
      # It is recommended that each filename is prefixed with a number smaller than "70" (e.g. 10-eth0.network).
      # https://www.freedesktop.org/software/systemd/man/latest/systemd.netdev.html
      "20-br0" = {
        netdevConfig = {
          Kind = "bridge";
          Name = "br0";
          # Sets a pre-determined mac address
          # Leave empty if you want the system to auto-assign a mac address to the bridge
          # MACAddress = "10:00:00:00:00:01";
        };
      };
  };
  networks = {
    # Configure the bridge for its desired function
    "40-br0" = {
      matchConfig.Name = "br0";
      # The address of the bridge
      # /29 is the netmask, it creates 2^(32-29) = 8 subnets
      # 2 are reserved (first and last), as Network Address and Broadcast Address
      # The bridge already takes up one subnet, so 3 addresses are already reserved
      # To bridge 2 networks, you need a netmask of <=29 for IPv4
      # https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=29&cip=192.168.100.1&ctype=ipv4&x=Calculate
      # 192.168.100.0 - 192.168.100.7
      # Network Address  Usable Host Range              Broadcast Address
      # 192.168.100.0    192.168.100.1 - 192.168.100.6  192.168.100.7
      address = [
        "192.168.100.1/29"
      ];
      # bridgeConfig = {};
      # Disable address autoconfig when no IP configuration is required
      # networkConfig.LinkLocalAddressing = "no";
      # linkConfig = {
        # or "routable" with IP addresses configured
        # RequiredForOnline = "carrier";
      # };
    };
  };
};|name=/etc/nixos/configuration.nix|lang=nix}}
Create and configure <code>containerA</code>:
{{File|3=containers.containerA = {
  autoStart = true;
  privateNetwork = true;
  hostBridge = "br0";
  # hostAddress = "192.168.100.1";  # Not used when using hostBridge
  localAddress = "192.168.100.2/29"; # Should have the netmask if hostBridge is used
  config =
    { config, pkgs, lib, ... }:
    {
      system.stateVersion = "26.05";
     
      networking = {
        # Changes the gateway to the Network Address of the bridge, so that it has access to the internet
        # The bridge has access to the internet
        defaultGateway = {
          address = "192.168.100.1";
        };
      };
     
      networking = {
        # Use systemd-resolved inside the container
        # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
        useHostResolvConf = lib.mkForce false;
      };
     
      services.resolved.enable = true;
    };
};|name=/etc/nixos/configuration.nix|lang=nix}}
Create and configure <code>containerB</code>:
{{File|3=containers.containerB = {
  autoStart = true;
  privateNetwork = true;
  hostBridge = "br0";
  # hostAddress = "192.168.100.1";  # Not used when using hostBridge
  localAddress = "192.168.100.3/29"; # Should have the netmask if hostBridge is used
  config =
    { config, pkgs, lib, ... }:
    {
      # test http server that uses port :80
      services.httpd = {
        enable = true;
      };
     
      system.stateVersion = "26.05";
     
      networking = {
        # Changes the gateway to the Network Address of the bridge, so that it has access to the internet
        # The bridge has access to the internet
        defaultGateway = {
          address = "192.168.100.1";
        };
      };
     
      networking = {
        firewall.allowedTCPPorts = [ 80 ];
        # Use systemd-resolved inside the container
        # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
        useHostResolvConf = lib.mkForce false;
       };
       };
      services.resolved.enable = true;
     };
     };
  }
};|name=/etc/nixos/configuration.nix|lang=nix}}
 
You can test the connection between <code>containerA</code> and <code>containerB</code> by loggining into <code>containerA</code> and pinging <code>containerB</code>, curling to <code>containerB</code>'s httpd server or pinging an internet website:
<syntaxhighlight lang="console">
# nixos-container root-login containerA
[root@containerA:~]# ping 192.168.100.3 -c3      # Ping containerB
[root@containerA:~]# curl http://192.168.100.3:80 # Curl to containerB's httpd server
[root@containerA:~]# ping nixos.org -c3          # Ping an internet website
</syntaxhighlight>
 
You can test the connection between the host machine and <code>containerA</code> or <code>containerB</code> by pinging <code>containerA</code>,  pinging <code>containerB</code> and curling to <code>containerB</code>'s httpd server:
<syntaxhighlight lang="console">
$ ping 192.168.100.2 -c3      # Ping containerA
$ ping 192.168.100.3 -c3      # Ping containerB
$ curl http://192.168.100.3:80 # Curl to containerB's httpd server
</syntaxhighlight>
 
Note that with the command <code>ip address</code>, even if the interfaces of the containers are displayed (<code>vb-containerA</code> and <code>vb-containerB</code>), they only have a MAC address assigned, they do not have a separate ip address displayed. For extra configuring, maybe use the option <code>containers.<name>.extraVeths</code>.
 
Made with help of the <code>systemd.network</code> wiki page<ref>[[Systemd/networkd]]</ref> and this discourse post<ref>https://discourse.nixos.org/t/how-to-connect-two-or-more-nixos-containers-together-their-internet-ports/77674/9?u=blastboomstrice</ref>.


== Troubleshooting ==
== Troubleshooting ==


=== I have changed the host's channel and some services are no longer functional ===
=== I have changed the host's channel and some services are no longer functional ===
'''Symptoms:'''
'''Symptoms:'''
* Lost data in PostgreSQL database
* Lost data in PostgreSQL database
Line 38: Line 378:
* [https://blog.beardhatcode.be/2020/12/Declarative-Nixos-Containers.html Blog Article - Declarative NixOS Containers]
* [https://blog.beardhatcode.be/2020/12/Declarative-Nixos-Containers.html Blog Article - Declarative NixOS Containers]
* [https://discourse.nixos.org/t/extra-container-run-declarative-containers-without-full-system-rebuilds/511 NixOS Discourse - Extra-container: Run declarative containers without full system rebuilds]
* [https://discourse.nixos.org/t/extra-container-run-declarative-containers-without-full-system-rebuilds/511 NixOS Discourse - Extra-container: Run declarative containers without full system rebuilds]
* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/virtualization/nixos-container/nixos-container.pl Nixpkgs - nixos-container.pl]
* [https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/ni/nixos-container/nixos-container.pl Nixpkgs - nixos-container.pl]
* [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/nixos-containers.nix Nixpkgs - nixos-containers.nix]
* [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/nixos-containers.nix Nixpkgs - nixos-containers.nix]
 
* [https://nixcademy.com/2023/08/29/nixos-nspawn/ nixos-nspawn]
* [https://github.com/tfc/nspawn-nixos tfc/nspawn-nixos]
* MicroVMs as a more isolated alternative, e.g. with https://github.com/astro/microvm.nix
[[Category:Server]]
[[Category:Server]]
[[Category:NixOS]]
[[Category:NixOS]]
[[Category:Container]]
Retrieved from "https://wiki.nixos.org/wiki/NixOS_Containers"