NixOS Containers: Difference between revisions

Revision as of 19:39, 18 May 2026

Setup native systemd-nspawn containers, which are running NixOS and are configured and managed by NixOS using the containers directive.

See Docker page for OCI container (Docker, Podman) configuration.

Host Configuration

For all of the examples below to work, you'll have to enable virtualization and the use of containers in your host systems nix configuration.

❄︎ /etc/nixos/configuration.nix

boot.enableContainers = true;
virtualisation.containers.enable = true;

Configuration

The following example creates a container called webserver running a httpd web server. It will start automatically at boot and has its private network subnet.

❄︎ /etc/nixos/configuration.nix

networking.nat = {
  enable = true;
  internalInterfaces = ["ve-+"];
  externalInterface = "ens3";
  # Lazy IPv6 connectivity for the container
  enableIPv6 = true;
};

containers.webserver = {
  autoStart = true;
  privateNetwork = true;
  hostAddress = "192.168.100.10";
  localAddress = "192.168.100.11";
  hostAddress6 = "fc00::1";
  localAddress6 = "fc00::2";
  config = { config, pkgs, lib, ... }: {

    services.httpd = {
      enable = true;
      adminAddr = "admin@example.org";
    };

    networking = {
      firewall.allowedTCPPorts = [ 80 ];

      # Use systemd-resolved inside the container
      # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
      useHostResolvConf = lib.mkForce false;
    };
    
    services.resolved.enable = true;

    system.stateVersion = "24.11";
  };
};

In order to reach the web application on the host system, we have to open Firewall port 80 and also configure NAT through networking.nat. The web service of the container will be available at http://192.168.100.11

Networking

☶︎

This article or section needs to be expanded. Further information may be found in the related discussion page. Please consult the pedia article metapage for guidelines on contributing.

By default, if privateNetwork is not set, the container shares the network with the host, enabling it to bind any port on any interface. However, when privateNetwork is set to true, the container gains its private virtual eth0 and ve-<container_name> on the host. This isolation is beneficial when you want the container to have its dedicated networking stack.

NAT (Network Address Translation)

Bridge

networking = {
  bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly
  
  # Get bridge-ip with DHCP
  useDHCP = false;
  interfaces."br0".useDHCP = true;

  # Set bridge-ip static
  interfaces."br0".ipv4.addresses = [{
    address = "192.168.100.3";
    prefixLength = 24;
  }];
  defaultGateway = "192.168.100.1";
  nameservers = [ "192.168.100.1" ];
};

containers.<name> = {
  privateNetwork = true;
  hostBridge = "br0"; # Specify the bridge name
  localAddress = "192.168.100.5/24";
  config = { };
};

Without privateNetwork (simpler)

If the service can be accessed by changing its port, the private network is not needed necessarily. Be careful to not use occupied ports. This example runs an Actual server on port 3003. It can be accessed through the host at http://localhost:3003. Since privateNetwork is not defined, it defaults to false.

containers.actualContainer = {
  autoStart = true;
  config = {...}: {
    services.actual = {
      enable = true;
      settings.port = 3003;
    };
  };
};

Usage

List containers

# machinectl list

Checking the status of the container

# systemctl status container@webserver

Login into the container

# nixos-container root-login webserver

Start or stop a container

# nixos-container start webserver
# nixos-container stop webserver

Destroy a container including its file system

# nixos-container destroy webserver

View log for container

# journalctl -M webserver

Further informations are available in the NixOS Manual, NixOS manual.

Tips and tricks

Define and create nixos-container from a Flake file

We can define and create a custom container called container from a file stored as flake.nix. In this case we use the unstable branch of the nixpkgs repository as a source.

{
  inputs.nixpkgs.url = "nixpkgs/nixos-unstable";

  outputs = { self, nixpkgs }: {

    nixosConfigurations.container = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ ({ pkgs, ... }: {
            boot.isContainer = true;

            networking.firewall.allowedTCPPorts = [ 80 ];

            services.httpd = {
              enable = true;
              adminAddr = "morty@example.org";
            };
          })
        ];
    };

  };
}

To create and run that container, enter following commands. In this example the flake.nix file is in the same directory.

# nixos-container create flake-test --flake .
host IP is 10.233.4.1, container IP is 10.233.4.2

# nixos-container start flake-test

Use agenix secrets in container

To add agenix secrets to a container bind mount the ssh-host.key and import the agenix.nixosModule and set age.identityPaths Source

{ agenix, ... }:
{

  containers."withSecret" = {

    # pass the private key to the container for agenix to decrypt the secret
    bindMounts."/etc/ssh/ssh_host_ed25519_key".isReadOnly = true;

    config =
      {
        config,
        lib,
        pkgs,
        ...
      }:
      {
        imports = [ agenix.nixosModules.default ]; # import agenix-module into the nixos-container

        age.identityPaths = [ "/etc/ssh/ssh_host_ed25519_key" ]; # isn't set automatically when openssh is not setup
        # import the secret
        age.secrets."secret-name" = {
          file = ../secrets/secret.age;
        };
      };
  };
}

Connect two nixos-containers with privateNetwork together with a network bridge

Target:

Create two containers, both with privateNetwork = true;

containerA at 192.168.100.2
- which will access containerB
containerB at 192.168.100.3
- which runs an httpd server at http://localhost:80

They should be connected with a bridge br0 and both should have internet address

Configuration:

Create and configure the bridge and the internet connection

❄︎ configuration.nix

# Give containers access to the internet
networking.nat = {
  enable = true;
  # Use "ve-*" when using nftables instead of iptables
  internalInterfaces = [ "br0" ]; # connect the bridge to the internet
  externalInterface = "wlp5s0";   # the interface that connects to the internet
  # Lazy IPv6 connectivity for the container
  enableIPv6 = true;
};

# Both systemd-networkd and NetworkManager can exist in parallel on the same machine,
# when they manage a distinct set of interfaces.
# If upstream connectivity is managed by NetworkManager (for example, NM handles wifi and networkd does VM networking),
# set systemd.network.wait-online.enable to false so that boot isn't blocked on connectivity that networkd will never provide.
# https://wiki.nixos.org/wiki/Systemd/networkd#When_to_use
boot.initrd.systemd.network.wait-online.enable = false;
systemd.network = {
  enable = true;
  wait-online.enable = false;
  netdevs = {
      # Create the bridge interface
      # The <number>- prefix is because systemd writes different files per configuration at /etc/systemd/network
      # All configuration files are collectively sorted and processed in alphanumeric order,
      # regardless of the directories in which they live.
      # However, files with identical filenames replace each other.
      # It is recommended that each filename is prefixed with a number smaller than "70" (e.g. 10-eth0.network).
      # https://www.freedesktop.org/software/systemd/man/latest/systemd.network.html
      "20-br0" = {
        netdevConfig = {
          Kind = "bridge";
          Name = "br0";
          # Sets a pre-determined mac address, leave empty if you want the system to auto-assign a mac address to the bridge
          # MACAddress = "10:00:00:00:00:01";
        };
      };
  };
  networks = {
      # Connect the bridge ports to the bridge
      # it was unnecessary
      # "30-testcontain1" = {
        # vb- probably refers to virtual bridge
        # adding a hostBridge to each container creates vb- interfaces
        # the container hostname (which unless you manually change it is the container name)
        # should be <=61 characters, otherwise the system cannot rebuild if you force it to create an interface on the host
        # the interfaces have altnames of max 61+3 length, 64 characters
        # "ve-<containerHostname>" is the interface altname you can use
        # matchConfig.Name = "vb-testcontainer1";
        # networkConfig.Bridge = "br0";
        # linkConfig.RequiredForOnline = "enslaved";
      # };
      # "30-testcontain2" = {
        # matchConfig.Name = "vb-testcontainer2";
        # networkConfig.Bridge = "br0";
        # linkConfig.RequiredForOnline = "enslaved";
      # };
    # Configure the bridge for its desired function
    "40-br0" = {
      matchConfig.Name = "br0";
      # the address of the bridge
      # /29 is the netmask, it creates 2^(32-29) = 8 subnets
      # of which 2 are reserved (first and last), Network Address and Broadcast Address
      # the bridge already takes up one subnet, so 3 addresses are already reserved
      # thus, to bridge 2 networks, you need a netmask of <=29
      # https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=29&cip=192.168.100.1&ctype=ipv4&x=Calculate
      # 192.168.100.0 - 192.168.100.7
      # Network Address   Usable Host Range              Broadcast Address
      # 192.168.100.0     192.168.100.1 - 192.168.100.6  192.168.100.7
      address = [
        "192.168.100.1/29"
      ];
      # routingPolicyRules = [
        # custom rules
      # ];
      # unnecessary to configure
      # routes = [
        # {
          # Gateway = "0.0.0.0";
          # Table = "bridge";
        # }
        # {
          # Gateway = "::";
          # Table = "bridge";
        # }
      # ];
      # bridgeConfig = {};
      # Disable address autoconfig when no IP configuration is required
      # networkConfig.LinkLocalAddressing = "no";
      # linkConfig = {
        # or "routable" with IP addresses configured
        # RequiredForOnline = "carrier";
      # };
    };
  };
};

Create and configure containerA

❄︎ configuration.nix

containers.containerA = {
  autoStart = true;
  privateNetwork = true;
  hostBridge = "br0";
  # hostAddress = "192.168.100.1";   # not used when using hostBridge
  localAddress = "192.168.100.2/29"; # should have the netmask if hostBridge is used
  config =
    { config, pkgs, lib, ... }:
    {
      system.stateVersion = "25.11";
      
      networking = {
        # changes the gateway to the Network Address of the bridge, so that it has access to the internet
        # the bridge has access to the internet
        defaultGateway = {
          address = "192.168.100.1";
        };
      };
      
      networking = {
        firewall = {
          enable = true;
          allowedTCPPorts = [
            # 80 unnecessary
          ];
        };
        # Use systemd-resolved inside the container
        # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
        useHostResolvConf = lib.mkForce false;
      };
      
      services.resolved.enable = true;
    };
};

Create and configure containerB

❄︎ configuration.nix

containers.containerB = {
  autoStart = true;
  privateNetwork = true;
  hostBridge = "br0";
  # hostAddress = "192.168.100.1";   # not used when using hostBridge
  localAddress = "192.168.100.3/29"; # should have the netmask if hostBridge is used
  config =
    { config, pkgs, lib, ... }:
    {
      # test http server that uses port :80
      services.httpd = {
        enable = true;
      };
      
      system.stateVersion = "25.11";
      
      networking = {
        # changes the gateway to the Network Address of the bridge, so that it has access to the internet
        # the bridge has access to the internet
        defaultGateway = {
          address = "192.168.100.1";
        };
      };
      
      networking = {
        firewall = {
          enable = true;
          allowedTCPPorts = [
            80
          ];
        };
        # Use systemd-resolved inside the container
        # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
        useHostResolvConf = lib.mkForce false;
      };

      services.resolved.enable = true;
    };
};

You can test the connection by loggining into containerA and pinging containerB, curling to its httpd server or pinging an internet website:

# nixos-container root-login containerA
[root@containerA:~]# ping 192.168.100.3 -c3 # ping containerB
[root@containerA:~]# curl http://192.168.100.3:80 # curl to containerB's httpd server
[root@containerA:~]# ping nixos.org -c3 # ping an internet website

Troubleshooting

I have changed the host's channel and some services are no longer functional

Symptoms:

Lost data in PostgreSQL database
MySQL has changed its path, where it creates the database

Solution

If you did not have a system.stateVersion option set inside your declarative container configuration, it will use the default one for the channel. Your data might be safe, if you did nothing meanwhile. Add the missing system.stateVersion to your container, rebuild, and possibly stop/start the container.