Official NixOS Wiki

NixOS Containers: Difference between revisions

← Older edit
DHCP (talk | contribs)
232 edits
VisualWikitext
DHCP (talk | contribs)
232 edits
m update system.stateVersion to 26.05
 
(7 intermediate revisions by one other user not shown)
Line 2: Line 2:


See [[Docker]] page for OCI container (Docker, Podman) configuration.
See [[Docker]] page for OCI container (Docker, Podman) configuration.
=== Host Configuration ===
For all of the examples below to work, you'll have to enable virtualization and the use of containers in your host systems nix configuration.{{file|||<nowiki>
boot.enableContainers = true;
virtualisation.containers.enable = true;
</nowiki>|name=/etc/nixos/configuration.nix|lang=nix}}


=== Configuration ===
=== Configuration ===


The following example creates a container called webserver running a httpd web server. It will start automatically at boot and has its private network subnet.
The following example creates a container called webserver running a httpd web server. It will start automatically at boot and has its private network subnet.
 
{{file|3=networking.nat = {
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
networking.nat = {
   enable = true;
   enable = true;
  # Use "ve-*" when using nftables instead of iptables
   internalInterfaces = ["ve-+"];
   internalInterfaces = ["ve-+"];
   externalInterface = "ens3";
   externalInterface = "ens3";
Line 46: Line 39:
     services.resolved.enable = true;
     services.resolved.enable = true;


     system.stateVersion = "24.11";
     system.stateVersion = "26.05";
   };
   };
};
};|name=/etc/nixos/configuration.nix|lang=nix}}
</nowiki>}}


In order to reach the web application on the host system, we have to open [[Firewall]] port 80 and also configure NAT through <code>networking.nat</code>. The web service of the container will be available at http://192.168.100.11
In order to reach the web application on the host system, we have to open [[Firewall]] port 80 and also configure NAT through <code>networking.nat</code>. The web service of the container will be available at http://192.168.100.11
Line 61: Line 53:
'''NAT (Network Address Translation)'''
'''NAT (Network Address Translation)'''


<syntaxhighlight lang="nix">
In order to allow the container to connect to the internet, you have to configure NAT through <code>networking.nat</code>.
</syntaxhighlight>
{{File|3=networking.nat = {
  enable = true;
  # Use "ve-*" when using nftables instead of iptables
  internalInterfaces = ["ve-+"];
  externalInterface = "ens3";
  # Lazy IPv6 connectivity for the container
  enableIPv6 = true;
};|name=/etc/nixos/configuration.nix|lang=nix}}'''Bridge'''


'''Bridge'''
Connect a container to a bridge using Network Manager interfaces:
 
{{File|3=networking = {
<syntaxhighlight lang="nix">
networking = {
   bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly
   bridges.br0.interfaces = [ "eth0s31f6" ]; # Adjust interface accordingly
    
    
Line 88: Line 85:
   localAddress = "192.168.100.5/24";
   localAddress = "192.168.100.5/24";
   config = { };
   config = { };
};
};|name=/etc/nixos/configuration.nix|lang=nix}}'''Without privateNetwork (simpler)'''
</syntaxhighlight>
 
==== Without privateNetwork (simpler) ====


If the service can be accessed by changing its port, the private network is not needed necessarily. Be careful to not use occupied ports. This example runs an [[Actual]] server on port 3003. It can be accessed through the host at <code>http://localhost:3003</code>. Since <code>privateNetwork</code> is not defined, it defaults to <code>false</code>.
If the service can be accessed by changing its port, the private network is not needed necessarily. Be careful to not use occupied ports. This example runs an [[Actual]] server on port 3003. It can be accessed through the host at <code>http://localhost:3003</code>. Since <code>privateNetwork</code> is not defined, it defaults to <code>false</code>.


<syntaxhighlight lang="nix">
{{File|3=containers.actualContainer = {
containers.actualContainer = {
   autoStart = true;
   autoStart = true;
   config = {...}: {
   config = {...}: {
Line 104: Line 97:
     };
     };
   };
   };
};
};|name=/etc/nixos/configuration.nix|lang=nix}}
</syntaxhighlight>


=== Usage ===
=== Usage ===
Line 135: Line 127:
</syntaxhighlight>
</syntaxhighlight>


View log for container<syntaxhighlight lang="console">
View log for container
<syntaxhighlight lang="console">
# journalctl -M webserver
# journalctl -M webserver
</syntaxhighlight>Further informations are available in the {{manual:nixos|sec=#ch-containers|chapter=NixOS manual}}.
</syntaxhighlight>
Further informations are available in the {{manual:nixos|sec=#ch-containers|chapter=NixOS manual}}.


== Tips and tricks ==
== Tips and tricks ==


==== Define and create nixos-container from a Flake file ====
=== Define and create nixos-container from a Flake file ===
We can define and create a custom container called <code>container</code> from a file stored as <code>flake.nix</code>. In this case we use the unstable branch of the nixpkgs repository as a source.<syntaxhighlight lang="nix">
 
{
We can define and create a custom container called <code>container</code> from a file stored as <code>flake.nix</code>. In this case we use the unstable branch of the nixpkgs repository as a source.
{{File|3={
   inputs.nixpkgs.url = "nixpkgs/nixos-unstable";
   inputs.nixpkgs.url = "nixpkgs/nixos-unstable";


Line 165: Line 160:


   };
   };
}
}|name=/etc/nixos/configuration.nix|lang=nix}}
</syntaxhighlight>To create and run that container, enter following commands. In this example the <code>flake.nix</code> file is in the same directory.<syntaxhighlight lang="console">
 
To create and run that container, enter following commands. In this example the <code>flake.nix</code> file is in the same directory.
<syntaxhighlight lang="console">
# nixos-container create flake-test --flake .
# nixos-container create flake-test --flake .
host IP is 10.233.4.1, container IP is 10.233.4.2
host IP is 10.233.4.1, container IP is 10.233.4.2
Line 173: Line 170:
</syntaxhighlight>
</syntaxhighlight>


==== Use agenix secrets in container ====
=== Use agenix secrets in container ===
To add <code>agenix</code> secrets to a container bind mount the <code>ssh-host.key</code> and import the <code>agenix.nixosModule</code> and set <code>age.identityPaths</code> [https://discourse.nixos.org/t/secrets-inside-nixos-containers/34403/6 Source]<syntaxhighlight lang="nix">
 
{ agenix, ... }:
To add <code>agenix</code> secrets to a container bind mount the <code>ssh-host.key</code> and import the <code>agenix.nixosModule</code> and set <code>age.identityPaths</code> [https://discourse.nixos.org/t/secrets-inside-nixos-containers/34403/6 Source]
{{File|3={ agenix, ... }:
{
{


Line 200: Line 198:
       };
       };
   };
   };
}
}|name=/etc/nixos/configuration.nix|lang=nix}}
</syntaxhighlight>


==== Connect two nixos-containers with privateNetwork together with a network bridge ====
=== Bridge together two nixos-containers ===
'''Target:'''
'''Target:'''


Create two containers, both with <code>privateNetwork = true;</code>
Create two containers, both with <code>privateNetwork = true;</code>:


* <code>containerA</code> at 192.168.100.2
* <code>containerA</code> at 192.168.100.2
Line 213: Line 210:
** which runs an httpd server at http://localhost:80
** which runs an httpd server at http://localhost:80


They should be connected with a bridge <code>br0</code> and both should have internet address
They should be connected with a bridge <code>br0</code> and both should have internet address.
 
Assuming Network Manager is used, so the introduction of <code>systemd.network</code> should not interfere with the rest of the setup.


'''Configuration:'''
'''Configuration:'''


* Create and configure the bridge and the internet connection
Create and configure the internet connection and the bridge:
{{File|3=# Give containers access to the internet
{{File|3=# Give containers access to the internet
networking.nat = {
networking.nat = {
   enable = true;
   enable = true;
  # Use "ve-*" when using nftables instead of iptables
   internalInterfaces = [ "br0" ]; # Connect the bridge to the internet
   internalInterfaces = [ "br0" ]; # connect the bridge to the internet
   externalInterface = "wlp5s0";  # Adjust according to your internet interface
   externalInterface = "wlp5s0";  # the interface that connects to the internet
   # Lazy IPv6 connectivity for the container
   # Lazy IPv6 connectivity for the container
   enableIPv6 = true;
   enableIPv6 = true;
Line 233: Line 231:
# set systemd.network.wait-online.enable to false so that boot isn't blocked on connectivity that networkd will never provide.
# set systemd.network.wait-online.enable to false so that boot isn't blocked on connectivity that networkd will never provide.
# https://wiki.nixos.org/wiki/Systemd/networkd#When_to_use
# https://wiki.nixos.org/wiki/Systemd/networkd#When_to_use
boot.initrd.systemd.network.wait-online.enable = false;
systemd.network = {
systemd.network = {
   enable = true;
   enable = true;
Line 239: Line 236:
   netdevs = {
   netdevs = {
       # Create the bridge interface
       # Create the bridge interface
       # The <number>- prefix is because systemd writes different files per configuration at /etc/systemd/network
       # Each interface is stored as a seperate file under /etc/systemd/network by default
       # All configuration files are collectively sorted and processed in alphanumeric order,
       # <number>-name is required so that it is not overwritten by other configurations of the same name
      # regardless of the directories in which they live.
      # However, files with identical filenames replace each other.
       # It is recommended that each filename is prefixed with a number smaller than "70" (e.g. 10-eth0.network).
       # It is recommended that each filename is prefixed with a number smaller than "70" (e.g. 10-eth0.network).
       # https://www.freedesktop.org/software/systemd/man/latest/systemd.network.html
       # https://www.freedesktop.org/software/systemd/man/latest/systemd.netdev.html
       "20-br0" = {
       "20-br0" = {
         netdevConfig = {
         netdevConfig = {
           Kind = "bridge";
           Kind = "bridge";
           Name = "br0";
           Name = "br0";
           # Sets a pre-determined mac address, leave empty if you want the system to auto-assign a mac address to the bridge
           # Sets a pre-determined mac address
          # Leave empty if you want the system to auto-assign a mac address to the bridge
           # MACAddress = "10:00:00:00:00:01";
           # MACAddress = "10:00:00:00:00:01";
         };
         };
Line 255: Line 251:
   };
   };
   networks = {
   networks = {
      # Connect the bridge ports to the bridge
      # it was unnecessary
      # "30-testcontain1" = {
        # vb- probably refers to virtual bridge
        # adding a hostBridge to each container creates vb- interfaces
        # the container hostname (which unless you manually change it is the container name)
        # should be <=61 characters, otherwise the system cannot rebuild if you force it to create an interface on the host
        # the interfaces have altnames of max 61+3 length, 64 characters
        # "ve-<containerHostname>" is the interface altname you can use
        # matchConfig.Name = "vb-testcontainer1";
        # networkConfig.Bridge = "br0";
        # linkConfig.RequiredForOnline = "enslaved";
      # };
      # "30-testcontain2" = {
        # matchConfig.Name = "vb-testcontainer2";
        # networkConfig.Bridge = "br0";
        # linkConfig.RequiredForOnline = "enslaved";
      # };
     # Configure the bridge for its desired function
     # Configure the bridge for its desired function
     "40-br0" = {
     "40-br0" = {
       matchConfig.Name = "br0";
       matchConfig.Name = "br0";
       # the address of the bridge
       # The address of the bridge
       # /29 is the netmask, it creates 2^(32-29) = 8 subnets
       # /29 is the netmask, it creates 2^(32-29) = 8 subnets
       # of which 2 are reserved (first and last), Network Address and Broadcast Address
       # 2 are reserved (first and last), as Network Address and Broadcast Address
       # the bridge already takes up one subnet, so 3 addresses are already reserved
       # The bridge already takes up one subnet, so 3 addresses are already reserved
       # thus, to bridge 2 networks, you need a netmask of <=29
       # To bridge 2 networks, you need a netmask of <=29 for IPv4
       # https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=29&cip=192.168.100.1&ctype=ipv4&x=Calculate
       # https://www.calculator.net/ip-subnet-calculator.html?cclass=any&csubnet=29&cip=192.168.100.1&ctype=ipv4&x=Calculate
       # 192.168.100.0 - 192.168.100.7
       # 192.168.100.0 - 192.168.100.7
Line 288: Line 266:
         "192.168.100.1/29"
         "192.168.100.1/29"
       ];
       ];
      # routingPolicyRules = [
        # custom rules
      # ];
      # unnecessary to configure
      # routes = [
        # {
          # Gateway = "0.0.0.0";
          # Table = "bridge";
        # }
        # {
          # Gateway = "::";
          # Table = "bridge";
        # }
      # ];
       # bridgeConfig = {};
       # bridgeConfig = {};
       # Disable address autoconfig when no IP configuration is required
       # Disable address autoconfig when no IP configuration is required
Line 311: Line 275:
     };
     };
   };
   };
};|name=configuration.nix|lang=nix}}
};|name=/etc/nixos/configuration.nix|lang=nix}}
* Create and configure <code>containerA</code>
 
Create and configure <code>containerA</code>:
{{File|3=containers.containerA = {
{{File|3=containers.containerA = {
   autoStart = true;
   autoStart = true;
   privateNetwork = true;
   privateNetwork = true;
   hostBridge = "br0";
   hostBridge = "br0";
   # hostAddress = "192.168.100.1";  # not used when using hostBridge
   # hostAddress = "192.168.100.1";  # Not used when using hostBridge
   localAddress = "192.168.100.2/29"; # should have the netmask if hostBridge is used
   localAddress = "192.168.100.2/29"; # Should have the netmask if hostBridge is used
   config =
   config =
     { config, pkgs, lib, ... }:
     { config, pkgs, lib, ... }:
     {
     {
       system.stateVersion = "25.11";
       system.stateVersion = "26.05";
        
        
       networking = {
       networking = {
         # changes the gateway to the Network Address of the bridge, so that it has access to the internet
         # Changes the gateway to the Network Address of the bridge, so that it has access to the internet
         # the bridge has access to the internet
         # The bridge has access to the internet
         defaultGateway = {
         defaultGateway = {
           address = "192.168.100.1";
           address = "192.168.100.1";
Line 333: Line 298:
        
        
       networking = {
       networking = {
        firewall = {
          enable = true;
          allowedTCPPorts = [
            # 80 unnecessary
          ];
        };
         # Use systemd-resolved inside the container
         # Use systemd-resolved inside the container
         # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
         # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
Line 346: Line 305:
       services.resolved.enable = true;
       services.resolved.enable = true;
     };
     };
};|name=configuration.nix|lang=nix}}
};|name=/etc/nixos/configuration.nix|lang=nix}}
* Create and configure <code>containerB</code>
 
Create and configure <code>containerB</code>:
{{File|3=containers.containerB = {
{{File|3=containers.containerB = {
   autoStart = true;
   autoStart = true;
   privateNetwork = true;
   privateNetwork = true;
   hostBridge = "br0";
   hostBridge = "br0";
   # hostAddress = "192.168.100.1";  # not used when using hostBridge
   # hostAddress = "192.168.100.1";  # Not used when using hostBridge
   localAddress = "192.168.100.3/29"; # should have the netmask if hostBridge is used
   localAddress = "192.168.100.3/29"; # Should have the netmask if hostBridge is used
   config =
   config =
     { config, pkgs, lib, ... }:
     { config, pkgs, lib, ... }:
Line 362: Line 322:
       };
       };
        
        
       system.stateVersion = "25.11";
       system.stateVersion = "26.05";
        
        
       networking = {
       networking = {
         # changes the gateway to the Network Address of the bridge, so that it has access to the internet
         # Changes the gateway to the Network Address of the bridge, so that it has access to the internet
         # the bridge has access to the internet
         # The bridge has access to the internet
         defaultGateway = {
         defaultGateway = {
           address = "192.168.100.1";
           address = "192.168.100.1";
Line 373: Line 333:
        
        
       networking = {
       networking = {
         firewall = {
         firewall.allowedTCPPorts = [ 80 ];
          enable = true;
          allowedTCPPorts = [
            80
          ];
        };
         # Use systemd-resolved inside the container
         # Use systemd-resolved inside the container
         # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
         # Workaround for bug https://github.com/NixOS/nixpkgs/issues/162686
Line 386: Line 341:
       services.resolved.enable = true;
       services.resolved.enable = true;
     };
     };
};|name=configuration.nix|lang=nix}}
};|name=/etc/nixos/configuration.nix|lang=nix}}
 
You can test the connection between <code>containerA</code> and <code>containerB</code> by loggining into <code>containerA</code> and pinging <code>containerB</code>, curling to <code>containerB</code>'s httpd server or pinging an internet website:
<syntaxhighlight lang="console">
# nixos-container root-login containerA
[root@containerA:~]# ping 192.168.100.3 -c3      # Ping containerB
[root@containerA:~]# curl http://192.168.100.3:80 # Curl to containerB's httpd server
[root@containerA:~]# ping nixos.org -c3          # Ping an internet website
</syntaxhighlight>
 
You can test the connection between the host machine and <code>containerA</code> or <code>containerB</code> by pinging <code>containerA</code>,  pinging <code>containerB</code> and curling to <code>containerB</code>'s httpd server:
<syntaxhighlight lang="console">
$ ping 192.168.100.2 -c3      # Ping containerA
$ ping 192.168.100.3 -c3      # Ping containerB
$ curl http://192.168.100.3:80 # Curl to containerB's httpd server
</syntaxhighlight>
 
Note that with the command <code>ip address</code>, even if the interfaces of the containers are displayed (<code>vb-containerA</code> and <code>vb-containerB</code>), they only have a MAC address assigned, they do not have a separate ip address displayed. For extra configuring, maybe use the option <code>containers.<name>.extraVeths</code>.


You can test the connection by loggining into containerA and pinging containerB, curling to its httpd server or pinging an internet website:<syntaxhighlight lang="console"># nixos-container root-login containerA
Made with help of the <code>systemd.network</code> wiki page<ref>[[Systemd/networkd]]</ref> and this discourse post<ref>https://discourse.nixos.org/t/how-to-connect-two-or-more-nixos-containers-together-their-internet-ports/77674/9?u=blastboomstrice</ref>.
[root@containerA:~]# ping 192.168.100.3 -c3 # ping containerB
[root@containerA:~]# curl http://192.168.100.3:80 # curl to containerB's httpd server
[root@containerA:~]# ping nixos.org -c3 # ping an internet website</syntaxhighlight>


== Troubleshooting ==
== Troubleshooting ==


==== I have changed the host's channel and some services are no longer functional ====
=== I have changed the host's channel and some services are no longer functional ===
'''Symptoms:'''
'''Symptoms:'''
* Lost data in PostgreSQL database
* Lost data in PostgreSQL database
Retrieved from "https://wiki.nixos.org/wiki/NixOS_Containers"