Networking: Difference between revisions

From NixOS Wiki
Axka (talk | contribs)
Add link aggregation
Axka (talk | contribs)
Update link aggregation section
Line 241: Line 241:
! Bonding mode !! Description !! Switch configuration
! Bonding mode !! Description !! Switch configuration
|-
|-
| <code>balance-rr</code> || Transmit packets round-robin. || Requires static EtherChannel enabled, not LACP-negotiated.
| <code>balance-rr</code> || '''Default'''. Transmit packets round-robin. || Requires static EtherChannel enabled, not LACP-negotiated.
|-
|-
| <code>active-backup</code> || Only one slave in the bond in active. If it fails, another one is picked to be active. || No configuration required on the switch.
| <code>active-backup</code> || Recommended for fault tolerance when 802.3ad isn't available. Only one slave in the bond in active. If it fails, another one is picked to be active. || No configuration required on the switch.
|-
|-
| <code>balance-xor</code> || Transmit packets based on the selected transmit hash policy. || Requires static EtherChannel enabled, not LACP-negotiated.
| <code>balance-xor</code> || Transmit packets based on the selected transmit hash policy. || Requires static EtherChannel enabled, not LACP-negotiated.
Line 249: Line 249:
| <code>broadcast</code> || Transmit everything on all slave interfaces. || Requires static EtherChannel enabled, not LACP-negotiated.
| <code>broadcast</code> || Transmit everything on all slave interfaces. || Requires static EtherChannel enabled, not LACP-negotiated.
|-
|-
| <code>802.3ad</code> || IEEE 802.3ad Dynamic link aggregation. Transmits packets based on the selected transmit hash policy. || Requires LACP-negotiated EtherChannel enabled. In simpler terms, dynamic LACP.
| <code>802.3ad</code> || '''Recommended'''. IEEE 802.3ad Dynamic link aggregation. Transmits packets based on the selected transmit hash policy. || Requires LACP-negotiated EtherChannel enabled. In simpler terms, dynamic LACP.
|-
|-
| <code>balance-tlb</code> || Adaptive transmit load balancing || No configuration required on the switch.
| <code>balance-tlb</code> || Adaptive transmit load balancing || No configuration required on the switch.
Line 260: Line 260:
=== NetworkManager ===
=== NetworkManager ===


{{Warning|This has not been fully tested.}}
{{Warning|This has not been fully tested. I'm not sure if all the properties are required.}}


{{file|/etc/nixos/configuration.nix|nix|<nowiki>
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
Line 266: Line 266:
     "Bond connection 1" = {
     "Bond connection 1" = {
       bond = {
       bond = {
        downdelay = "0";
         miimon = "100"; # Monitor MII link every 100ms
         miimon = "1";
         mode = "802.3ad";
         mode = "802.3ad"; # dynamic LACP
         xmit_hash_policy = "layer3+4"; # IP and TCP/UDP hash
         updelay = "0";
       };
       };
       connection = {
       connection = {
Line 290: Line 289:
         id = "bond0 port 1";
         id = "bond0 port 1";
         type = "ethernet";
         type = "ethernet";
         interface-name = "eth1"; # Replace this
         interface-name = "enp2s0";
         controller = "bond0";
         controller = "bond0";
         port-type = "bond";
         port-type = "bond";
Line 299: Line 298:
         id = "bond0 port 2";
         id = "bond0 port 2";
         type = "ethernet";
         type = "ethernet";
         interface-name = "eth2"; # Replace this
         interface-name = "enp3s0";
         controller = "bond0";
         controller = "bond0";
         port-type = "bond";
         port-type = "bond";
Line 307: Line 306:
</nowiki>}}
</nowiki>}}


=== systemd-networkd ===
=== systemd-networkd and scripted networking ===


=== legacy scripted networking? ===
See [[Systemd/networkd#Bonding]] for more detailed configuration possibilities.
 
{{file|/etc/nixos/configuration.nix|nix|<nowiki>
  networking.bonds = {
    bond0 = {
      interfaces = [ "enp2s0" "enp3s0" ];
      driverOptions = {
        miimon = "100"; # Monitor MII link every 100ms
        mode = "802.3ad";
        xmit_hash_policy = "layer3+4"; # IP and TCP/UDP hash
      };
    };
  };
</nowiki>}}


=== Teaming ===
=== Teaming ===

Revision as of 01:57, 26 January 2025

Networking config always goes in your system configuration. This can be done declaratively as shown in the following sections or through non-declarative tools such as NetworkManager.

Configuration

Static IP for network adapter

The following example configures a static IPv4 and IPv6 address and a default gateway for the interface ens3

networking = {
  interfaces.ens3 = {
    ipv6.addresses = [{
      address = "2a01:4f8:1c1b:16d0::1";
      prefixLength = 64;
    }];
    ipv4.addresses = [{
      address = "192.0.2.2";
      prefixLength = 24;
    }];
  };
  defaultGateway = {
    address = "192.0.2.1";
    interface = "ens3";
  };
  defaultGateway6 = {
    address = "fe80::1";
    interface = "ens3";
  };
};

Hosts file

To edit /etc/hosts just add something like this to your configuration.nix:

networking.hosts = {
  "127.0.0.2" = ["other-localhost"];
  "192.0.2.1" = ["mail.example.com" "imap.example.com"];
};

Port forwarding

In this example we're going to forward the port 80 via NAT from our internal network interface ens3 to the host 10.100.0.3 on our external interface wg0.

networking = {
  firewall = {
    enable = true;
    allowedTCPPorts = [ 80 ];
  };
  nat = {
    enable = true;
    internalInterfaces = [ "ens3" ];
    externalInterface = "wg0";
    forwardPorts = [
      {
        sourcePort = 80;
        proto = "tcp";
        destination = "10.100.0.3:80";
      }
    ];
  };
  # Previous section is equivalent to :
  nftables = {
    enable = true;
    ruleset = ''
        table ip nat {
          chain PREROUTING {
            type nat hook prerouting priority dstnat; policy accept;
            iifname "ens3" tcp dport 80 dnat to 10.100.0.3:80
          }
        }
    '';
  };
};

For IPv6 port forwarding, the example would look like this. Incoming connections on the address 2001:db8:: and port 80 will be forwarded to [fe80::1234:5678:9abc:def0]:80.

networking = {
  firewall = {
    enable = true;
    allowedTCPPorts = [ 80 ];
  };
  nat = {
    enable = true;
    internalInterfaces = [ "ens3" ];
    externalInterface = "wg0";
    enableIPv6 = true;
    internalIPv6s = [ "2001:db8::/64" ];
    externalIPv6 = "fe80::1234:5678:9abc:def0";
    forwardPorts = [
      {
        sourcePort = 80;
        proto = "tcp";
        destination = "fe80::1234:5678:9abc:def0]:80";
      }
    ];
  };
  # Previous section is equivalent to :
  nftables = {
    enable = true;
    ruleset = ''
        table ip6 nat {
          chain PREROUTING {
            type nat hook prerouting priority dstnat; policy accept;
            iifname "ens3" ip6 daddr [2001:db8::] tcp dport 80 dnat to [fe80::1234:5678:9abc:def0]:80
          }
        }
    '';
  };
};

IPv6

Prefix delegation with fixed DUID

Sometimes the hosting provider manages IPv6 networks via a so-called DUID or clientid. This snippet is required to make the network routable:

{ config, pkgs, ... }:

let
  # Get this from your hosting provider
  clientid = "00:11:22:33:44:55:66:77:88:99";
  interface = "enp2s0";
  subnet =  "56";
  network = "2001:bbb:3333:1111::/${subnet}";
  own_ip =  "2001:bbb:3333:1111::1/${subnet}";
in {
  # ... snip ...

  networking.enableIPv6 = true;
  networking.useDHCP = true;
  networking.dhcpcd.persistent = true;
  networking.dhcpcd.extraConfig = ''
    clientid "${clientid}"
    noipv6rs
    interface ${interface}
    ia_pd 1/${network} ${interface}
    static ip6_address=${own_ip}
  '';
  environment.etc."dhcpcd.duid".text = clientid;

}

Source: gleber gist for online.net IPv6 config in NixOS

Note: Recent versions of dhcpcd move the duid file to /var/db/dcpcd/duid. For that to work, you have to replace the above environment.etc line with something like:

systemd.services.dhcpcd.preStart = ''
  cp ${pkgs.writeText "duid" "<ID>"} /var/db/dhcpcd/duid
'';

IPv6-mostly

For IPv6 mostly networks the situation in Linux is a little bit dire. A 464XLAT CLAT implementation on the client device has to be running.

For example run clatd:

{
  services.clatd.enable = true;
}

Caveats:

  • disable IPv4 manually for DHCPv4 clients that do not accept Option 108 (IPv6-Only Preferred Option)
  • set NAT64 prefix manually, if client doesn't support RA/PREF64 (RFC 8781) or DNS64 (RFC 7050):
{
  services.clatd.settings = {
    plat-prefix = "64:ff9b::/96";
  };
}
  • clatd needs to be restarted, if the network has changed

Sources:

VLANs

Refer to networking.vlans in the manual.

Below is a complete networking example showing two interfaces, one with VLAN trunk tagging and one without.

enp2s1 is a normal network interface at 192.168.1.2 with no VLAN information.

enp2s0 is the virtual LAN trunk with two tagged VLANs, vlan100 and vlan101.

vlan100 is in the 10.1.1.X network and vlan101 is in the 10.10.10.X network.

The hostID should be unique among your machines, as mentioned in the manual.

Complete networking section example:

    networking = {
      hostId = "deadb33f";
      hostName = "nixos";
      domain = "example.com";
      dhcpcd.enable = false;
      interfaces.enp2s1.ipv4.addresses = [{
        address = "192.168.1.2";
        prefixLength = 28;
      }];
      vlans = {
        vlan100 = { id=100; interface="enp2s0"; };
        vlan101 = { id=101; interface="enp2s0"; };
      };
      interfaces.vlan100.ipv4.addresses = [{
        address = "10.1.1.2";
        prefixLength = 24;
      }];
      interfaces.vlan101.ipv4.addresses = [{
        address = "10.10.10.3";
        prefixLength = 24;
      }];
      defaultGateway = "192.168.1.1";
      nameservers = [ "1.1.1.1" "8.8.8.8" ];
    };

Link aggregation

Link aggregation, also known as bonding or trunking is the combining of multiple network links in parallel. This guide focuses on creating a Link Aggregation Group (LAG, bond, or trunk) using LACP (Link Aggregation Content Protocol).

Bonding modes
Bonding mode Description Switch configuration
balance-rr Default. Transmit packets round-robin. Requires static EtherChannel enabled, not LACP-negotiated.
active-backup Recommended for fault tolerance when 802.3ad isn't available. Only one slave in the bond in active. If it fails, another one is picked to be active. No configuration required on the switch.
balance-xor Transmit packets based on the selected transmit hash policy. Requires static EtherChannel enabled, not LACP-negotiated.
broadcast Transmit everything on all slave interfaces. Requires static EtherChannel enabled, not LACP-negotiated.
802.3ad Recommended. IEEE 802.3ad Dynamic link aggregation. Transmits packets based on the selected transmit hash policy. Requires LACP-negotiated EtherChannel enabled. In simpler terms, dynamic LACP.
balance-tlb Adaptive transmit load balancing No configuration required on the switch.
balance-alb Adaptive load balancing No configuration required on the switch.

NetworkManager

Warning: This has not been fully tested. I'm not sure if all the properties are required.
/etc/nixos/configuration.nix
  networking.networkmanager.ensureProfiles.profiles = {
    "Bond connection 1" = {
      bond = {
        miimon = "100"; # Monitor MII link every 100ms
        mode = "802.3ad";
        xmit_hash_policy = "layer3+4"; # IP and TCP/UDP hash
      };
      connection = {
        id = "Bond connection 1";
        interface-name = "bond0"; # Make sure this matches the controller properties
        type = "bond";
      };
      ipv4 = {
        method = "auto";
      };
      ipv6 = {
        addr-gen-mode = "stable-privacy";
        method = "auto";
      };
      proxy = { };
    };
    # No more automatically generated "Wired connection 1"
    "bond0 port 1" = {
      connection = {
        id = "bond0 port 1";
        type = "ethernet";
        interface-name = "enp2s0";
        controller = "bond0";
        port-type = "bond";
      };
    };
    "bond0 port 2" = {
      connection = {
        id = "bond0 port 2";
        type = "ethernet";
        interface-name = "enp3s0";
        controller = "bond0";
        port-type = "bond";
      };
    };
  };

systemd-networkd and scripted networking

See Systemd/networkd#Bonding for more detailed configuration possibilities.

/etc/nixos/configuration.nix
  networking.bonds = {
    bond0 = {
      interfaces = [ "enp2s0" "enp3s0" ];
      driverOptions = {
        miimon = "100"; # Monitor MII link every 100ms
        mode = "802.3ad";
        xmit_hash_policy = "layer3+4"; # IP and TCP/UDP hash
      };
    };
  };

Teaming

Using the teaming driver provides more configuration capabilities since more descision-making is done in userspace [1].

References