K3s: Difference between revisions

From NixOS Wiki
imported>Bsima
add k3s package & sudo
imported>Mic92
add multi-node setup
Line 17: Line 17:


After enabling, you can access your cluster through <code>sudo k3s kubectl</code> i.e. <code>sudo k3s kubectl cluster-info</code>, or by using the generated kubeconfig file in <code>/etc/rancher/k3s/k3s.yaml</code>
After enabling, you can access your cluster through <code>sudo k3s kubectl</code> i.e. <code>sudo k3s kubectl cluster-info</code>, or by using the generated kubeconfig file in <code>/etc/rancher/k3s/k3s.yaml</code>
== Multi-node setup ==
See this [https://github.com/Mic92/doctor-cluster-config/tree/master/modules/k3s real world example]. You might want to ignore some parts of it i.e. the monitoring as this is specific to our setup.
The k3s server needs to import <code>modules/k3s/server.nix</code> and an agent <code>modules/k3s/agent.nix</code>.
Tipp: You might run into issues with coredns not being reachable from agent nodes. Right now we disable the NixOS firewall all together until we find a better solution.


== ZFS support ==
== ZFS support ==

Revision as of 18:06, 5 November 2021

K3s is a easier to use version of kubernetes. It bundles all components of a kubernetes cluster into one single binary.

Single node setup

{
  # This is required so that pod can reach the API server (running on port 6443 by default)
  networking.firewall.allowedTCPPorts = [ 6443 ];
  services.k3s.enable = true;
  services.k3s.role = "server";
  services.k3s.extraFlags = toString [
    # "--kubelet-arg=v=4" # Optionally add additional args to k3s
  ];
  environment.systemPackages = [ pkgs.k3s ];
}

After enabling, you can access your cluster through sudo k3s kubectl i.e. sudo k3s kubectl cluster-info, or by using the generated kubeconfig file in /etc/rancher/k3s/k3s.yaml

Multi-node setup

See this real world example. You might want to ignore some parts of it i.e. the monitoring as this is specific to our setup. The k3s server needs to import modules/k3s/server.nix and an agent modules/k3s/agent.nix. Tipp: You might run into issues with coredns not being reachable from agent nodes. Right now we disable the NixOS firewall all together until we find a better solution.

ZFS support

k3s's builtin containerd does not support the zfs snapshotter. However it is possible to configure it to use an external containerd:

  virtualisation.containerd.enable = true;
  # TODO describe how to enable zfs snapshotter in containerd
  services.k3s.extraFlags = toString [
    "--container-runtime-endpoint unix:///run/containerd/containerd.sock"
  ];