Kubernetes: Difference between revisions

← Older edit

@@ Line 1: / Line 1: @@
+[https://kubernetes.io/ Kubernetes] is an open-source container orchestration system for automating software deployment, scaling, and management.
+This wiki article extends the documentation in [https://nixos.org/manual/nixos/stable/#sec-kubernetes NixOS manual].
 == [[wikipedia:en:KISS principle|KISS]] ==
@@ Line 26: / Line 30: @@
 { config, pkgs, ... }:
 let
+  # When using easyCerts=true the IP Address must resolve to the master on creation.
+ # So use simply 127.0.0.1 in that case. Otherwise you will have errors like this https://github.com/NixOS/nixpkgs/issues/59364
    kubeMasterIP = "10.1.1.2";
    kubeMasterHostname = "api.kube";
@@ Line 246: / Line 252: @@
 * reboot all your nodes
 * continue with [https://rook.io/docs/rook/v1.2/ceph-quickstart.html the official quickstart guide]
-* in <code>operator.yaml</code>, set <code>CSI_FORCE_CEPHFS_KERNEL_CLIENT</code> to <code>false</code>
+* in <code>operator.yaml</code>, help the CSI plugins find the hosts' ceph kernel modules by adding (or uncommenting -- they're in the example config) these entries:
+  CSI_CEPHFS_PLUGIN_VOLUME: |
+  - name: lib-modules
+    hostPath:
+      path: /run/current-system/kernel-modules/lib/modules/
+  CSI_RBD_PLUGIN_VOLUME: |
+  - name: lib-modules
+    hostPath:
+      path: /run/current-system/kernel-modules/lib/modules/
 === NVIDIA ===
@@ Line 285: / Line 299: @@
    emptyDir:
    medium: Memory
+</syntaxhighlight>
+=== Arm64 ===
+Nix might pull in <code>coredns</code> and <code>etcd</code> images that are incompatible with arm, To resolve this add the following to your master node's configuration:
+==== etcd ====
+<syntaxhighlight lang=nix>
+  ...
+  services.kubernetes = {...};
+  systemd.services.etcd = {
+    environment = {
+      ETCD_UNSUPPORTED_ARCH = "arm64";
+    };
+  };
+  ...
+</syntaxhighlight>
+==== coredns ====
+<syntaxhighlight lang=nix>
+  services.kubernetes = {
+    ...
+    # use coredns
+    addons.dns = {
+      enable = true;
+      coredns = {
+        finalImageTag = "1.10.1";
+        imageDigest = "sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e";
+        imageName = "coredns/coredns";
+        sha256 = "0c4vdbklgjrzi6qc5020dvi8x3mayq4li09rrq2w0hcjdljj0yf9";
+      };
+    };
+   ...
+  };
 </syntaxhighlight>
@@ Line 291: / Line 336: @@
 There are various community projects aimed at facilitating working with Kubernetes combined with Nix:
 * [https://github.com/saschagrunert/kubernix kubernix]: simple setup of development clusters using Nix
-* [https://github.com/cmollekopf/kube-nix kube-nix]
+* [https://kubenix.org/ kubenix] - [https://github.com/hall/kubenix GitHub (updated 2023)]
-* [https://github.com/hall/kubenix kubenix (updated 2023)]
+* [https://github.com/justinas/nixos-ha-kubernetes nixos-ha-kubernetes]
 == References ==
@@ Line 304: / Line 349: @@
 [[Category:Applications]]
-[[Category:Servers]]
+[[Category:Server]]
-[[Category:orchestration]]
+[[Category:Container]]
+[[Category:NixOS Manual]]