K3s: Difference between revisions
imported>Rockofox m Small spelling mistake |
imported>Superherointj Add Storage section with Longhorn and NFS information |
||
Line 159: | Line 159: | ||
value: all | value: all | ||
to test its working exec onto a pod and run <code>nvidia-smi</code>. For more configurability of nvidia related matters in k3s look in [https://docs.k3s.io/advanced#nvidia-container-runtime-support k3s-docs] | to test its working exec onto a pod and run <code>nvidia-smi</code>. For more configurability of nvidia related matters in k3s look in [https://docs.k3s.io/advanced#nvidia-container-runtime-support k3s-docs] | ||
== Storage == | |||
=== Longhorn === | |||
NixOS configuration required for Longhorn: | |||
<syntaxHighlight lang=nix> | |||
environment.systemPackages = [ pkgs.nfs-utils ]; | |||
services.openiscsi = { | |||
enable = true; | |||
name = "hostname-initiatorhost"; | |||
}; | |||
</syntaxHighlight> | |||
Longhorn container has trouble with NixOS path. Solution is to override PATH environment variable, such as: | |||
<syntaxHighlight lang=bash> | |||
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/run/wrappers/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin | |||
</syntaxHighlight> | |||
==== Kyverno Policy for Fixing Longhorn Container ==== | |||
<syntaxHighlight lang=yaml> | |||
--- | |||
apiVersion: v1 | |||
kind: ConfigMap | |||
metadata: | |||
name: longhorn-nixos-path | |||
namespace: longhorn-system | |||
data: | |||
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/run/wrappers/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin | |||
--- | |||
apiVersion: kyverno.io/v1 | |||
kind: ClusterPolicy | |||
metadata: | |||
name: longhorn-add-nixos-path | |||
annotations: | |||
policies.kyverno.io/title: Add Environment Variables from ConfigMap | |||
policies.kyverno.io/subject: Pod | |||
policies.kyverno.io/category: Other | |||
policies.kyverno.io/description: >- | |||
Longhorn invokes executables on the host system, and needs | |||
to be aware of the host systems PATH. This modifies all | |||
deployments such that the PATH is explicitly set to support | |||
NixOS based systems. | |||
spec: | |||
rules: | |||
- name: add-env-vars | |||
match: | |||
resources: | |||
kinds: | |||
- Pod | |||
namespaces: | |||
- longhorn-system | |||
mutate: | |||
patchStrategicMerge: | |||
spec: | |||
initContainers: | |||
- (name): "*" | |||
envFrom: | |||
- configMapRef: | |||
name: longhorn-nixos-path | |||
containers: | |||
- (name): "*" | |||
envFrom: | |||
- configMapRef: | |||
name: longhorn-nixos-path | |||
--- | |||
</syntaxHighlight> | |||
=== NFS === | |||
NixOS configuration required for NFS: | |||
<syntaxHighlight lang=nix> | |||
boot.supportedFilesystems = [ "nfs" ]; | |||
services.rpcbind.enable = true; | |||
</syntaxHighlight> | |||
== Troubleshooting == | == Troubleshooting == |