Talk:Proxmox Linux Container: Difference between revisions
imported>Vater mNo edit summary |
imported>Vater mNo edit summary |
||
| Line 357: | Line 357: | ||
| | | | ||
| | | | ||
: <code>nano /etc/nixos/configuration.nix</code> | : <code>nano /etc/nixos/configuration.nix</code> | ||
<pre> | <pre> | ||
| Line 384: | Line 373: | ||
} | } | ||
</pre> | </pre> | ||
: <code>nixos-rebuild switch</code> | : <code>nixos-rebuild switch</code> | ||
<pre> | <pre> | ||
building Nix... | building Nix... | ||
building the system configuration... | building the system configuration... | ||
these derivations will be built: | these derivations will be built: | ||
/nix/store/ | /nix/store/4c6mbv3y3i6m9qzv48j0ncy7163x3m7b-system-units.drv | ||
/nix/store/nfc7hz3shyrc1jm2nqsgh5m7izcj8psc-etc.drv | |||
/nix/store/fmx0v41zcyr7g38qjvdsdk5fifkjxfhx-nixos-system-nixos-21.05.3787.564cb4d81d4.drv | |||
building '/nix/store/4c6mbv3y3i6m9qzv48j0ncy7163x3m7b-system-units.drv'... | |||
building '/nix/store/nfc7hz3shyrc1jm2nqsgh5m7izcj8psc-etc.drv'... | |||
building '/nix/store/fmx0v41zcyr7g38qjvdsdk5fifkjxfhx-nixos-system-nixos-21.05.3787.564cb4d81d4.drv'... | |||
/nix/store/ | |||
/nix/store/ | |||
building '/nix/store/ | |||
building '/nix/store/ | |||
building '/nix/store/ | |||
activating the configuration... | activating the configuration... | ||
setting up /etc... | setting up /etc... | ||
reloading user units for root... | reloading user units for root... | ||
setting up tmpfiles | setting up tmpfiles | ||
</pre> | |||
< | : <code>systemctl list-units --failed</code> | ||
<pre> | <pre> | ||
UNIT LOAD ACTIVE SUB DESCRIPTION | |||
0 loaded units listed. | |||
</pre> | </pre> | ||
| https://github.com/NixOS/nixpkgs/issues/9735#issuecomment-783535726 | | https://github.com/NixOS/nixpkgs/issues/9735#issuecomment-783535726 | ||
Revision as of 20:38, 15 October 2021
NixOS container (LXC) on Proxmox Virtual Environment 7
| WUI | CLI | Reference | |
|---|---|---|---|
| find the (a current) NixOS template |
for example for 21.05
| ||
| download the NixOS template |
|
going to the folder where all the templates for Promox Virtual Environment are stored
downloading the NixOS template file (optional you should) move the downloaded NixOS template file to a nicer (more individual) file name
(optional you can) going back
| |
| create a (first) new NixOS container with Proxmox Virtual Environment |
/dev/rbd0 Creating filesystem with 2097152 4k blocks and 524288 inodes Filesystem UUID: 3f4cf224-8062-4cd3-918c-49f891af1aa1 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 extracting archive '/mnt/pve/cephfs/template/cache/nixos-21.05_2021-10-10.tar.xz' Total bytes read: 836218880 (798MiB, 21MiB/s) Architecture detection failed: open '/bin/sh' failed: No such file or directory Falling back to amd64. Use `pct set VMID --arch ARCH` to change. /etc/os-release file not found and autodetection failed, falling back to 'unmanaged' TASK OK (optional you can) check the (pve) lxc config file (and it should look like something like the following)
arch: amd64 cores: 1 features: nesting=1 hostname: CT1000 memory: 512 net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=FE:1E:11:E6:D2:8F,ip=dhcp,type=veth ostype: unmanaged rootfs: storage:vm-1000-disk-0,size=8G swap: 512 unprivileged: 1 |
(optional you can) check the (pve) lxc config file (and it should look like something like the following)
arch: amd64 features: nesting=1 hostname: CT1000 memory: 512 net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=1E:D8:FE:E9:F1:71,ip=dhcp,type=veth ostype: unmanaged rootfs: storage:vm-1000-disk-0,size=4G swap: 512 unprivileged: 1 | |
| (configure the available (virtual) network device for the Proxmox Virtual Environment container) |
| ||
| customize the (pve) lxc startup setup routine (on every node where you want to start a NixOS container) |
run_buffer: 316 Script exited with status 1 lxc_init: 816 Failed to run lxc.hook.pre-start for container "1000" __lxc_start: 2007 Failed to initialize container "1000" TASK ERROR: startup for container '1000' failed
sub unified_cgroupv2_support {
my ($self) = @_;
return if !$self->{plugin}; # unmanaged
$self->protected_call(sub { $self->{plugin}->unified_cgroupv2_support() });
}
(after fixing all the other stuff) WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version. TASK WARNINGS: 1 |
||
| customize the individual Proxmox Virtual Environment container configuration for NixOS |
sync_wait: 36 An error occurred in another process (expected sequence number 7) TASK ERROR: startup for container '1000' failed |
sync_wait: 36 An error occurred in another process (expected sequence number 7) __lxc_start: 2073 Failed to spawn container "1000" TASK ERROR: startup for container '1000' failed editing the specific (pve) lxc config file (to a option for lxc.init.cmd)
lxc.init.cmd: /init | |
| (activate nesting for NixOS in the Proxmox Virtual Environment container) |
(optional you can) check the (pve) lxc config file (and it should look like something like the following with nesting=1 on the line features:)
features: nesting=1 | ||
| start the NixOS container with Proxmox Virtual Environment |
|
WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version. Task finished with 1 warning(s)!
status: running | |
| having a running NixOS container on Proxmox Virtual Environment | |||
| login into the NixOS container |
<<< Welcome to NixOS 21.05.3740.ce7a1190a0f (x86_64) - pts/0 >>> Log in as "root" with an empty password. nixos login: nixos login: root [root@nixos:~]# |
sh-4.4# sh-4.4# . /etc/profile [root@nixos:/]# | |
| update the channel for NixOS (needed!) |
unpacking channels... created 1 symlinks in user environment | ||
| (optional) test rebuild of NixOS |
(optional) check the default configuration file for NixOS
{ config, pkgs, ... }:
{
imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];
}
building Nix...
building the system configuration...
activating the configuration...
setting up /etc...
setting up tmpfiles
warning: the following units failed: sys-kernel-debug.mount
● sys-kernel-debug.mount - Kernel Debug File System
Loaded: loaded (/nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/example/systemd/system/sys-kernel-debug.mount; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-10-11 06:48:30 UTC; 265ms ago
Where: /sys/kernel/debug
What: debugfs
Docs: https://www.kernel.org/doc/Documentation/filesystems/debugfs.txt
https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
IP: 0B in, 0B out
CPU: 2ms
Oct 11 06:48:30 nixos systemd[1]: Mounting Kernel Debug File System...
Oct 11 06:48:30 nixos mount[17997]: mount: /sys/kernel/debug: permission denied.
Oct 11 06:48:30 nixos systemd[1]: sys-kernel-debug.mount: Mount process exited, code=exited, status=32/n/a
Oct 11 06:48:30 nixos systemd[1]: sys-kernel-debug.mount: Failed with result 'exit-code'.
Oct 11 06:48:30 nixos systemd[1]: Failed to mount Kernel Debug File System.
warning: error(s) occurred while switching to the new configuration
| ||
| checking systemd |
UNIT LOAD ACTIVE SUB DESCRIPTION ● sys-kernel-debug.mount loaded failed failed Kernel Debug File System LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. | ||
| fixing the configuration file for NixOS |
{
imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];
systemd.suppressedSystemUnits = [
"sys-kernel-debug.mount"
];
}
building Nix... building the system configuration... these derivations will be built: /nix/store/4c6mbv3y3i6m9qzv48j0ncy7163x3m7b-system-units.drv /nix/store/nfc7hz3shyrc1jm2nqsgh5m7izcj8psc-etc.drv /nix/store/fmx0v41zcyr7g38qjvdsdk5fifkjxfhx-nixos-system-nixos-21.05.3787.564cb4d81d4.drv building '/nix/store/4c6mbv3y3i6m9qzv48j0ncy7163x3m7b-system-units.drv'... building '/nix/store/nfc7hz3shyrc1jm2nqsgh5m7izcj8psc-etc.drv'... building '/nix/store/fmx0v41zcyr7g38qjvdsdk5fifkjxfhx-nixos-system-nixos-21.05.3787.564cb4d81d4.drv'... activating the configuration... setting up /etc... reloading user units for root... setting up tmpfiles
UNIT LOAD ACTIVE SUB DESCRIPTION 0 loaded units listed. |
https://github.com/NixOS/nixpkgs/issues/9735#issuecomment-783535726 | |
| (optional) test changing the configuration of the NixOS container by installing a package (ddate) and disabling a (default running) service |
{
imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];
services.openssh.enable = false; environment.systemPackages = with pkgs; [
ddate
];
}
building Nix... building the system configuration...
Today is Prickle-Prickle, the 65th day of Bureaucracy in the YOLD 3187 | ||
| (optional) check status of systemd |
● nixos
State: running
Jobs: 0 queued
Failed: 0 units
Since: Mon 2021-10-11 11:52:39 UTC; 1h 24min ago
CGroup: /
├─user.slice
│ └─user-0.slice
│ ├─session-c1.scope
│ │ ├─393 /nix/store/61z6l8p3f14hgz29j607bg1d37sn5d86-shadow-4.8.1/bin/login --
│ │ ├─506 -bash
│ │ └─527 top
│ └─user@0.service
│ └─init.scope
│ ├─499 /run/current-system/systemd/lib/systemd/systemd --user --deserialize 17
│ └─500 (sd-pam
├─.lxc
│ ├─ 290 /bin/sh
│ ├─3315 systemctl status
│ └─3316 less
├─init.scope
│ └─1 systemd
└─system.slice
├─systemd-journald.service
│ └─200 /nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/lib/systemd/systemd-journald
├─nix-daemon.service
│ └─473 nix-daemon --daemon
├─console-getty.service
│ └─392 agetty --login-program /nix/store/61z6l8p3f14hgz29j607bg1d37sn5d86-shadow-4.8.1/bin/login --noclear --keep-baud console 115200,38400,9600 linux
├─dhcpcd.service
│ └─468 dhcpcd: [master] [ip4] [ip6]
├─nscd.service
│ └─450 nscd
├─system-container\x2dgetty.slice
│ └─container-getty@1.service
│ └─394 agetty --login-program /nix/store/61z6l8p3f14hgz29j607bg1d37sn5d86-shadow-4.8.1/bin/login --noclear --keep-baud pts/1 115200,38400,9600 vt220
├─dbus.service
│ └─390 /nix/store/qxpxg30bmj99rvvsacqzkgayzvxz6bb1-dbus-1.12.20/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
└─systemd-logind.service
└─364 /nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/lib/systemd/systemd-logind
| ||
| … | |||
Is there a need for non unprivileged NixOS containers?
- Is there a scenario where you would like to have a privileged (
unprivileged = 0) container? - I do not know.
Is there a cgroup2 only NixOS containers?
- Is there a configuration with cgroups versions 2 only - where cgroups version 1 is not needed - (so that is possible to rollback the manipulation of the pve lxc startup script (
/usr/share/perl5/PVE/LXC/Setup.pm))? Or can we create such a (pre)build image (tar)? - I do not know.
grep cgroup /proc/filesystems
nodev cgroup nodev cgroup2
- Notes
https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html
(pve) lxc configuration option lxc.cgroup2.devices.allow: a
(pve) lxc configuration option lxc.cgroup.devices.deny = a
https://wiki.debian.org/LXC/CGroupV2
https://search.nixos.org/options?channel=unstable&from=0&query=cgroup
documentation
proxmox
- https://pve.proxmox.com/pve-docs/
- https://pve.proxmox.com/wiki/Linux_Container
- https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
lxc
- …
debian
- …
linux
- …