Talk:Proxmox Linux Container: Difference between revisions

From NixOS Wiki
← Older editNewer edit →
VisualWikitext
imported>Vater
imported>Vater
No edit summary
Line 33: Line 33:
| download the NixOS template
| download the NixOS template
|  
|  
* (if you want to switch to the ''Storage View'' and) go to one of your nodes where you can (find and) store ''CT Templates''.
* (if you want to switch to the <tt>Storage View</tt> and) go to one of your nodes where you can (find and) store <tt>CT Templates</tt>
* choose the ''Download from URL'' button
* choose the <tt>Download from URL</tt> button
** add the link for the NiixOS template file ''https://hydra.nixos.org/job/nixos/release-21.05/nixos.containerTarball.x86_64-linux/latest/download-by-type/file/system-tarball'' (to the text field of <tt>URL:</tt>)
** add the link for the NiixOS template file ''https://hydra.nixos.org/job/nixos/release-21.05/nixos.containerTarball.x86_64-linux/latest/download-by-type/file/system-tarball'' (to the text field of <tt>URL:</tt>)
** add a nice file name for the current downloadable build of the template (to the text field of <tt>File name:</tt>)
** add a nice file name for the current downloadable build of the template (to the text field of <tt>File name:</tt>)
Line 50: Line 50:
| create a (first) new NixOS container with Proxmox Virtual Environment  
| create a (first) new NixOS container with Proxmox Virtual Environment  
|  
|  
* (if you want to switch to the ''Folder View'' and go to ''Nodes'' and choose the your nodes where you create the container. (this note will be preselected as ''Node'' in the form.))
* (if you want to switch to the <tt>Folder View</tt> and go to <tt>Nodes</tt> and choose the your nodes where you create the container. (this note will be preselected as ''Node'' in the form.))
* choose the ''Create CT'' button
* choose the <tt>Create CT</tt> button
*: tab <tt>General</tt>
*: tab <tt>General</tt>
*:* (for the following we expect) ''1000'' is prefilled (to the text field of <tt>CT ID:</tt>) or you have added it (because it not already taken)
*:* (for the following we expect) ''1000'' is prefilled (to the text field of <tt>CT ID:</tt>) or you have added it (because it not already taken)
Line 62: Line 62:
*:** the name for the container (to the text field of <tt>Hostname:</tt>)
*:** the name for the container (to the text field of <tt>Hostname:</tt>)
*:** …
*:** …
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>Template</tt>
*: tab <tt>Template</tt>
*:* (for the following we expect) the entry ''cephfs'' is prefilled (at the drop down menu of <tt>Storage:</tt>)
*:* (for the following we expect) the entry ''cephfs'' is prefilled (at the drop down menu of <tt>Storage:</tt>)
*:* (for the following we expect you) find and choose the entry ''nixos-21.05_2021-10-10.tar.xz'' (at the drop down menu of <tt>Storage:</tt>)
*:* (for the following we expect you) find and choose the entry ''nixos-21.05_2021-10-10.tar.xz'' (at the drop down menu of <tt>Storage:</tt>)
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>Root Disk</tt>
*: tab <tt>Root Disk</tt>
*:* (for the following we expect) the entry ''storage'' is prefilled (at the drop down menu of <tt>Storage:</tt>)
*:* (for the following we expect) the entry ''storage'' is prefilled (at the drop down menu of <tt>Storage:</tt>)
*:* (for the following we expect) ''8'' is prefilled (to the text field of <tt>Disk size (GiB):</tt>)
*:* (for the following we expect) ''8'' is prefilled (to the text field of <tt>Disk size (GiB):</tt>)
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>CPU</tt>
*: tab <tt>CPU</tt>
*:* (for the following we expect) ''1'' is prefilled (to the text field of <tt>Cores:</tt>)
*:* (for the following we expect) ''1'' is prefilled (to the text field of <tt>Cores:</tt>)
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>Memory</tt>
*: tab <tt>Memory</tt>
*:* (for the following we expect) ''512'' is prefilled (to the text field of <tt>Memory (MiB):</tt>)
*:* (for the following we expect) ''512'' is prefilled (to the text field of <tt>Memory (MiB):</tt>)
*:* (for the following we expect) ''512'' is prefilled (to the text field of <tt>Swap (MiB):</tt>)
*:* (for the following we expect) ''512'' is prefilled (to the text field of <tt>Swap (MiB):</tt>)
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>Network</tt>
*: tab <tt>Network</tt>
*:* (for the following we expect) ''eth0'' is prefilled (to the text field of <tt>Name:</tt>)
*:* (for the following we expect) ''eth0'' is prefilled (to the text field of <tt>Name:</tt>)
Line 91: Line 91:
*:* (for the following we expect) the entry <tt>Static</tt> is prefilled (at the ratio button menu of <tt>IPv6:</tt>) and you have no network access for IPv6 avilibale
*:* (for the following we expect) the entry <tt>Static</tt> is prefilled (at the ratio button menu of <tt>IPv6:</tt>) and you have no network access for IPv6 avilibale
*:*: ?!? if you have network for IPv6 and you want to have access to your network for IPv6 you have fill out the text field of <tt>IPv6/CIDR:</tt> and the text field of <tt>Gateway (IPv6):</tt>) ?!?
*:*: ?!? if you have network for IPv6 and you want to have access to your network for IPv6 you have fill out the text field of <tt>IPv6/CIDR:</tt> and the text field of <tt>Gateway (IPv6):</tt>) ?!?
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>DNS</tt>
*: tab <tt>DNS</tt>
*:* <s>(for the following we expect) (the text field of <tt>DNS domain:</tt>) is emtpy (and so prefilled  with ''use host settings'')</s>
*:* <s>(for the following we expect) (the text field of <tt>DNS domain:</tt>) is emtpy (and so prefilled  with <tt>use host settings</tt>)</s>
*:* <s>(for the following we expect) (the text field of <tt>DNS servers:</tt>) is emtpy (and so prefilled  with ''use host settings'')</s>
*:* <s>(for the following we expect) (the text field of <tt>DNS servers:</tt>) is emtpy (and so prefilled  with <tt>use host settings</tt>)</s>
*:* choose the ''Next'' button
*:* choose the <tt>Next</tt> button
*: tab <tt>Confirm</tt>
*: tab <tt>Confirm</tt>
*:: (optional) check the configuration (keys with values)
*:: (optional) check the configuration (keys with values)
*:* choose the ''Finish'' button
*:* choose the <tt>Finish</tt> button
<pre>
<pre>
/dev/rbd0
/dev/rbd0
Line 131: Line 131:
|  
|  
: <code>pct create </code>…
: <code>pct create </code>…
: <s><code>pct create 1000 --arch amd64 --description nixos-template cephfs:vztmpl/nixos-21.05_2021-10-10.tar.xz --ostype unmanaged --net0 name=eth0 --storage storage --unprivileged 1</code></s>
: <s><code>pct create 1000 cephfs:vztmpl/nixos-21.05_2021-10-10.tar.xz --ostype unmanaged --net0 name=eth0,firewall=1,ip=dhcp,bridge=vmbr0 --storage storage --unprivileged 1 --features nesting=1</code></s>
<pre></pre>
<pre></pre>
----
----
Line 138: Line 138:
<pre>
<pre>
arch: amd64
arch: amd64
features: nesting=1
hostname: CT1000
hostname: CT1000
memory: 1024
memory: 512
net0: name=eth0
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=1E:D8:FE:E9:F1:71,ip=dhcp,type=veth
ostype: unmanaged
ostype: unmanaged
rootfs: storage:vm-1000-disk-0,size=4G
rootfs: storage:vm-1000-disk-0,size=4G
Line 225: Line 226:
| start the NixOS container with Proxmox Virtual Environment
| start the NixOS container with Proxmox Virtual Environment
|  
|  
* …
* choose the <tt>Start</tt> button
|  
|  
: <code>pct start </code>…
: <code>pct start 1000</code>
: <s><code>pct start 1000</code></s>
<pre>
<pre></pre>
WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version.
Task finished with 1 warning(s)!
</pre>
: <code>pct status 1000</code>
<pre>
status: running
</pre>
|-
|-
| having a running NixOS container on Proxmox Virtual Environment
| having a running NixOS container on Proxmox Virtual Environment
Line 360: Line 369:
==== Is there a cgroup2 only NixOS containers? ====
==== Is there a cgroup2 only NixOS containers? ====


; Is there a configuration with cgroups versions 2 only - where cgroups version 1 is not needed - (so that is possible to rollback the manipulation of the pve lxc startup script (''/usr/share/perl5/PVE/LXC/Setup.pm''))? Or can we create such a (pre)build image (''tar'')?: I do not know.
; Is there a configuration with cgroups versions 2 only - where cgroups version 1 is not needed - (so that is possible to rollback the manipulation of the pve lxc startup script (<code>/usr/share/perl5/PVE/LXC/Setup.pm</code>))? Or can we create such a (pre)build image (''tar'')?: I do not know.


: <code>grep cgroup /proc/filesystems</code>
: <code>grep cgroup /proc/filesystems</code>

Revision as of 11:12, 11 October 2021

NixOS container (LXC) on Proxmox Virtual Environment 7

WUI CLI Reference
find the (a current) NixOS template

for example for 21.05

the link for the latest (daily) tarball for a container (of 21.05) is

https://hydra.nixos.org/job/nixos/release-21.05/nixos.containerTarball.x86_64-linux/latest/download-by-type/file/system-tarball
download the NixOS template
  • (if you want to switch to the Storage View and) go to one of your nodes where you can (find and) store CT Templates
  • choose the Download from URL button

going to the folder where all the templates for Promox Virtual Environment are stored

cd /mnt/pve/cephfs/template/cache

downloading the NixOS template file

wget -c https://hydra.nixos.org/job/nixos/release-21.05/nixos.containerTarball.x86_64-linux/latest/download-by-type/file/system-tarball

(optional you should) move the downloaded NixOS template file to a nicer (more individual) file name

mv system-tarball nixos-21.05_2021-10-10.tar.xz

(optional you can) going back

cd -
create a (first) new NixOS container with Proxmox Virtual Environment
  • (if you want to switch to the Folder View and go to Nodes and choose the your nodes where you create the container. (this note will be preselected as Node in the form.))
  • choose the Create CT button
    tab General
    • (for the following we expect) 1000 is prefilled (to the text field of CT ID:) or you have added it (because it not already taken)
    • (for the following we expect) the checkbox Unprivileged container: is preselected choosen
    • (for the following we expect) the checkbox Nesting: is preselected choosen
    • add a (useless, but from the form required) passphrase (to the text field of Password:)
    • add the same (useless, but from the form required) passphrase (to the text field of Confirm password:)
    • (optional you can) add other options of the form, like
      • the node for the container (at the drop down menu of Node:)
      • the name for the container (to the text field of Hostname:)
      • …
    • choose the Next button
    tab Template
    • (for the following we expect) the entry cephfs is prefilled (at the drop down menu of Storage:)
    • (for the following we expect you) find and choose the entry nixos-21.05_2021-10-10.tar.xz (at the drop down menu of Storage:)
    • choose the Next button
    tab Root Disk
    • (for the following we expect) the entry storage is prefilled (at the drop down menu of Storage:)
    • (for the following we expect) 8 is prefilled (to the text field of Disk size (GiB):)
    • choose the Next button
    tab CPU
    • (for the following we expect) 1 is prefilled (to the text field of Cores:)
    • choose the Next button
    tab Memory
    • (for the following we expect) 512 is prefilled (to the text field of Memory (MiB):)
    • (for the following we expect) 512 is prefilled (to the text field of Swap (MiB):)
    • choose the Next button
    tab Network
    • (for the following we expect) eth0 is prefilled (to the text field of Name:)
    • (for the following we expect) (the text field of MAC address:) is emtpy (and so prefilled with auto)
    • (for the following we expect) the entry vmbr0 is prefilled (at the drop down menu of Bridge:)
      we expect that you have a bridge vmbr0 configured
      otherwise?
    • (for the following we expect) (the text field of VLAN Tag:) is emtpy (and so prefilled with no VLAN)
    • (for the following we expect) (the text field of Rate limit (MB/s) Tag:) is emtpy (and so prefilled with unlimited)
    • (for the following we expect) the checkbox Firewall: is preselected choosen
    • (for the following we expect) choose DHCP (at the ratio button menu of IPv4:)
      ?!? otherwise the container will have no network access for IPv4 (or you have fill out the text field of IPv4/CIDR: and the text field of Gateway (IPv4):) ?!?
    • (for the following we expect) the entry Static is prefilled (at the ratio button menu of IPv6:) and you have no network access for IPv6 avilibale
      ?!? if you have network for IPv6 and you want to have access to your network for IPv6 you have fill out the text field of IPv6/CIDR: and the text field of Gateway (IPv6):) ?!?
    • choose the Next button
    tab DNS
    • (for the following we expect) (the text field of DNS domain:) is emtpy (and so prefilled with use host settings)
    • (for the following we expect) (the text field of DNS servers:) is emtpy (and so prefilled with use host settings)
    • choose the Next button
    tab Confirm
    (optional) check the configuration (keys with values)
    • choose the Finish button
/dev/rbd0
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 3f4cf224-8062-4cd3-918c-49f891af1aa1
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
extracting archive '/mnt/pve/cephfs/template/cache/nixos-21.05_2021-10-10.tar.xz'
Total bytes read: 836218880 (798MiB, 21MiB/s)
Architecture detection failed: open '/bin/sh' failed: No such file or directory

Falling back to amd64.
Use `pct set VMID --arch ARCH` to change.
/etc/os-release file not found and autodetection failed, falling back to 'unmanaged'
TASK OK

(optional you can) check the (pve) lxc config file (and it should look like something like the following) less /etc/pve/lxc/1000.conf 

arch: amd64
cores: 1
features: nesting=1
hostname: CT1000
memory: 512
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=FE:1E:11:E6:D2:8F,ip=dhcp,type=veth
ostype: unmanaged
rootfs: storage:vm-1000-disk-0,size=8G
swap: 512
unprivileged: 1
pct create …
pct create 1000 cephfs:vztmpl/nixos-21.05_2021-10-10.tar.xz --ostype unmanaged --net0 name=eth0,firewall=1,ip=dhcp,bridge=vmbr0 --storage storage --unprivileged 1 --features nesting=1




(optional you can) check the (pve) lxc config file (and it should look like something like the following)
less /etc/pve/lxc/1000.conf



arch: amd64
features: nesting=1
hostname: CT1000
memory: 512
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=1E:D8:FE:E9:F1:71,ip=dhcp,type=veth
ostype: unmanaged
rootfs: storage:vm-1000-disk-0,size=4G
swap: 512
unprivileged: 1






(configure the available (virtual) network device for the Proxmox Virtual Environment container)










customize the (pve) lxc startup setup routine (on every node where you want to start a NixOS container)





run_buffer: 316 Script exited with status 1
lxc_init: 816 Failed to run lxc.hook.pre-start for container "1000"
__lxc_start: 2007 Failed to initialize container "1000"
TASK ERROR: startup for container '1000' failed





nano /usr/share/perl5/PVE/LXC/Setup.pm



sub unified_cgroupv2_support {
    my ($self) = @_;




    return if !$self->{plugin}; # unmanaged



    $self->protected_call(sub { $self->{plugin}->unified_cgroupv2_support() });
}





(after fixing all the other stuff)



WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version.
TASK WARNINGS: 1





https://forum.proxmox.com/threads/92381/#post-402350






customize the individual Proxmox Virtual Environment container configuration for NixOS



sync_wait: 36 An error occurred in another process (expected sequence number 7)
TASK ERROR: startup for container '1000' failed





sync_wait: 36 An error occurred in another process (expected sequence number 7)
__lxc_start: 2073 Failed to spawn container "1000"
TASK ERROR: startup for container '1000' failed





editing the specific (pve) lxc config file (to a option for lxc.init.cmd)



nano /etc/pve/lxc/1000.conf




lxc.init.cmd: /init






(activate nesting for NixOS in the Proxmox Virtual Environment container)







(optional you can) check the (pve) lxc config file (and it should look like something like the following with nesting=1 on the line features:)
less /etc/pve/lxc/1000.conf



arch: amd64
cores: 2
features: nesting=1
hostname: CT1000
memory: 1024
net0: name=eth0,bridge=vmbr0,hwaddr=A1:B2:C3:D4:E5:F6,ip=dhcp,type=veth
ostype: unmanaged
rootfs: storage:vm-1000-disk-0,size=4G
swap: 512
unprivileged: 1






start the NixOS container with Proxmox Virtual Environment



  • …
    • 

  • choose the Start button




pct start 1000


WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version.
Task finished with 1 warning(s)!



pct status 1000


status: running






having a running NixOS container on Proxmox Virtual Environment








login into the NixOS container



  • … (node(Folder View) LXC Container -> container)
    • 

  • Console


<<< Welcome to NixOS 21.05.3740.ce7a1190a0f (x86_64) - pts/0 >>>

Log in as "root" with an empty password.


nixos login: 



nixos login: root




[root@nixos:~]# 





lxc-attach 1000


sh-4.4# 



sh-4.4# . /etc/profile 




[root@nixos:/]# 






update the channel for NixOS (needed!)





nix-channel --update


unpacking channels...
created 1 symlinks in user environment






(optional) test rebuild of NixOS





(optional) check the default configuration file for NixOS



cat /etc/nixos/configuration.nix


{ config, pkgs, ... }:

{
  imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];

  
}



nixos-rebuild test


building Nix...
building the system configuration...
activating the configuration...
setting up /etc...
setting up tmpfiles
warning: the following units failed: sys-kernel-debug.mount

● sys-kernel-debug.mount - Kernel Debug File System
     Loaded: loaded (/nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/example/systemd/system/sys-kernel-debug.mount; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2021-10-11 06:48:30 UTC; 265ms ago
      Where: /sys/kernel/debug
       What: debugfs
       Docs: https://www.kernel.org/doc/Documentation/filesystems/debugfs.txt
             https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
         IP: 0B in, 0B out
        CPU: 2ms

Oct 11 06:48:30 nixos systemd[1]: Mounting Kernel Debug File System...
Oct 11 06:48:30 nixos mount[17997]: mount: /sys/kernel/debug: permission denied.
Oct 11 06:48:30 nixos systemd[1]: sys-kernel-debug.mount: Mount process exited, code=exited, status=32/n/a
Oct 11 06:48:30 nixos systemd[1]: sys-kernel-debug.mount: Failed with result 'exit-code'.
Oct 11 06:48:30 nixos systemd[1]: Failed to mount Kernel Debug File System.
warning: error(s) occurred while switching to the new configuration






fixing the configuration file for NixOS





(optional) check the default configuration file for NixOS



cat /etc/nixos/configuration.nix


{ config, pkgs, ... }:

{
  imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];

  
}



nano /etc/nixos/configuration.nix


{

  imports = [ <nixpkgs/nixos/modules/virtualisation/lxc-container.nix> ];





  systemd.suppressedSystemUnits = [
    "sys-kernel-debug.mount"
  ];





}



https://github.com/NixOS/nixpkgs/issues/9735#issuecomment-783535726




…









Is there a need for non unprivileged NixOS containers?


Is there a scenario where you would like to have a privileged (unprivileged = 0) container?


I do not know.


Is there a cgroup2 only NixOS containers?


Is there a configuration with cgroups versions 2 only - where cgroups version 1 is not needed - (so that is possible to rollback the manipulation of the pve lxc startup script (/usr/share/perl5/PVE/LXC/Setup.pm))? Or can we create such a (pre)build image (tar)?


I do not know.


grep cgroup /proc/filesystems


nodev	cgroup
nodev	cgroup2



Notes




https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html

(pve) lxc configuration option lxc.cgroup2.devices.allow: a

(pve) lxc configuration option lxc.cgroup.devices.deny = a

https://wiki.debian.org/LXC/CGroupV2

https://search.nixos.org/options?channel=unstable&from=0&query=cgroup

https://git.proxmox.com/?p=pve-container.git;a=blob;f=src/PVE/LXC/Setup/Base.pm;h=a5b77d32f82747ea558d0398919414945b133dc0;hb=HEAD#l523



documentation


proxmox




lxc



  • …


debian



  • …


linux



  • …


Retrieved from "https://wiki.nixos.org/w/index.php?title=Talk:Proxmox_Linux_Container&oldid=9378"