Proxmox Virtual Environment: Difference between revisions
imported>Vater m Vater moved page Proxmox to Proxmox Virtual Environment: proxmox has several products |
imported>Vater mNo edit summary |
||
| Line 1: | Line 1: | ||
<blockquote> | <blockquote> | ||
The instructions should work for PVE 7.2 and later with NixOS 22.05 and later. Users of previous versions may need to patch pve-container to use NixOS LXC images (instructions are below too). | |||
</blockquote> | </blockquote> | ||
It is possible to generate generic qcow2 images and attach them to VMs with <code>qm importdisk</code> as shown [https://pve.proxmox.com/wiki/Migration_of_servers_to_Proxmox_VE#Importing_to_Proxmox_VE here] | == KVM == | ||
It is possible to generate generic qcow2 images and attach them to VMs with <code>qm importdisk</code> as shown [https://pve.proxmox.com/wiki/Migration_of_servers_to_Proxmox_VE#Importing_to_Proxmox_VE here]. | |||
A better option is to generate a VMA image that can be imported as a VM on proxmox. With this method, many VM configuration options such as CPU, memory, netowrk interfaces, and serial terminals can be specified in nix instead of manually setting them on the proxmox UI. | A better option is to generate a VMA image that can be imported as a VM on proxmox. With this method, many VM configuration options such as CPU, memory, netowrk interfaces, and serial terminals can be specified in nix instead of manually setting them on the proxmox UI. | ||
== Generating VMA == | === Generating VMA === | ||
<blockquote>The first run will take some time, as a patched version of qemu with support for the VMA format needs to be built | <blockquote> | ||
The first run will take some time, as a patched version of qemu with support for the VMA format needs to be built | |||
</blockquote> | </blockquote> | ||
<pre>nix run github:nix-community/nixos-generators -- --format proxmox</pre> | <pre> | ||
nix run github:nix-community/nixos-generators -- --format proxmox | |||
</pre> | |||
Pass additional nix configuration to the template with <code>--configuration filename.nix</code>. In addition to NixOS module options, proxmox-specific options present in [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/proxmox-image.nix nixos/modules/virtualisation/proxmox-image.nix] can be used to set core, memory, disk and other VM hardware options. | Pass additional nix configuration to the template with <code>--configuration filename.nix</code>. In addition to NixOS module options, proxmox-specific options present in [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/proxmox-image.nix nixos/modules/virtualisation/proxmox-image.nix] can be used to set core, memory, disk and other VM hardware options. | ||
== Deploying on proxmox == | === Deploying on proxmox === | ||
The generated vma.zst file can be copied to <code>/var/lib/vz/dump/</code> (or any other configured VM dump storage path). A new VM can be spun up from it either using the GUI or the CLI: | The generated vma.zst file can be copied to <code>/var/lib/vz/dump/</code> (or any other configured VM dump storage path). A new VM can be spun up from it either using the GUI or the CLI: | ||
<pre>qmrestore /var/lib/vz/dump/vzdump-qemu-nixos-21.11.git.d41882c7b98M.vma.zst <vmid> --unique true</pre> | <pre> | ||
<blockquote>note: the MAC | qmrestore /var/lib/vz/dump/vzdump-qemu-nixos-21.11.git.d41882c7b98M.vma.zst <vmid> --unique true | ||
</pre> | |||
<blockquote> | |||
note: the MAC address of net0 defaults to <code>00:00:00:00:00:00</code>. This must either be overriden thruogh <code>proxmox.qemuConf.net0</code>, or the <code>unique</code> attribute must be set to true when importing the image on proxmox. | |||
</blockquote> | </blockquote> | ||
By default, the generated image is set up to expose a serial terminal interface for ease of access. | By default, the generated image is set up to expose a serial terminal interface for ease of access. | ||
<pre>root@proxmox-server:~# qm start <vmid> | <pre> | ||
root@proxmox-server:~# qm start <vmid> | |||
root@proxmox-server:~# qm terminal <vmid> | root@proxmox-server:~# qm terminal <vmid> | ||
starting serial terminal on interface serial0 (press Ctrl+O to exit) | starting serial terminal on interface serial0 (press Ctrl+O to exit) | ||
| Line 45: | Line 54: | ||
[root@nixos:~]#</pre> | [root@nixos:~]# | ||
== Network configuration == | </pre> | ||
=== Network configuration === | |||
Cloud-init can be enabled with | Cloud-init can be enabled with | ||
<pre>services.cloud-init.network.enable = true;</pre> | <pre> | ||
services.cloud-init.network.enable = true; | |||
</pre> | |||
This will enable systemd-networkd, allowing cloud-init to set up network interfaces on boot. | This will enable systemd-networkd, allowing cloud-init to set up network interfaces on boot. | ||
= LXC = | == LXC == | ||
=== Generating LXC template === | |||
<pre> | |||
nix run github:nix-community/nixos-generators -- --format proxmox-lxc | |||
</pre> | |||
=== Privileged LXCs === | |||
== Privileged LXCs == | |||
While it’s not necessary, <code>proxmoxLXC.privileged</code> can be set to true to enable the DebugFS mount in privileged LXCs. If enabled on unprivileged LXCs, this will fail to mount. | While it’s not necessary, <code>proxmoxLXC.privileged</code> can be set to true to enable the DebugFS mount in privileged LXCs. If enabled on unprivileged LXCs, this will fail to mount. | ||
== Network configuration == | === Network configuration === | ||
The proxmox LXC template uses systemd-networkd by default to allow network configuration by proxmox. <code>proxmoxLXC.manageNetwork</code> can be set to true to disable this. | The proxmox LXC template uses systemd-networkd by default to allow network configuration by proxmox. <code>proxmoxLXC.manageNetwork</code> can be set to true to disable this. | ||
== deploying on proxmox == | === deploying on proxmox === | ||
Copy the tarball to proxmox, then create a new LXC with this template through the web UI or the CLI. The “nesting” feature needs to be enabled. Newer versions of proxmox will have it enabled by default. | Copy the tarball to proxmox, then create a new LXC with this template through the web UI or the CLI. The “nesting” feature needs to be enabled. Newer versions of proxmox will have it enabled by default. | ||
| Line 73: | Line 89: | ||
The template built above without any options does not come with <code>/etc/nixos/configuration.nix</code>. A minimal working example is presented below. Be sure to run <code>nix-channel --update</code> before <code>nixos-rebuild switch</code>. | The template built above without any options does not come with <code>/etc/nixos/configuration.nix</code>. A minimal working example is presented below. Be sure to run <code>nix-channel --update</code> before <code>nixos-rebuild switch</code>. | ||
<pre>{ pkgs, modulesPath, ... }: | <pre> | ||
{ pkgs, modulesPath, ... }: | |||
{ | { | ||
| Line 83: | Line 100: | ||
pkgs.vim | pkgs.vim | ||
]; | ]; | ||
}</pre> | } | ||
</pre> | |||
== Patching pve-container == | === Patching pve-container === | ||
<blockquote>This is not needed on pve-container version 4.1-5 (released on 27 April 2022) and newer. If you have an older version, you will have to patch it to add NixOS LXC support. If you have a newer version, skip ahead to the next section. | <blockquote> | ||
This is not needed on pve-container version 4.1-5 (released on 27 April 2022) and newer. If you have an older version, you will have to patch it to add NixOS LXC support. If you have a newer version, skip ahead to the next section. | |||
</blockquote> | </blockquote> | ||
* install some dependencies <code>apt install git devscripts gdebi</code> | * install some dependencies <code>apt install git devscripts gdebi</code> | ||
* clone https://github.com/proxmox/pve-container | * clone https://github.com/proxmox/pve-container | ||
<pre>root@pve:~# git clone https://github.com/proxmox/pve-container | <pre> | ||
root@pve:~# git clone https://github.com/proxmox/pve-container | |||
... | ... | ||
root@pve:~# cd pve-container/</pre> | root@pve:~# cd pve-container/ | ||
</pre> | |||
* get the installed version of pve-container | * get the installed version of pve-container | ||
<pre>root@pve:~/pve-container# pveversion -v | grep pve-container | <pre> | ||
pve-container: 4.1-2</pre> | root@pve:~/pve-container# pveversion -v | grep pve-container | ||
pve-container: 4.1-2 | |||
</pre> | |||
* check out the commit of the version you want | * check out the commit of the version you want | ||
<pre>root@pve:~/pve-container# git log --grep "4.1-2" | <pre> | ||
root@pve:~/pve-container# git log --grep "4.1-2" | |||
commit 5d5f81f645bd1e8fd0ffff878fe249253e1be777 | commit 5d5f81f645bd1e8fd0ffff878fe249253e1be777 | ||
Author: Thomas Lamprecht <t.lamprecht@proxmox.com> | Author: Thomas Lamprecht <t.lamprecht@proxmox.com> | ||
| Line 109: | Line 132: | ||
bump version to 4.1-2 | bump version to 4.1-2 | ||
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com></pre> | Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com> | ||
<pre>root@pve:~/pve-container# git checkout 5d5f81f645bd1e8fd0ffff878fe249253e1be777 | </pre> | ||
<pre> | |||
root@pve:~/pve-container# git checkout 5d5f81f645bd1e8fd0ffff878fe249253e1be777 | |||
Note: switching to '5d5f81f645bd1e8fd0ffff878fe249253e1be777'. | Note: switching to '5d5f81f645bd1e8fd0ffff878fe249253e1be777'. | ||
... | ... | ||
HEAD is now at 5d5f81f bump version to 4.1-2</pre> | HEAD is now at 5d5f81f bump version to 4.1-2 | ||
</pre> | |||
* cherry-pick the [https://github.com/proxmox/pve-container/commit/6226d0101652914744cb5c657414bf286ccd857d patch that adds NixOS LXC support] | * cherry-pick the [https://github.com/proxmox/pve-container/commit/6226d0101652914744cb5c657414bf286ccd857d patch that adds NixOS LXC support] | ||
<pre>root@pve:~/pve-container# git cherry-pick 6226d0101652914744cb5c657414bf286ccd857d | <pre> | ||
root@pve:~/pve-container# git cherry-pick 6226d0101652914744cb5c657414bf286ccd857d | |||
Auto-merging src/PVE/LXC/Config.pm | Auto-merging src/PVE/LXC/Config.pm | ||
[detached HEAD 6f3cd03] Setup: add NixOS support | [detached HEAD 6f3cd03] Setup: add NixOS support | ||
| Line 129: | Line 156: | ||
* Install build deps | * Install build deps | ||
<pre>root@pve:~/pve-container# mk-build-deps | <pre> | ||
root@pve:~/pve-container# mk-build-deps | |||
... | ... | ||
The package has been created. | The package has been created. | ||
Attention, the package has been created in the current directory, | Attention, the package has been created in the current directory, | ||
not in ".." as indicated by the message above!</pre> | not in ".." as indicated by the message above! | ||
<pre>root@pve:~/pve-container# gdebi pve-container-build-deps_4.1-2_all.deb | </pre> | ||
<pre> | |||
root@pve:~/pve-container# gdebi pve-container-build-deps_4.1-2_all.deb | |||
Reading package lists... Done | Reading package lists... Done | ||
... | ... | ||
| Line 140: | Line 170: | ||
... | ... | ||
Unpacking pve-container-build-deps (4.1-2) ... | Unpacking pve-container-build-deps (4.1-2) ... | ||
Setting up pve-container-build-deps (4.1-2) ...</pre> | Setting up pve-container-build-deps (4.1-2) ... | ||
</pre> | |||
* build the patched pve-container | * build the patched pve-container | ||
<pre>root@pve:~/pve-container# make | <pre> | ||
root@pve:~/pve-container# make | |||
... | ... | ||
dpkg-buildpackage: info: binary-only upload (no source included) | dpkg-buildpackage: info: binary-only upload (no source included) | ||
lintian pve-container_4.1-2_all.deb | lintian pve-container_4.1-2_all.deb | ||
warning: running with root privileges is not recommended!</pre> | warning: running with root privileges is not recommended! | ||
</pre> | |||
* install the deb | * install the deb | ||
<pre>root@pve:~/pve-container# dpkg -i pve-container_4.1-2_all.deb</pre> | <pre> | ||
root@pve:~/pve-container# dpkg -i pve-container_4.1-2_all.deb | |||
</pre> | |||
* verify that the installed pve-container package added NixOS support | * verify that the installed pve-container package added NixOS support | ||
<pre>root@pve:~# ls /usr/share/perl5/PVE/LXC/Setup/NixOS.pm | <pre> | ||
/usr/share/perl5/PVE/LXC/Setup/NixOS.pm</pre> | root@pve:~# ls /usr/share/perl5/PVE/LXC/Setup/NixOS.pm | ||
/usr/share/perl5/PVE/LXC/Setup/NixOS.pm | |||
</pre> | |||