Proxmox Linux Container: Difference between revisions

From NixOS Wiki
imported>Samueldr
Inits page with rough~ish notes
 
Nani8ot (talk | contribs)
Add note to only run /run/current-system/activate when no commands can be found, e.g. ls.
 
(14 intermediate revisions by 10 users not shown)
Line 1: Line 1:
{{outdated|See [[Proxmox]] for instructions that apply to more recent versions of nixpkgs and proxmox.}}
= Building Proxmox Linux Containers with Nix =
You can use 'make-system-tarball' to create a Linux Container that can run on Proxmox, for example like https://codeberg.org/raboof/nix-mastodon-bot/src/branch/main/default.nix#L31
= NixOS =
With a small amount of work, it is possible to use NixOS as a LXC container under Proxmox.
With a small amount of work, it is possible to use NixOS as a LXC container under Proxmox.
== Generate a container tarball ==
A better way to deploy a NixOS container tarball on Proxmox is described at [[Proxmox Virtual Environment#LXC]].


== Finding a container tarball ==
== Finding a container tarball ==


First, choose a channel in https://nixos.org/channels/. You will then need to either follow the <em>via Hydra evaluation XXXX</em> link, or go to the URL listed in the <tt>src-url</tt> file.
Go to https://hydra.nixos.org/project/nixos, choose a release (small ones don't have the tarball we need), and open its Jobs tab.
 
On that page, search for <strong>nixos.containerTarball</strong>, open the link corresponding to your architecture (probably x86_64). Choose the latest success and click on the number. That is the ID of the result. Under <em>Build products</em>, download the <strong>system-tarball</strong>, which will be named <tt>nixos-system-x86_64-linux.tar.xz</tt>.
On that page, use the search box to search for <strong>containerTarball.x86_64</strong>. In the results page, follow the link that is the ID of the result. Under <em>Build products</em>, download the <strong>system-tarball</strong>, which will be named <tt>nixos-system-x86_64-linux.tar.xz</tt>.


Note the Build ID, it will be used when renaming the tarball... right when it's downloaded:
Note the Build ID, it will be used when renaming the tarball... right when it's downloaded:


```
<pre>
mv nixos-system-x86_64-linux.tar.xz nixos-18.03-default_$BUILDID_amd64.tar.xz
mv nixos-system-x86_64-linux.tar.xz nixos-${RELEASE}-default_${BUILDID}_amd64.tar.xz
```
</pre>


This is a proxmox convention, and is useful to follow.
This is a proxmox convention, and is useful to follow.
Line 23: Line 33:
=== CLI operations ===
=== CLI operations ===


In the CLI, create a container which will be used as a template. I recommend setting a memorable ID.
In the CLI, create a container using the following command. If created as a template, I recommend setting a memorable ID.


<pre>
<pre>
pct create 99999 \
pct create 99999 \
   --description nixos-template \
   --description nixos-template \
   local:vztmpl/nixos-18.03-default_$BUILDID_amd64.tar.xz \
   local:vztmpl/nixos-${RELEASE}-default_${BUILDID}_amd64.tar.xz \
   -ostype unmanaged \
   -ostype unmanaged \
   -net0 name=eth0 \
   -net0 name=eth0 \
Line 39: Line 49:


The last step using the CLI, edit <tt>/etc/pve/lxc/99999.conf</tt> to add this line:
The last step using the CLI, edit <tt>/etc/pve/lxc/99999.conf</tt> to add this line:
<pre>
lxc.init.cmd: /sbin/init
</pre>


Before NixOS version 21.11 it was
<pre>
<pre>
lxc.init.cmd: /init
lxc.init.cmd: /init
</pre>
</pre>
.


=== Creating a template ===
=== Creating a template ===
Line 49: Line 64:


To create a new NixOS container, you can then clone the template instead of doing the preceding steps.
To create a new NixOS container, you can then clone the template instead of doing the preceding steps.
== Expected issues ==
=== Entering the container by <code>pct enter</code> ===


== Expected issues ==
enter a (nixos) container from the proxmox host
: <syntaxhighlight lang="shell">[root@proxmox:~]# pct enter <vmid></syntaxhighlight>
: <syntaxhighlight lang="shell">sh-5.2#</syntaxhighlight>
 
: <s>set (missing) environment variables</s>
:: <s><syntaxhighlight lang="shell">sh-5.2# source /etc/set-environment</syntaxhighlight></s>
:: <s><syntaxhighlight lang="shell">sh-5.2#</syntaxhighlight></s>
or
: set (missing) environment variables with <code>[[wikipedia:en:dot (command)#source|.]]</code>
:: <syntaxhighlight lang="shell">sh-5.2# . /etc/profile</syntaxhighlight>
:: <syntaxhighlight lang="bash">[root@nixos:~]#</syntaxhighlight>
If commands (like ls) still can't be found, try running:
:: <syntaxhighlight lang="shell">
[root@nixos:~] /run/current-system/activate
</syntaxhighlight>
<!--
or
: set (missing) environment variables by opening a new shell
:: <syntaxhighlight lang="shell">sh-5.2# sh -l</syntaxhighlight>
:: <syntaxhighlight lang="bash">[root@nixos:~]#</syntaxhighlight>
-->


=== nixos-rebuild switch ===
<code>nixos-rebuild switch</code> may fail with errors for special mount points. This does not look like it affects the container.
<code>nixos-rebuild switch</code> may fail with errors for special mount points. This does not look like it affects the container.


Line 70: Line 108:
warning: error(s) occurred while switching to the new configuration
warning: error(s) occurred while switching to the new configuration
</pre>
</pre>
=== Black Console in Proxmox ===
The Proxmox console may appear black when launched. It is at the login prompt but no existing text is rendered. Just type "root" and hit enter and new text will get rendered just fine. If you are still unable to get the console to render, you can try changing the console to `/dev/console` instead of the standard tty.

Latest revision as of 13:48, 6 September 2024

Building Proxmox Linux Containers with Nix

You can use 'make-system-tarball' to create a Linux Container that can run on Proxmox, for example like https://codeberg.org/raboof/nix-mastodon-bot/src/branch/main/default.nix#L31

NixOS

With a small amount of work, it is possible to use NixOS as a LXC container under Proxmox.

Generate a container tarball

A better way to deploy a NixOS container tarball on Proxmox is described at Proxmox Virtual Environment#LXC.

Finding a container tarball

Go to https://hydra.nixos.org/project/nixos, choose a release (small ones don't have the tarball we need), and open its Jobs tab. On that page, search for nixos.containerTarball, open the link corresponding to your architecture (probably x86_64). Choose the latest success and click on the number. That is the ID of the result. Under Build products, download the system-tarball, which will be named nixos-system-x86_64-linux.tar.xz.

Note the Build ID, it will be used when renaming the tarball... right when it's downloaded:

mv nixos-system-x86_64-linux.tar.xz nixos-${RELEASE}-default_${BUILDID}_amd64.tar.xz

This is a proxmox convention, and is useful to follow.

Preparing the container

But first, you will need to upload the container tarball to the storage.

Then, once this is done, you will need to access the CLI for the proxmox host. SSH or the web CLI will be fine.

CLI operations

In the CLI, create a container using the following command. If created as a template, I recommend setting a memorable ID.

pct create 99999 \
  --description nixos-template \
  local:vztmpl/nixos-${RELEASE}-default_${BUILDID}_amd64.tar.xz \
  -ostype unmanaged \
  -net0 name=eth0 \
  -storage local-lvm

Note that in the previous command, the -ostype option is necessary; this way Proxmox will not try to do fancy stuff to the system.

The last step using the CLI, edit /etc/pve/lxc/99999.conf to add this line:

lxc.init.cmd: /sbin/init

Before NixOS version 21.11 it was

lxc.init.cmd: /init

.

Creating a template

It is not be possible to use the facilities to create a NixOS container under Proxmox. It is possible, though, to create a template that may be cloned as a base. In the GUI, with the container selected, click the More menu at the top, then Convert to template. Accept.

To create a new NixOS container, you can then clone the template instead of doing the preceding steps.

Expected issues

Entering the container by pct enter

enter a (nixos) container from the proxmox host

[root@proxmox:~]# pct enter <vmid>
sh-5.2#
set (missing) environment variables
sh-5.2# source /etc/set-environment
sh-5.2#

or

set (missing) environment variables with .
sh-5.2# . /etc/profile
[root@nixos:~]#

If commands (like ls) still can't be found, try running:

[root@nixos:~] /run/current-system/activate

nixos-rebuild switch

nixos-rebuild switch may fail with errors for special mount points. This does not look like it affects the container.

[root@nixos:/]# nixos-rebuild switch
building Nix...
building the system configuration...
activating the configuration...
setting up /etc...
mount: /dev: cannot remount devtmpfs read-write, is write-protected.
mount: /dev/pts: cannot remount devpts read-write, is write-protected.
mount: /dev/shm: cannot remount tmpfs read-write, is write-protected.
mount: /proc: cannot remount proc read-write, is write-protected.
mount: /run: cannot remount tmpfs read-write, is write-protected.
mount: /run/keys: cannot mount ramfs read-only.
mount: /run/wrappers: cannot remount tmpfs read-write, is write-protected.
setting up tmpfiles
warning: error(s) occurred while switching to the new configuration

Black Console in Proxmox

The Proxmox console may appear black when launched. It is at the login prompt but no existing text is rendered. Just type "root" and hit enter and new text will get rendered just fine. If you are still unable to get the console to render, you can try changing the console to `/dev/console` instead of the standard tty.