|
|
| (5 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| This is a userspace draft and is not supported by NixOS Wiki.
| | Moved to here: |
| | https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/Root%20on%20ZFS.html |
|
| |
|
| == Enable ZFS on Existing Installation ==
| | Last version is: https://wiki.nixos.org/w/index.php?title=User:2r/NixOS_on_ZFS&oldid=5406 |
| Add the following lines to configuration:
| |
| <pre>boot.supportedFilesystems = [ "zfs" ];
| |
| networking.hostId = "deadbeef";</pre>
| |
| Host ID should be unique, generate one with <code>head -c 8 /etc/machine-id</code>.
| |
| | |
| Rebuild system with <code>nixos-rebuild switch</code>.
| |
| | |
| == Install NixOS on ZFS ==
| |
| === Layout ===
| |
| ==== Partitions ====
| |
| As swap on ZFS will cause deadlock and does not support hibernation, a separate swap partition should be created.
| |
| | |
| {| class="wikitable"
| |
| |+ GPT partition table
| |
| |-
| |
| |
| |
| | ESP
| |
| | bpool
| |
| | rpool
| |
| | swap
| |
| | BIOS boot sector
| |
| |-
| |
| |width=5% valign=top| Filesystem
| |
| |width=20% valign=top| vfat
| |
| |width=20% valign=top| ZFS, feature limited for GRUB compatibility.
| |
| |width=20% valign=top| ZFS
| |
| |width=20% valign=top| swap
| |
| |width=5% valign=top| N/A
| |
| |-
| |
| |width=5% valign=top| Content
| |
| |width=20% valign=top| <code>grubx64.efi</code>
| |
| |width=20% valign=top| <code>/boot</code>
| |
| |width=20% valign=top| <code>/</code>
| |
| |width=20% valign=top| swap
| |
| |width=5% valign=top| N/A
| |
| |-
| |
| |width=5% valign=top| Encryption
| |
| |width=20% valign=top| No, can be validated with Secure Boot
| |
| |width=20% valign=top| LUKS1
| |
| |width=20% valign=top| ZFS Encrytion
| |
| |width=20% valign=top| random/LUKS2
| |
| |width=5% valign=top| N/A
| |
| |}
| |
| ==== Datasets ====
| |
| As NixOS lacks a service to handle native ZFS mounting at boot, such as <code>zfs-mount-generator</code>, all mountable datasets must be created with <code>mountpoint=legacy</code> to be mounted with <code>fileSystems</code> option.
| |
| | |
| Datasets with <code>canmount=off mountpoint=none</code> are used as containers, that is, no data is stored directly under such datasets, but child datasets can inherit their properties or imitate directory structures, such as <code>/var/log</code>.
| |
| | |
| {| class="wikitable"
| |
| |+ Dataset layout
| |
| |-
| |
| |colspan="3"| Containers
| |
| |
| |
| | mountpoint
| |
| | canmount
| |
| | comment
| |
| |-
| |
| | bpool
| |
| | sys
| |
| | BOOT
| |
| | default
| |
| | /boot
| |
| | noauto
| |
| |-
| |
| |rowspan="3"| rpool
| |
| |rowspan="3"| sys
| |
| | ROOT
| |
| | default
| |
| | /
| |
| | noauto
| |
| |-
| |
| |rowspan="2"| DATA
| |
| | local
| |
| | /
| |
| | off
| |
| | container for datasets that do not need backup, such as /nix
| |
| |-
| |
| | safe
| |
| | /
| |
| | off
| |
| | container for datasets that need backup, such as /{root,home,home/user}
| |
| |}
| |
| ==== Encryption ====
| |
| Boot pool can be encrypted with LUKS1 to prevent initrd tempering, however ZFS on LUKS is discouraged on root pool as LUKS abstracts physical devices and thus not desirable. Also, data needs to be encrypted per disk, thus slower than per file, as with ZFS native encryption.
| |
| | |
| ZFS native encryption does not encrypt dataset paths and default properties. Custom properties containing colon <code>custom:property</code> is encrypted.
| |
| | |
| Also, as ZFS currently does not support replacing master key, once the passphrase/keyfile is compromised, the encrypted dataset must be destroyed to protect confidentiality. Therefore, users are advised to choose a strong password at the beginning.
| |
| | |
| === Preparations ===
| |
| Download NixOS 64 Bit Minimal ISO image and SHA-256 checksum. Verify checksum and write the image to an external disk. Boot the computer from the disk afterwards.
| |
| | |
| After booting the computer, optionally configure SSH server and connect from another computer via SSH.
| |
| | |
| === Identify target disks ===
| |
| List available disks with
| |
| <pre>ls -d /dev/disk/by-id/* | grep -v part</pre>
| |
| | |
| === Partition ===
| |
| <pre>for i in {/dev/disk/by-id/disk1,/dev/disk/by-id/disk2}; do
| |
| sgdisk --zap-all $i
| |
| sgdisk -n1:1M:+1G -t1:EF00 $i
| |
| sgdisk -n2:0:+4G -t2:BE00 $i
| |
| sgdisk -n3:0:0 -t3:BF00 $i
| |
| | |
| # with swap
| |
| # sgdisk -n3:0:-8G -t3:BF00 $i
| |
| # sgdisk -n4:0:0 -t4:8308 $i
| |
| | |
| # with BIOS
| |
| # sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
| |
| done</pre>
| |
| | |
| === Create pools ===
| |
| ==== Multi-disk ====
| |
| Change
| |
| <pre>zpool create \
| |
| ... \
| |
| /dev/disk/by-id/disk1-partX</pre> | |
| to
| |
| <pre>zpool create \
| |
| ... \
| |
| mirror \ # or raidz1 (discouraged), raidz2, raidz3
| |
| /dev/disk/by-id/disk1-partX \
| |
| /dev/disk/by-id/disk2-partX</pre>
| |
| ==== Boot pool ====
| |
| <pre>zpool create \
| |
| -o ashift=12 \
| |
| -o autotrim=on \
| |
| -d -o feature@async_destroy=enabled \
| |
| -o feature@bookmarks=enabled \
| |
| -o feature@embedded_data=enabled \
| |
| -o feature@empty_bpobj=enabled \
| |
| -o feature@enabled_txg=enabled \
| |
| -o feature@extensible_dataset=enabled \
| |
| -o feature@filesystem_limits=enabled \
| |
| -o feature@hole_birth=enabled \
| |
| -o feature@large_blocks=enabled \
| |
| -o feature@lz4_compress=enabled \
| |
| -o feature@spacemap_histogram=enabled \
| |
| -O acltype=posixacl \
| |
| -O canmount=off \
| |
| -O compression=lz4 \
| |
| -O devices=off \
| |
| -O normalization=formD \
| |
| -O relatime=on \
| |
| -O xattr=sa \
| |
| -O mountpoint=none \
| |
| -R /mnt \
| |
| bpool \
| |
| /dev/disk/by-id/disk1-part2</pre>
| |
| ==== Root pool ====
| |
| <pre>zpool create \
| |
| -o ashift=12 \
| |
| -o autotrim=on \
| |
| -R /mnt \
| |
| -O acltype=posixacl \
| |
| -O canmount=off \
| |
| -O compression=zstd \
| |
| -O dnodesize=auto \
| |
| -O normalization=formD \
| |
| -O relatime=on \
| |
| -O xattr=sa \
| |
| -O mountpoint=none \
| |
| rpool \
| |
| /dev/disk/by-id/disk1-part3</pre>
| |
| === Create datasets ===
| |
| ==== Boot container ====
| |
| <pre>zfs create \
| |
| -o canmount=off \
| |
| -o mountpoint=none \
| |
| bpool/sys</pre>
| |
| ==== Root container ====
| |
| Encrypted
| |
| <pre>zfs create \
| |
| -o canmount=off \
| |
| -o mountpoint=none \
| |
| -o encryption=on \
| |
| -o keylocation=prompt \
| |
| -o keyformat=passphrase \
| |
| rpool/sys</pre>
| |
| Unencrypted
| |
| <pre>zfs create \
| |
| -o canmount=off \
| |
| -o mountpoint=none \
| |
| rpool/sys</pre>
| |
| ==== System datasets ====
| |
| <pre>zfs create -o canmount=off -o mountpoint=none bpool/NixOS/BOOT
| |
| zfs create -o canmount=off -o mountpoint=none rpool/NixOS/ROOT
| |
| zfs create -o canmount=off -o mountpoint=none rpool/NixOS/DATA
| |
| zfs create -o canmount=off -o mountpoint=/ rpool/NixOS/DATA/local
| |
| zfs create -o canmount=off -o mountpoint=/ rpool/NixOS/DATA/safe
| |
| zfs create -o mountpoint=legacy -o canmount=noauto bpool/NixOS/BOOT/default
| |
| zfs create -o mountpoint=legacy -o canmount=noauto rpool/NixOS/ROOT/default
| |
| mount -t zfs rpool/NixOS/ROOT/default /mnt
| |
| mkdir /mnt/boot
| |
| mount -t zfs bpool/NixOS/BOOT/default /mnt/boot
| |
| for i in {nix,}; do
| |
| zfs create -o canmount=on -o mountpoint=legacy rpool/NixOS/DATA/local/$i
| |
| mkdir -p /mnt/$i
| |
| mount -t zfs rpool/NixOS/DATA/local/$i /mnt/$i
| |
| done
| |
| for i in {root,home,home/user}; do
| |
| zfs create -o canmount=on -o mountpoint=legacy rpool/NixOS/DATA/safe/$i
| |
| mkdir -p /mnt/$i
| |
| mount -t zfs rpool/NixOS/DATA/safe/$i /mnt/$i
| |
| done
| |
| chmod 750 /mnt/root
| |
| chmod 700 /mnt/home/user</pre>
| |
| === System Installation ===
| |
| Except a few additional options, the procedure is identical with normal installation.
| |
| | |
| ==== General ====
| |
| Generate Host ID<pre>head -c 8 /etc/machine-id</pre>
| |
| | |
| Generate and edit config file<pre>nixos-generate-config --root /mnt
| |
| vim /mnt/etc/nixos/configuration.nix</pre>
| |
| Add ZFS:
| |
| <pre>boot.supportedFilesystems = [ "zfs" ];
| |
| networking.hostId = "deadbeef";</pre>
| |
| | |
| If not using <code>/dev/disk/by-id</code>:<pre>boot.zfs.devNodes = "/dev/disk/by-path";</pre>
| |
| | |
| If swap partition:<pre>swapDevices = [
| |
| {
| |
| device = "/dev/disk/by-id/disk1-part4";
| |
| randomEncryption.enable = true;
| |
| }
| |
| ];</pre>
| |
| | |
| ==== GRUB ====
| |
| ===== UEFI =====
| |
| For UEFI, delete systemd-boot and other <code>boot.loader</code> options and use GRUB:
| |
| <pre>boot.loader = {
| |
| # grub.efiInstallAsRemovable = true;
| |
| efi.canTouchEfiVariables = true;
| |
| efi.efiSysMountPoint = "/boot/efi";
| |
| grub.enable = true;
| |
| grub.version = 2;
| |
| grub.copyKernels = true;
| |
| grub.efiSupport = true;
| |
| grub.devices = [ "nodev" ];
| |
| };</pre>
| |
| For multiple disks, mirror EFI system partition <code>/boot/efi</code> with <code>boot.loader.grub.mirroredBoots</code> option. See <code>man configuration.nix</code>.
| |
| ===== BIOS =====
| |
| For BIOS, delete <code>boot.loader</code> options and use GRUB:
| |
| <pre>boot.loader = {
| |
| grub.enable = true;
| |
| grub.version = 2;
| |
| grub.copyKernels = true;
| |
| grub.devices = [ "/dev/disk/by-id/disk1" "/dev/disk/by-id/disk2" ];
| |
| };</pre>
| |
| ==== Installation ====
| |
| After finishing configuration, install system:
| |
| <pre>nixos-install --root /mnt</pre>
| |
| === Finishing installation ===
| |
| Take an initial snapshot:
| |
| <pre>zfs snapshot -r rpool/sys@install
| |
| zfs snapshot -r bpool/sys@install</pre>
| |
| Unmount EFI system partition:<pre>umount /mnt/boot/efi</pre>Also unmount mirrored ESP, if any.
| |
| | |
| Export pools:<pre>zpool export bpool
| |
| zpool export rpool</pre>
| |