Storage optimization: Difference between revisions

Adda0 (talk | contribs)
m Fix trailing typo.
Tags: Mobile edit Mobile web edit
DoggoBit (talk | contribs)
m propose merge with Nix store
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Merge|Nix store}}
A recurring problem with NixOS is lack of space on <code>/</code>. Even if you only occasionally use Nix, it is easy for <code>/nix/store</code> to grow beyond reasonable sizes. What follows are generic notes on how to reduce the growth of the [[Nix store]].  
A recurring problem with NixOS is lack of space on <code>/</code>. Even if you only occasionally use Nix, it is easy for <code>/nix/store</code> to grow beyond reasonable sizes. What follows are generic notes on how to reduce the growth of the [[Nix store]].  


Line 25: Line 27:
== Garbage collection ==
== Garbage collection ==


The Nix store accumulates entries which are no longer useful.<ref group="cf.">[https://nixos.org/nix/manual/#sec-garbage-collection Nix Manual, 11. Garbage Collection]</ref> They can be deleted with {{ic|nix-collect-garbage}} <ref group="cf.">{{man|nix-collect-garbage|sec=1}}</ref> or {{ic|nix-store --gc}}.<ref group="cf.">{{man|nix-store|sec=1}}, under {{ic|OPERATION --GC}}</ref>
By default, the Nix store will not remove any entries that are no longer used, thus it can accumulate derivations you might no longer need.<ref group="cf.">[https://nixos.org/nix/manual/#sec-garbage-collection Nix Manual, 11. Garbage Collection]</ref> They can be deleted with {{ic|nix-collect-garbage}} <ref group="cf.">{{man|nix-collect-garbage|sec=1}}</ref> or {{ic|nix-store --gc}}.<ref group="cf.">{{man|nix-store|sec=1}}, under {{ic|OPERATION --GC}}</ref>
 
=== Removing old generations ===
NixOS keeps old configurations of your system around so that you can always rollback to a previous configuration if something goes wrong. You can also select which generation to boot into via GRUB. However these previous generations are [[Storage optimization#Garbage collection roots|GC roots]] (see below) that can keep around old, unnecessary software in your nix store. You can check what system generations you have by:<syntaxhighlight lang="console">$ sudo nix-env -p /nix/var/nix/profiles/system --list-generations
...
  58  2021-09-04 02:56:54
  59  2021-09-05 07:12:43
  60  2021-09-05 22:12:13  (current)</syntaxhighlight>
 
You can remove all other NixOS generations besides the current one:
<syntaxhighlight lang="console">
$ sudo nix-collect-garbage -d
...
4394 store paths deleted, 3467.28 MiB freed
</syntaxhighlight>
 
There are also user-specific generations for different things (eg. home-manager). These can be removed with:
<syntaxhighlight lang="console">
$ nix-collect-garbage -d
</syntaxhighlight>
 
=== Garbage collection roots ===


Note that if a result file still exists in the file system, and your Nix configuration has both <code>keep-outputs = true</code> and <code>keep-derivations = true</code>, all the dependencies used to build it will be kept. To see which result files prevent garbage collection, run:
Note that if a result file still exists in the file system, and your Nix configuration has both <code>keep-outputs = true</code> and <code>keep-derivations = true</code>, all the dependencies used to build it will be kept. To see which result files prevent garbage collection, run:
Line 31: Line 54:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
$ nix-store --gc --print-roots
$ nix-store --gc --print-roots
/home/danbst/dev/test-shell/.shell.drv -> /nix/store/4diqwczyjipdqyi7aj34wfagblbhfjr9-nixops-1.4
/home/danbst/dev/test-shell/.shell.drv-2 -> /nix/store/62h3c4d6rdnlxichixqg8h9jxi8nhxk0-stdenv
/home/danbst/dev/test-shell/.shell.drv-2-doc -> /nix/store/14gnv1q1w0n9qwa3q23idsqvn51354y8-bash-4.3-p42-doc
/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0
/home/danbst/testing/.nix-gc-roots/shell.drv -> /nix/store/v3vqf48awjjzjivrx15kfqdh1d7cg4mq-sshpass-1.05
...
/home/danbst/testing/.nix-gc-roots/shell.drv-12 -> /nix/store/a2li4sl9pxh9aflqia2gp7w88ayvjwci-bash-4.3-p42
/home/danbst/testing/.nix-gc-roots/shell.drv-12-doc -> /nix/store/kcswyb1d8zimkym0pjfi2fj1dly1w34w-bash-4.3-p42-doc
/home/danbst/testing/.nix-gc-roots/shell.drv-12-info -> /nix/store/njb817fwiafswzwvj9skw7w7k6b3fnbi-bash-4.3-p42-info
/home/ec2-user/result -> /nix/store/q35aq2sh5dbyka6g6f6qb7b8msxwds5m-nixos-system-iron-16.03.1299.a8e0739
/nix/var/nix/profiles/per-container/analyt/system-3-link -> /nix/store/snrj72189wh9va23fawl3v80v92xnxlm-nixos-system-iron-16.03.1291.efe2d64
/nix/var/nix/profiles/per-container/d-live/system-6-link -> /nix/store/cp2c58hnczsjk5h69ksajq5xfhsyhl6v-nixos-system-iron-16.03.1299.a8e0739
/nix/var/nix/profiles/per-container/d-test/system-4-link -> /nix/store/n1w7ywjg65x8iimchznxcyygbgmyfh55-nixos-system-iron-16.03.1287.6ac7ffd
/nix/var/nix/profiles/per-container/dashboard/system-41-link -> /nix/store/7qk19pkwgq0h3a1q9dcql3nks40rr75s-nixos-system-iron-16.03.1340.5a090dd
...
...
/nix/var/nix/profiles/per-container/ttt/system-1-link -> /nix/store/1kj9qs5gl3421jlkl3jfc2kqdsl8akwr-nixos-system-ttt-16.03.977.1da05df
/nix/var/nix/profiles/per-user/danbst/channels-1-link -> /nix/store/s0qay9qyqrn92zayldbvvj3zrfcl7a72-user-environment
/nix/var/nix/profiles/per-user/danbst/profile-28-link -> /nix/store/69ds606146dqml04sm0fbpqwnv2w8i3q-user-environment
/nix/var/nix/profiles/per-user/ec2-user/profile-7-link -> /nix/store/y2hc7zsnkzys9ba6xaijvjhff03rcgpy-user-environment
/nix/var/nix/profiles/per-user/root/channels-4-link -> /nix/store/254b6pkhhnjywvj5c0lp2vdai8nz4p0g-user-environment
/nix/var/nix/profiles/per-user/root/channels-4-link -> /nix/store/254b6pkhhnjywvj5c0lp2vdai8nz4p0g-user-environment
/nix/var/nix/profiles/system-398-link -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom
/nix/var/nix/profiles/system-398-link -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom
Line 97: Line 102:


Look for system derivations in particular. Those are created on many occasions, for example when running <code>nixos-rebuild  build-vm</code>
Look for system derivations in particular. Those are created on many occasions, for example when running <code>nixos-rebuild  build-vm</code>
=== Removing old generations ===
NixOS keeps old configurations of your system around so that you can always rollback to a previous configuration if something goes wrong. You can also select which generation to boot into via GRUB.
However these previous generations are GC roots that can keep around old, unnecessary software in your nix store. You can check what system generations you have with
<syntaxhighlight lang="console">
$ sudo nix-env -p /nix/var/nix/profiles/system --list-generations
...
  58  2021-09-04 02:56:54
  59  2021-09-05 07:12:43
  60  2021-09-05 22:12:13  (current)
</syntaxhighlight>
You can remove all but the current generation with
<syntaxhighlight lang="console">
$ sudo nix-collect-garbage -d
...
4394 store paths deleted, 3467.28 MiB freed
</syntaxhighlight>
There are also user-specific generations for different things (eg. home-manager). These can be removed with
<syntaxhighlight lang="console">
$ nix-collect-garbage -d
</syntaxhighlight>


=== Reboot ===
=== Reboot ===
As you see, the reference in <code>/run/booted-system</code> is a GC root, so it won't be cleared until reboot. If you don't want to reboot, just <code>rm /run/booted-system</code> that link and rerun <code>sudo nix-collect-garbage</code>.
As you see, the reference in <code>/run/booted-system</code> is a GC root, so it won't be cleared until reboot. If you don't want to reboot, just <code>rm /run/booted-system</code> that link and rerun <code>sudo nix-collect-garbage</code>.


Line 190: Line 170:
# Create a new partition
# Create a new partition
# Mount this new partition over <code>/mnt</code> <syntaxhighlight lang="console">
# Mount this new partition over <code>/mnt</code> <syntaxhighlight lang="console">
# mount /dev/disk/by-label/nix /mnt
# mount -o defaults,noatime /dev/disk/by-label/nix /mnt/nix
</syntaxhighlight>
</syntaxhighlight>
# Copy everything from <code>/nix</code> to <code>/mnt</code> ''Trailing slashes are important'', in that without them, <code>rsync</code> will create an additional directory of the same name at the destination. <syntaxhighlight lang="console">
# Copy everything from <code>/nix</code> to <code>/mnt</code> ''Trailing slashes are important'', in that without them, <code>rsync</code> will create an additional directory of the same name at the destination. <syntaxhighlight lang="console">
# rsync --archive --hardlinks --acls --one-file-system --verbose /nix/store/ /mnt/store
# rsync --archive --hard-links --acls --one-file-system --verbose /nix/{store,var} /mnt/nix
# rsync --archive --hardlinks --acls --one-file-system --verbose /nix/var/ /mnt/var
</syntaxhighlight>
</syntaxhighlight>
# Mount the new partition as the new <code>/nix</code> <syntaxhighlight lang="console">
# Mount the new partition as the new <code>/nix</code> <syntaxhighlight lang="console">
# umount /mnt
# umount /mnt/nix
# mount /dev/disk/by-label/nix /nix
# mount /dev/disk/by-label/nix /nix
</syntaxhighlight>
</syntaxhighlight>
Line 238: Line 217:


<references group="cf."/>
<references group="cf."/>
[[Category:NixOS]]
[[Category:Nix]]