Storage optimization: Difference between revisions

Line 1:

A recurring problem with NixOS is lack of space on <code>/</code>. Even if you ~~are using~~ Nix ~~only occasionally~~, it is easy for <code>/nix/store</code> to go beyond 50GiB. Here are generic notes on how to not run out of space too often.

A recurring problem with NixOS is lack of space on <code>/</code>. Even if you only ocasionally use Nix, it is easy for <code>/nix/store</code> to grow beyond 50GiB. Here are generic notes on how to not run out of space too often.

== Optimizing the store ==

The option and command below save space by hardlinking store files:

==== Automatically ====

Line 8:

Line 9:

</syntaxhighlight>

~~Run the optimise command once since~~ this option only applies to new files.

But this option only applies to new files: you'll still need to manually optimise your store once, after you enable this option.

==== Manually ====

Line 19:

Line 20:

Note that if a result file still exists in the file system, and your Nix configuration has both <code>keep-outputs = true</code> and <code>keep-derivations = true</code>, all the dependencies used to build it will be kept. To see which result files prevent garbage collection, run:

<syntaxhighlight lang="console">$ nix-store --gc --print-roots

$ nix-store --gc --print-roots

/home/danbst/dev/test-shell/.shell.drv -> /nix/store/4diqwczyjipdqyi7aj34wfagblbhfjr9-nixops-1.4

/home/danbst/dev/test-shell/.shell.drv-2 -> /nix/store/62h3c4d6rdnlxichixqg8h9jxi8nhxk0-stdenv

Line 43:

Line 45:

/root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git

/run/booted-system -> /nix/store/8jkrl9jyq7hqxb6xpwcaghpdm26gq98j-nixos-system-iron-16.0916.09pre.custom

/run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom</syntaxhighlight>

/run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom

</syntaxhighlight>

~~Information about gc~~ roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:

GC roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:

<syntaxhighlight lang="bash">find -H /nix/var/nix/gcroots/auto -type l | xargs -I {} sh -c 'readlink {}; realpath {}; echo'</syntaxhighlight>

find -H /nix/var/nix/gcroots/auto -type l | xargs -I {} sh -c 'readlink {}; realpath {}; echo'

</syntaxhighlight>

This acts a simpler (but faster) version of <code>--print-roots</code> and could be implemented as a bash alias for convenience.

Line 57:

Line 62:

If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your file system will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:

<syntaxhighlight lang="bash">/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0

/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0

/home/ec2-user/result -> /nix/store/q35aq2sh5dbyka6g6f6qb7b8msxwds5m-nixos-system-iron-16.03.1299.a8e0739

/root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git</syntaxhighlight>

/root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git

</syntaxhighlight>

How much space do these (apparently) abandoned derivations use?

<syntaxhighlight lang="console">$ du -sch $(nix-store -qR /root/forkstat/result /home/ec2-user/result /home/danbst/stack/new/website/server/result)

$ du -sch $(nix-store -qR /root/forkstat/result /home/ec2-user/result /home/danbst/stack/new/website/server/result)

...

3.4G total</syntaxhighlight>

3.4G total

</syntaxhighlight>

Not all of the derivations are garbage in this case, but quite a few are:

Line 73:

Line 85:

690 store paths deleted, 1817.99 MiB freed

</syntaxhighlight>

Look for system derivations in particular. Those are created on many occasions, for example when running <code>nixos-rebuild build-vm</code>

Line 108:

Line 121:

It is also possible to automatically run garbage collection whenever there is not enough space left.<ref group="cf.">{{nix:option|min-free}} and {{nix:option|max-free}}</ref> For example, to free up to 1GiB whenever there is less than 100MiB left:

nix.extraOptions = ''

Line 118:

Line 132:

== Moving the store ==

{{ic|/nix}} can reside on another device. This is useful if your root device is very small, and ~~that~~ you have another, larger drive ~~at hand~~.

{{ic|/nix}} can reside on another device. This is useful if your root device is very small, and you have another, larger drive available.

If the second mountpoint is on the same device, some benefit can still be gained by formatting the partition it points to with a different file system. For example: on a Raspberry Pi, f2fs could possibly be used for a gain in I/O throughput.

@@ Line 1: / Line 1: @@
-A recurring problem with NixOS is lack of space on <code>/</code>. Even if you are using Nix only occasionally, it is easy for <code>/nix/store</code> to go beyond 50GiB. Here are generic notes on how to not run out of space too often.
+A recurring problem with NixOS is lack of space on <code>/</code>. Even if you only ocasionally use Nix, it is easy for <code>/nix/store</code> to grow beyond 50GiB. Here are generic notes on how to not run out of space too often.
 == Optimizing the store ==
 The option and command below save space by hardlinking store files:
 ==== Automatically ====
 <syntaxhighlight lang="nix">
@@ Line 8: / Line 9: @@
 </syntaxhighlight>
-Run the optimise command once since this option only applies to new files.
+But this option only applies to new files: you'll still need to manually optimise your store once, after you enable this option.
 ==== Manually ====
@@ Line 19: / Line 20: @@
 Note that if a result file still exists in the file system, and your Nix configuration has both <code>keep-outputs = true</code> and <code>keep-derivations = true</code>, all the dependencies used to build it will be kept. To see which result files prevent garbage collection, run:
-<syntaxhighlight lang="console">$ nix-store --gc --print-roots
+<syntaxhighlight lang="console">
+$ nix-store --gc --print-roots
 /home/danbst/dev/test-shell/.shell.drv -> /nix/store/4diqwczyjipdqyi7aj34wfagblbhfjr9-nixops-1.4
 /home/danbst/dev/test-shell/.shell.drv-2 -> /nix/store/62h3c4d6rdnlxichixqg8h9jxi8nhxk0-stdenv
@@ Line 43: / Line 45: @@
 /root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git
 /run/booted-system -> /nix/store/8jkrl9jyq7hqxb6xpwcaghpdm26gq98j-nixos-system-iron-16.0916.09pre.custom
-/run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom</syntaxhighlight>
+/run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom
+</syntaxhighlight>
-Information about gc roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:
+GC roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:
-<syntaxhighlight lang="bash">find -H /nix/var/nix/gcroots/auto -type l | xargs -I {} sh -c 'readlink {}; realpath {}; echo'</syntaxhighlight>
+<syntaxhighlight lang="bash">
+find -H /nix/var/nix/gcroots/auto -type l | xargs -I {} sh -c 'readlink {}; realpath {}; echo'
+</syntaxhighlight>
 This acts a simpler (but faster) version of <code>--print-roots</code> and could be implemented as a bash alias for convenience.
@@ Line 57: / Line 62: @@
 If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your file system will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:
-<syntaxhighlight lang="bash">/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0
+<syntaxhighlight lang="bash">
+/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0
 /home/ec2-user/result -> /nix/store/q35aq2sh5dbyka6g6f6qb7b8msxwds5m-nixos-system-iron-16.03.1299.a8e0739
-/root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git</syntaxhighlight>
+/root/forkstat/result -> /nix/store/i5glmg3wk2a48x52rhd92zip1cmc0kq9-forkstat-git
+</syntaxhighlight>
 How much space do these (apparently) abandoned derivations use?
-<syntaxhighlight lang="console">$ du -sch $(nix-store -qR /root/forkstat/result /home/ec2-user/result /home/danbst/stack/new/website/server/result)
+<syntaxhighlight lang="console">
+$ du -sch $(nix-store -qR /root/forkstat/result /home/ec2-user/result /home/danbst/stack/new/website/server/result)
 ...
-.4G    total</syntaxhighlight>
+.4G    total
+</syntaxhighlight>
 Not all of the derivations are garbage in this case, but quite a few are:
@@ Line 73: / Line 85: @@
 store paths deleted, 1817.99 MiB freed
 </syntaxhighlight>
 Look for system derivations in particular. Those are created on many occasions, for example when running <code>nixos-rebuild  build-vm</code>
@@ Line 108: / Line 121: @@
 It is also possible to automatically run garbage collection whenever there is not enough space left.<ref group="cf.">{{nix:option|min-free}} and {{nix:option|max-free}}</ref> For example, to free up to 1GiB whenever there is less than 100MiB left:
 <syntaxhighlight lang="nix">
 nix.extraOptions = ''
@@ Line 118: / Line 132: @@
 == Moving the store ==
-{{ic|/nix}} can reside on another device. This is useful if your root device is very small, and that you have another, larger drive at hand.
+{{ic|/nix}} can reside on another device. This is useful if your root device is very small, and you have another, larger drive available.
 If the second mountpoint is on the same device, some benefit can still be gained by formatting the partition it points to with a different file system. For example: on a Raspberry Pi, f2fs could possibly be used for a gain in I/O throughput.