Storage optimization: Difference between revisions

Line 8:

</syntaxhighlight>

Run the optimise command once since this option only applies to new files

Run the optimise command once since this option only applies to new files.

==== Manually ====

Run <code>nix-store --optimise</code> ~~this~~ will take some time to do

Run <code>nix-store --optimise</code>. This will take some time to complete.

== Garbage collection ==

Line 44:

/run/booted-system -> /nix/store/8jkrl9jyq7hqxb6xpwcaghpdm26gq98j-nixos-system-iron-16.0916.09pre.custom

/run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom</syntaxhighlight>

Information about gc roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:

Line 57:

Line 56:

=== Look for <code>result</code> symlinks ===

If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your FS will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:

If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your file system will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:

<syntaxhighlight lang="bash">/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0

/home/ec2-user/result -> /nix/store/q35aq2sh5dbyka6g6f6qb7b8msxwds5m-nixos-system-iron-16.03.1299.a8e0739

Line 82:

Line 81:

=== Pinning ===

When you invoke <code>nix-shell</code> with

When you invoke <code>nix-shell</code> with:

$ nix-instantiate shell.nix --indirect --add-root $DIR/.nix-gc-roots/shell.drv ...

</syntaxhighlight>

~~then you'll have a persistent environment which won't be garbage collected. It is useful when you don't want to spend time waiting for redownloads every time you enter the shell.~~

A little problem exists though. GC roots are numbered sequentially, so if you change <code>shell.nix</code> to contain '''~~less~~''' derivations, ~~and~~ name of last GC root ~~will start~~ with <code>shell.drv-7</code>, then <code>shell.drv-{8,9,10,11,12}*</code> will be dangling and unused. To overcome this problem you should remove GC roots dir periodically (or just before <code>nix-shell</code>)

Then you'll have a persistent environment which won't be garbage collected. This is useful when you don't want to spend time waiting for redownloads every time you enter the shell.

A little problem exists though. GC roots are numbered sequentially, so if you change <code>shell.nix</code> to contain '''fewer''' derivations, such that the name of the last GC root starts with <code>shell.drv-7</code>, then <code>shell.drv-{8,9,10,11,12}*</code> will be dangling and unused. To overcome this problem you should remove GC roots dir periodically (or just before running <code>nix-shell</code>).

Obviously, you should remove the GC roots directory for projects you don't plan to work on.

Line 123:

Regardless of <code>/nix</code>'s filesystem, it can also be mounted with <code>noatime</code> (as seen in the example below). This will reduce metadata writes, improving I/O and the device's lifespan.

This is easiest to set up while installing NixOS, but <code>/nix</code> can be moved on a live system:

Line 129:

Line 128:

# Create a new partition and mount it over <code>/mnt</code>

# <code>rsync -aAxv</code> everything from <code>/nix</code> to <code>/mnt</code>

# ~~bind~~ <code>/mnt</code> to <code>/nix</code> (e.g. using <code>mount</code>).

# Bind <code>/mnt</code> to <code>/nix</code> (e.g. using <code>mount</code>).

# Restart nix daemon with <code>systemctl restart nix-daemon.service </code> <code>systemctl restart nix-daemon.socket </code>.

# Rerun <code>nixos-rebuild switch</code> with something like <syntaxhighlight lang="bash">fileSystems."/nix" = { device = "/dev/disk/by-label/nix"; neededForBoot = true; options = [ "noatime" ]; };</syntaxhighlight>

# ~~reboot~~ to be sure <code>/nix/store</code> is properly mounted.

# Reboot to be sure <code>/nix/store</code> is properly mounted.

# Once you are sure everything works, you can delete the old store doing a bind mount <code>/</code> to <code>/old_root</code>, and remove <code>/old_root/nix</code>.

Keep in mind that all commands like <code>mount</code> and <code>bash</code> point to some executable in <code>/nix/store</code>, so never mount an empty disk over <code>/nix</code> or <code>/nix/store</code>, otherwise you will be locked out until reboot!

@@ Line 8: / Line 8: @@
 </syntaxhighlight>
-Run the optimise command once since this option only applies to new files
+Run the optimise command once since this option only applies to new files.
 ==== Manually ====
-Run <code>nix-store --optimise</code> this will take some time to do
+Run <code>nix-store --optimise</code>. This will take some time to complete.
 == Garbage collection ==
@@ Line 44: / Line 44: @@
 /run/booted-system -> /nix/store/8jkrl9jyq7hqxb6xpwcaghpdm26gq98j-nixos-system-iron-16.0916.09pre.custom
 /run/current-system -> /nix/store/wmndyzzrbc9fyjw844jmvzwgwgcinq7s-nixos-system-iron-16.0916.09pre.custom</syntaxhighlight>
 Information about gc roots can be found in <code>/nix/var/nix/gcroots</code>. The following script demonstrates how this directory can be used to (for example) query the state of manually made result symlinks:
@@ Line 57: / Line 56: @@
 === Look for <code>result</code> symlinks ===
-If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your FS will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:
+If you use <code>nix-build</code>, but not <code>--no-build-output</code>, your file system will be filled with <code>result</code> symlinks to various derivations. In the example above, note the following symlinks:
 <syntaxhighlight lang="bash">/home/danbst/stack/new/website/server/result -> /nix/store/1jhmp6vl364p32r8bjigk65qh1xa562f-server-0.1.0.0
 /home/ec2-user/result -> /nix/store/q35aq2sh5dbyka6g6f6qb7b8msxwds5m-nixos-system-iron-16.03.1299.a8e0739
@@ Line 82: / Line 81: @@
 === Pinning ===
-When you invoke <code>nix-shell</code> with
+When you invoke <code>nix-shell</code> with:
 <syntaxhighlight lang="console">
 $ nix-instantiate shell.nix --indirect --add-root $DIR/.nix-gc-roots/shell.drv ...
 </syntaxhighlight>
-then you'll have a persistent environment which won't be garbage collected. It is useful when you don't want to spend time waiting for redownloads every time you enter the shell.
-A little problem exists though. GC roots are numbered sequentially, so if you change <code>shell.nix</code> to contain '''less''' derivations, and name of last GC root will start with <code>shell.drv-7</code>, then <code>shell.drv-{8,9,10,11,12}*</code> will be dangling and unused. To overcome this problem you should remove GC roots dir periodically (or just before <code>nix-shell</code>)
+Then you'll have a persistent environment which won't be garbage collected. This is useful when you don't want to spend time waiting for redownloads every time you enter the shell.
+A little problem exists though. GC roots are numbered sequentially, so if you change <code>shell.nix</code> to contain '''fewer''' derivations, such that the name of the last GC root starts with <code>shell.drv-7</code>, then <code>shell.drv-{8,9,10,11,12}*</code> will be dangling and unused. To overcome this problem you should remove GC roots dir periodically (or just before running <code>nix-shell</code>).
 Obviously, you should remove the GC roots directory for projects you don't plan to work on.
@@ Line 123: / Line 123: @@
 Regardless of <code>/nix</code>'s filesystem, it can also be mounted with <code>noatime</code> (as seen in the example below). This will reduce metadata writes, improving I/O and the device's lifespan.
 This is easiest to set up while installing NixOS, but <code>/nix</code> can be moved on a live system:
@@ Line 129: / Line 128: @@
 # Create a new partition and mount it over <code>/mnt</code>
 # <code>rsync -aAxv</code> everything from <code>/nix</code> to <code>/mnt</code>
-# bind <code>/mnt</code> to <code>/nix</code> (e.g. using <code>mount</code>).
+# Bind <code>/mnt</code> to <code>/nix</code> (e.g. using <code>mount</code>).
 # Restart nix daemon with <code>systemctl restart nix-daemon.service </code> <code>systemctl restart nix-daemon.socket </code>.
 # Rerun <code>nixos-rebuild switch</code> with something like <syntaxhighlight lang="bash">fileSystems."/nix" = { device = "/dev/disk/by-label/nix";  neededForBoot = true; options = [ "noatime" ]; };</syntaxhighlight>
-# reboot to be sure <code>/nix/store</code> is properly mounted.
+# Reboot to be sure <code>/nix/store</code> is properly mounted.
 # Once you are sure everything works, you can delete the old store doing a bind mount <code>/</code> to <code>/old_root</code>, and remove <code>/old_root/nix</code>.
 Keep in mind that all commands like <code>mount</code> and <code>bash</code> point to some executable in <code>/nix/store</code>, so never mount an empty disk over <code>/nix</code> or <code>/nix/store</code>, otherwise you will be locked out until reboot!