Nix Hash: Difference between revisions

From NixOS Wiki
imported>Viktornordling
mNo edit summary
imported>Artturin
Link to docs
Line 9: Line 9:
=== Usage ===
=== Usage ===


Many derivations are so-called ''fixed-output'' derivations, meaning that you need to know and specify the hash of the output in advance. As an example, let's look at <code>fetchurl</code>:
Many derivations are so-called ''fixed-output'' derivations, meaning that you need to know and specify the hash of the output in advance. As an example, let's look at nixpkgs function <code>fetchurl</code>:
 
<syntaxHighlight lang=nix>
src = fetchurl {
  url = "https://example.org/downloads/source-code.zip";
  sha256 = "1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91";
};
</syntaxHighlight>
 
You can specify the hash in '''any base''' that's supported. Thus, <code>sha256 = "21d536debb3076d4f6e9044bd9ef15c8c58b29f9cbd4ad406b058310a565debc";</code> is equally allowed.
 
An alternative – and supposedly preferred – way of specifying hashes are so-called "SRI hashes". They're pretty simple, as the hash contains the algorithm used and always is in base64:


<syntaxHighlight lang=nix>
<syntaxHighlight lang=nix>
Line 28: Line 17:
};
};
</syntaxHighlight>
</syntaxHighlight>
If you find a hash that uses colon as a separator (<code><type>:<hash></code>), don't use that. This relies on undocumented behavior and is not officially supported.


=== Updating Packages ===
=== Updating Packages ===


When you are updating packages in nixpkgs.
[https://nixos.org/manual/nixpkgs/stable/#chap-pkgs-fetchers-caveats Using TOFU to get the new hash]
 
An easy way to get the Hash is TOFU (Trust On First Use), where we replace the hash in a .nix file with:
 
<syntaxHighlight lang=nix>
{
  hash = "sha256-IdU23rswdtT26QRL2e8VyMWLKfnL1K1AawWDEKVl3rw=";
};
</syntaxHighlight>
 
<syntaxHighlight lang=nix>
{
  hash = "";
};
</syntaxHighlight>
 
Nix assumes with blank quotes in the hash means a TOFU on a nix-build is needed and will get the Hash for you enter into the old .nix file.
 
<syntaxHighlight lang=nix>
{
  hash = "sha256-wlb6er8L2EaqgJbmbATBdSxx1BGcJXNcsu+/4UBmYdQ=";
};
</syntaxHighlight>
 
Once you have replaced the Hash, run nix-build or equivalent command again and it will verify the Hash and continue your build to update the package.


=== What exactly is hashed ===
=== What exactly is hashed ===
Line 71: Line 34:
=== Tools ===
=== Tools ===


The tool of choice for hashing is <code>nix-hash</code>, although it will be deprecated [https://github.com/NixOS/nix/issues/1191#issuecomment-273839319 one day] and replaced by subcommands of the [[Nix_command|<code>nix</code> command]]. Below is a comparison between the current <code>nix-hash</code> and the '''experimental''' replacements which already can be used today, but are subject to change.
[https://nixos.org/manual/nix/stable/command-ref/nix-hash.html nix-hash]
 
[https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-hash nix hash]
{|class="wikitable"
!nix-hash
!nix command
!explanation
|-
| <code>nix-hash --flat --type $HASHTYPE</code>
| <code>nix hash file --base16 --type $HASHTYPE</code>, see [[Nix_command/hash-file]]
| Hash a file by using a “flat” hash which directly hashes a file and behaves like the <code>{md5,sha1,sha256,sha512}sum</code> utilities.
|-
| <code>nix-hash --flat --base32 --type $HASHTYPE</code>
| <code> nix hash file --base32 --type $HASHTYPE</code>
| Like above, but with the more used base32 output.
|-
| <code>nix-hash --type $HASHTYPE</code>
| <code> nix hash path --base16 --type $HASHTYPE</code>, see [[Nix_command/hash-path]]
| Compute the hash of a given path's dump in the NAR format.
|-
| <code>nix-hash --base32 --type $HASHTYPE</code>
| <code> nix hash path --base32 --type $HASHTYPE</code>
| Like above, but with the more common base32 representation.
|-
| <code>nix-hash --to-base32 --type $HASHTYPE</code>
| <code>nix to-base32 --type $HASHTYPE</code>, see [[Nix_command/to-base32]]
| Convert a hash of <code>$HASHTYPE</code> to its (nix-specific) base32 representation.
|-
| <code>nix-hash --to-base16 --type $HASHTYPE</code>
| <code>nix to-base16 --type $HASHTYPE</code>, see [[Nix_command/to-base16]]
| Convert a hash of <code>$HASHTYPE</code> to its base16 representation.
|-
| (not supported)
| <code>nix hash to-sri --type $HASHTYPE</code>, see [[Nix_command/to-sri]]
| Convert a hash of <code>$HASHTYPE</code> to its SRI representation.
|-
| (not supported)
| <code>nix hash to-base64 --type $HASHTYPE</code>, see [[Nix_command/to-base64]]
| Convert a hash of <code>$HASHTYPE</code> to its base64 representation which is the SRI representation without the hash type indication.
|}
 
<code>$HASHTYPE</code> is either <code>md5</code> (deprecated in nixpkgs), <code>sha1</code>, <code>sha256</code> (current nixpkgs standard) or <code>sha512</code> ([https://github.com/NixOS/nix/issues/1191#issuecomment-273839319 candidate for the next standard hash]).  The main differences between <code>nix-hash</code> and the <code>nix</code> subcommands is the lack of support for SRI and base64 in the former and the lack of stability in the latter. The defaults settings for the two tools are as follows:
 
{|class="wikitable"
!setting
!nix-hash default
! nix command default
|-
| output format
| base16
| SRI with base64 hash representation
|-
| hash algorithm
| md5
| sha256
|}


When dealing with remote files, <code>nix-prefetch-url</code> offers a handy shortcut for downloading the file into the Nix store and printing out its hash. (<code>nix-prefetch-url --unpack</code> is its <code>fetchzip</code> equivalent.)
When dealing with remote files, <code>nix-prefetch-url</code> offers a handy shortcut for downloading the file into the Nix store and printing out its hash. (<code>nix-prefetch-url --unpack</code> is its <code>fetchzip</code> equivalent.)

Revision as of 03:36, 10 October 2023

Hashes in Nix

Cryptographic hashes play an essential role in a lot of places in the Nix ecosystem. When using a hash somewhere, two criteria are essential to do so properly: the algorithm used and the encoding (and, to some extent, what is hashed).

Supported algorithms are md5, sha1, sha256, sha512. The first two are deprecated and should not be used anymore, but you may still stumble upon them in existing code.

A hash – which is simply a sequence of bytes – is usually encoded in order to be representable as string. Common encodings are base16 (commonly called "hex"), base32 and base64. Note that the base32 is a custom one that is not documented nor standardized in any way! If possible, use the provided hashing tools to convert hashes to it (see below). base32 is used by Nix in a lot of places because it is shorter than hex but can still safely be part of a file path (as it contains no slashes).

Usage

Many derivations are so-called fixed-output derivations, meaning that you need to know and specify the hash of the output in advance. As an example, let's look at nixpkgs function fetchurl:

src = fetchurl {
  url = "https://example.org/downloads/source-code.zip";
  hash = "sha256-IdU23rswdtT26QRL2e8VyMWLKfnL1K1AawWDEKVl3rw=";
};

Updating Packages

Using TOFU to get the new hash

What exactly is hashed

Some content can either be hashed "flat" or "recursively". "flat" (sometimes also called "file") is simply taking the hash of the file, byte by byte, and will give you the same result as for example `sha256sum -b myfile.zip`. "recursive" (or sometimes "path") hashing takes multiple files, path names and metadata (attributes) into consideration. It works by NARing the input before hashing.

For fetchurl, the option to switch between both is called recursiveHash and defaults to false.

fetchzip on the other hand will download the file, unzip it and then recursively hash the output. There's no option.

The motivation behind this is that sometimes, the content is always the same, but the archive may change. This is because zip files are inherently non-deterministic, and might be generated automatically. If they are regenerated, they'll have a different hash, although the content is the same. recursiveHash works around that.

Tools

nix-hash nix hash

When dealing with remote files, nix-prefetch-url offers a handy shortcut for downloading the file into the Nix store and printing out its hash. (nix-prefetch-url --unpack is its fetchzip equivalent.)

Libraries

Further reading

  • Eelco Dolstra's PhD thesis, section 5.1.
  • Github Issue about which encoding is used where, and what pitfalls can arise from it.