Enterprise: Difference between revisions

From NixOS Wiki
imported>Bobvanderlinden
netrc and fetching private resources
 
imported>Bobvanderlinden
add Reproducibility across workstations and some minor changes
Line 1: Line 1:
When trying to use Nix and NixOS in corporations there are a number of issues one will run into. This page tries to provide a solution to each of these issues.
When trying to use Nix and NixOS in corporations there are a number of issues one will run into. This page tries to provide a solution to each of these issues.
== Reproducibility across workstations ==
Most examples of Nix expressions use the following to import Nixpkgs:
let pkgs = import <nixpkgs> { }
in ...
This is fine when you're the only one using this expression, however when others also use this expression you will run into reproducability issues. <code>&lt;nixpkgs&gt;</code> is a reference to the <code>nixpkgs</code> entry in your <code>NIX_PATH</code> or nix-channels. That means different workstations will have different versions of Nixpkgs.
To solve this problem we will pin the version of Nixpkgs in our package definition. The following example will pin on the latest revision of release-18.03 of Nixpkgs at the time of writing:
let pkgs = import ((import <nixpkgs> { }).fetchFromGitHub {
  owner = "NixOS";
  repo = "nixpkgs";
  rev = "c674fa8eac306ffe79348b39cd814cec18da3425";
  sha256 = "1jpyhir2fvbg92frq8rzahds4jxbi0p9p6rjs6lwfbxfxc3yvg8i";
}) { }
in ...
When nixpkgs is pinned all users of the expression will have the exact same build, no matter what nix-channels or NIX_PATH are configured.


== Private resources ==
== Private resources ==
Line 18: Line 39:
     password SECRET
     password SECRET


Also, make sure the nix-daemon and nixbld users have permission to this file.
Next the netrc file needs to be accessible in the builds. We will configure Nix to allow access to this file directly from the build sandboxes. Edit your <code>/etc/nix/nix.conf</code> file so that it includes the following lines:
Next the netrc file needs to be accessible in the builds. We will configure Nix to allow access to this file directly from the build sandboxes. Edit your <code>/etc/nix/nix.conf</code> file so that it includes the following lines:


Line 33: Line 55:
   };
   };
  }
  }
  in
  in ...
...


Now all fetchurl calls will use the specified netrc file with the credentials of your choice.
Now all fetchurl calls will use the specified netrc file with the credentials of your choice.

Revision as of 20:21, 12 May 2018

When trying to use Nix and NixOS in corporations there are a number of issues one will run into. This page tries to provide a solution to each of these issues.

Reproducibility across workstations

Most examples of Nix expressions use the following to import Nixpkgs:

let pkgs = import <nixpkgs> { }
in ...

This is fine when you're the only one using this expression, however when others also use this expression you will run into reproducability issues. <nixpkgs> is a reference to the nixpkgs entry in your NIX_PATH or nix-channels. That means different workstations will have different versions of Nixpkgs.

To solve this problem we will pin the version of Nixpkgs in our package definition. The following example will pin on the latest revision of release-18.03 of Nixpkgs at the time of writing:

let pkgs = import ((import <nixpkgs> { }).fetchFromGitHub {
  owner = "NixOS";
  repo = "nixpkgs";
  rev = "c674fa8eac306ffe79348b39cd814cec18da3425";
  sha256 = "1jpyhir2fvbg92frq8rzahds4jxbi0p9p6rjs6lwfbxfxc3yvg8i";
}) { }
in ...

When nixpkgs is pinned all users of the expression will have the exact same build, no matter what nix-channels or NIX_PATH are configured.

Private resources

Building internal projects will require fetching of internal (private) source code and other resources. These resources usually are protected some form of credentials.

fetchurl

fetchurl is used to retrieve HTTP resources, but is also used by fetchFromGithub. For private resources this will usually result in an error like the following:

curl: (22) The requested URL returned error: 401 Unauthorized

Nix will not know about your credentials in your home directory, as the builders have no access to those files. However, Nix has a few options borrowed from curl that will help in this situation. A netrc file can be used that holds the credentials for all domains that require authorisation. More information on netrc can be found in the GNU manual. For our example, we will create the file in /etc/nix/netrc. The contents will look similar to the following:

machine DOMAINNAME
    login USERNAME
    password SECRET

Also, make sure the nix-daemon and nixbld users have permission to this file. Next the netrc file needs to be accessible in the builds. We will configure Nix to allow access to this file directly from the build sandboxes. Edit your /etc/nix/nix.conf file so that it includes the following lines:

build-sandbox-paths = /etc/nix/netrc

Lastly, the builds need to know that they need to use the netrc file in fetchurl. We will override the definition of fetchurl to include --netrc-file /etc/nix/netrc in the curl options used by fetchurl. The following shows how this might look in your Nix file:

let pkgs = import <nixpkgs> {
  config = {
    packageOverrides = pkgs: rec {
      fetchurlPrivate = opts: pkgs.fetchurl (opts // {
        curlOpts = "${pkgs.lib.optionalString (opts ? curlOpts) "${opts.curlOpts}"} --netrc-file /etc/nix/netrc";
      });
      fetchrurl = fetchurlPrivate;
  };
}
in ...

Now all fetchurl calls will use the specified netrc file with the credentials of your choice.