Enterprise: Difference between revisions

Latest revision as of 18:20, 30 August 2024

When trying to use Nix and NixOS in corporations there are a number of issues one will run into. This page tries to provide a solution to each of these issues.

Private resources

Building internal projects will require fetching of internal (private) source code and other resources. These resources usually are protected by some form of credentials.

fetchurl

fetchurl is used to retrieve HTTP resources, but is also used by fetchFromGitHub. For private resources this will usually result in an error like the following:

curl: (22) The requested URL returned error: 401 Unauthorized

Nix will not know about your credentials in your home directory, as the builders have no access to those files. However, Nix has a few options borrowed from curl that will help in this situation. A netrc file can be used that holds the credentials for all domains that require authorisation. More information on netrc can be found in the GNU manual. For our example, we will create the file in /etc/nix/netrc. The contents will look similar to the following:

machine DOMAINNAME
    login USERNAME
    password SECRET

Next the netrc file needs to be accessible in the builds. We will configure Nix to allow access to this file directly from the build sandboxes. Edit your /etc/nix/nix.conf file so that it includes the following lines:

netrc-file = /etc/nix/netrc

Lastly, the default way of fetching urls is using curl inside a build sandbox. This is a powerful command, but it will not use (and cannot use) a netrc file that is outside of the build sandbox. netrc-file is thus only applicable to fetches being done by Nix itself. In addition, we do not want to place the netrc file inside the sandbox, because that could potentially leak private credentials into builds. fetchurlBoot uses this builtin function and makes sure the call-sign is mostly compatible with the regular fetchurl. This function is used most often to bootstrap some of the more basic packages like curl itself, but it can also be very useful for fetching files outside of the sandbox.

Since fetchurlBoot is mostly compatible with fetchurl we can override fetchurl where needed:

mypackage = callPackage <mypackage.nix> {
  fetchurl = stdenv.fetchurlBoot;
};

Now the package is built exactly the same way as before, but resources will be fetched using fetchurlBoot. fetchurlBoot will in turn download the resources within Nix itself, which will use the netrc-file and use the right credentials for the domain names that you have defined.

TLS Intercepting Proxy

TLS-Intercepting proxies will intercept each and every TLS connection and replace the original certificate with it's own to be able to introspect the traffic. This of course creates validation issues with the "official" ca-certificate project. Since nix pr #2181 you are able to set your intercepting Proxy certificate via NIX_SSL_CERT_FILE to a file on your system which contains the root and intermediate certificates of your proxy. See also the appropriate section in the nix manual

The proxy itself can be set via the environment variables HTTP_PROXY and HTTPS_PROXY.

@@ Line 1: / Line 1: @@
-When trying to use Nix and NixOS in corporations there are a number of hurdles to overcome that aren't always clear from the start. This page describes a number of practices that will help integrating Nix within a company.
+When trying to use Nix and NixOS in corporations there are a number of issues one will run into. This page tries to provide a solution to each of these issues.
-== Nixpkgs Overlays ==
-Most companies will need additional packages that should not be in Nixpkgs. This includes internal packages, but also variations and patched versions of existing packages. A practical way to include these is by using an overlay. More information about overlays can be found in the [https://nixos.org/nixpkgs/manual/#chap-overlays Nixpkgs manual].
-A good example of a company owned and open source overlay is [https://github.com/mozilla/nixpkgs-mozilla nixpkgs-mozilla].
-== Reproducibility across workstations and pinning==
-Most examples of Nix expressions use the following to import Nixpkgs:
- let pkgs = import <nixpkgs> { }
- in ...
-This is fine when you're the only one using this expression, however when others also use this expression you will run into reproducability issues. <code>&lt;nixpkgs&gt;</code> is a reference to the <code>nixpkgs</code> entry in your <code>NIX_PATH</code> or nix-channels. That means different workstations will have different versions of Nixpkgs.
-To solve this problem we will pin the version of Nixpkgs in our package definition. The following example will pin on the latest revision of release-18.03 of Nixpkgs at the time of writing:
- let pkgs = import ((import <nixpkgs> { }).fetchFromGitHub {
-   owner = "NixOS";
-   repo = "nixpkgs";
-   rev = "c674fa8eac306ffe79348b39cd814cec18da3425";
-   sha256 = "1jpyhir2fvbg92frq8rzahds4jxbi0p9p6rjs6lwfbxfxc3yvg8i";
- }) { }
- in ...
-When nixpkgs is pinned all users of the expression will have the exact same build, no matter what nix-channels or NIX_PATH are configured.
 == Private resources ==
-Building internal projects will require fetching of internal (private) source code and other resources. These resources usually are protected some form of credentials.
+Building internal projects will require fetching of internal (private) source code and other resources. These resources usually are protected by some form of credentials.
 === fetchurl ===
-<code>fetchurl</code> is used to retrieve HTTP(S) resources, but is also used by <code>fetchFromGithub</code>. For private resources this will usually result in an error like the following:
+<code>fetchurl</code> is used to retrieve HTTP resources, but is also used by <code>fetchFromGitHub</code>. For private resources this will usually result in an error like the following:
   curl: (22) The requested URL returned error: 401 Unauthorized
-Nix will not know about your credentials in your home directory, as the builders have no access to those files. However, Nix has a few options borrowed from curl that will help in this situation. A netrc file can be used that holds the credentials for all domains that require authorisation. More information on netrc can be found in the [https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html GNU manual].
+Nix will not know about your credentials in your home directory, as the builders have no access to those files. However, Nix has a few options borrowed from <code>curl</code> that will help in this situation. A netrc file can be used that holds the credentials for all domains that require authorisation. More information on netrc can be found in the [https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html GNU manual].
 For our example, we will create the file in <code>/etc/nix/netrc</code>. The contents will look similar to the following:
@@ Line 45: / Line 18: @@
       password SECRET
-Also, make sure the nix-daemon and nixbld users have permission to this file.
 Next the netrc file needs to be accessible in the builds. We will configure Nix to allow access to this file directly from the build sandboxes. Edit your <code>/etc/nix/nix.conf</code> file so that it includes the following lines:
-  build-sandbox-paths = /etc/nix/netrc
+  netrc-file = /etc/nix/netrc
+Lastly, the default way of fetching urls is using <code>curl</code> inside a build sandbox. This is a powerful command, but it will not use (and cannot use) a netrc file that is outside of the build sandbox. <code>netrc-file</code> is thus only applicable to fetches being done by Nix itself. In addition, we do not want to place the netrc file inside the sandbox, because that could potentially leak private credentials into builds. <code>fetchurlBoot</code> uses this builtin function and makes sure the call-sign is mostly compatible with the regular <code>fetchurl</code>. This function is used most often to bootstrap some of the more basic packages like <code>curl</code> itself, but it can also be very useful for fetching files outside of the sandbox.
+Since '''fetchurlBoot''' is mostly compatible with '''fetchurl''' we can override '''fetchurl''' where needed:
+<syntaxHighlight lang=nix>
+mypackage = callPackage <mypackage.nix> {
+  fetchurl = stdenv.fetchurlBoot;
+};
+</syntaxHighlight>
-Lastly, the builds need to know that they need to use the netrc file in fetchurl. We will override the definition of fetchurl to include <code>--netrc-file /etc/nix/netrc</code> in the curl options used by fetchurl. The following shows how this might look in your Nix file:
+Now the package is built exactly the same way as before, but resources will be fetched using '''fetchurlBoot'''. '''fetchurlBoot''' will in turn download the resources within Nix itself, which will use the netrc-file and use the right credentials for the domain names that you have defined.
- let pkgs = import <nixpkgs> {
+== TLS Intercepting Proxy ==
-   config = {
+TLS-Intercepting proxies will intercept each and every TLS connection and replace the original certificate with it's own to be able to introspect the traffic. This of course creates validation issues with the "official" ca-certificate project.
-     packageOverrides = pkgs: rec {
+Since [https://github.com/NixOS/nix/issues/1896 nix pr #2181] you are able to set your intercepting Proxy certificate via <code>NIX_SSL_CERT_FILE</code> to a file on your system which contains the root and intermediate certificates of your proxy.
-       fetchurlPrivate = opts: pkgs.fetchurl (opts // {
+See also [https://nixos.org/nix/manual/#sec-nix-ssl-cert-file the appropriate section in the nix manual]
-         curlOpts = "${pkgs.lib.optionalString (opts ? curlOpts) "${opts.curlOpts}"} --netrc-file /etc/nix/netrc";
-       });
-       fetchrurl = fetchurlPrivate;
-   };
- }
- in ...
-Now all fetchurl calls will use the specified netrc file with the credentials of your choice.
+The proxy itself can be set via the environment variables <code>HTTP_PROXY</code> and <code>HTTPS_PROXY</code>.