Install NixOS on GCE

This is a recipe for creating a NixOS machine on Google Compute Engine (GCE) which is part of Google Cloud Platform.

This tutorial assumes you have already set up and account and project under Google Cloud Platform.

This tutorial is confirmed from NixOS 19.03 onward only (see release notes for 19.03).

Note also that nixops provides automated tooling to deploy NixOS on GCE instances. This tutorial covers manually deploying a NixOS GCE instance.

Obtain the gsutil utility

gsutil is part of google-cloud-sdk; you can obtain this several ways:

1. Using nixpkgs: nix-env --install google-cloud-sdk
2. Using an existing non-Nix GCE instance (eg Debian)
3. Following Google's instructions here: cloud.google.com

Select a NixOS image

Get a list of currently available GCE images with gsutil ls -l gs://nixos-images:

$ gsutil ls -l gs://nixos-images
 397747554  2020-05-03T02:36:23Z  gs://nixos-images/google-cloud-nixos-20.03.1639.73e73c7d6b5.raw.tar.gz
 256556736  2014-12-17T10:51:00Z  gs://nixos-images/nixos-14.10pre-git-x86_64-linux.raw.tar.gz
 291081495  2015-01-16T16:36:46Z  gs://nixos-images/nixos-14.12.323.91643074-x86_64-linux.raw.tar.gz
 290985235  2014-12-19T12:45:58Z  gs://nixos-images/nixos-14.12.542.4c9ef9f7-x86_64-linux.raw.tar.gz
TOTAL: 4 objects, 1236371020 bytes (1.15 GiB)

NOTE: Newer images (from 20.09 on) won't be available at the bucket above, and will instead need to be found at <nixpkgs/nixos/modules/virtualisation/gce-images.nix>.

Import an image into your project

1. In console.cloud.google.com/compute/images, select CREATE IMAGE:
   1. Name : nixos-20-03
   2. Source : Cloud Storage file
   3. Cloud Storage file : gs://nixos-images/google-cloud-nixos-20.03.1639.73e73c7d6b5.raw.tar.gz
2. Click Create

Create a VM instance

1. In console.cloud.google.com/compute/instances, select CREATE INSTANCE
   1. Boot disk : Custom images
      1. Image : nixos-20-03
   2. Important: do not add SSH keys, NixOS is set up for Google OS Login
   3. Metadata
      1. key : enable-oslogin
      2. value : TRUE
2. Click Create
3. Wait until your VM instance is ready
4. Under Connect, click SSH

Optional: add user account

Once you are logged into your NixOS machine, you can create a user account for yourself with administrator privileges:

1. chmod u+w /etc/nixos/configuration.nix

2. nano -w /etc/nixos/configuration.nix

3. Add the following to the configuration:

security.sudo.wheelNeedsPassword = false;
users.extraUsers.<your-username> = {
  createHome = true;
  home = "/home/<your-username>";
  description = "<your-name>";
  group = "users"; 
  extraGroups = [ "wheel" ];
  useDefaultShell = true;
  openssh.authorizedKeys.keys = [ "<contents of your ~/.ssh/id_rsa.pub>" ];
};

4. Save this file and run nixos-rebuild switch --upgrade

5. Reboot and log back in with your user account

Snapshots

At this point you may want to snapshot this image and use this snapshot to make future VMs.

Bootstrapping a NixOS image from the build of your choice

You normally can use a preexisting NixOS image, such as the ones found in gs://nixos-images and gs://nixos-cloud-images. Sometimes the images there haven't been updated in a long time, or you need a newer one to work around a bug. If you need to make a new one, here's how.

Perform these steps from any Linux machine.

You'll need setup the credentials for Google Cloud (via gsutil config or similar- if you have a GCE service account and have created an associated access key, the following is nice and quick: readlink -f ./relative/path/to/key | gsutil config -e, though it will prompt you for permission to lock down the file permissions on the key if they currently allow others to read the key- Note: the service account must be configured with a Role that allows it to write to your storage bucket), with the Cloud Storage and Compute Engine APIs enabled. You'll also need a Storage bucket. These steps will assume you've already made a bucket and it's named gs://example. Prepare a local copy of the nixpkgs repository in the state you want to build from. If you want to build a released version, this means checking out one of the release branches from the nixpkgs repository. Make sure you haven't left any unwanted local changes in it. These examples assume you've checked it out at /home/example/nixpkgs-clean.

$ BUCKET_NAME=example /home/example/nixpkgs-clean/nixos/maintainers/scripts/gce/create-gce.sh

This will create an image and upload it to the bucket example

Note: If you build an image from a commit later then this one, you will need to add enable-oslogin = "TRUE" to the instance metadata, to be able to login.