|
|
(18 intermediate revisions by 11 users not shown) |
Line 1: |
Line 1: |
| NOTE: as of March 2020 this page is not accurate and many changes have taken place
| | This is a recipe for creating a NixOS machine on Google Compute Engine (GCE) which is part of [https://cloud.google.com/ Google Cloud Platform]. |
| in NixOS and in GCE. There may be useful information here but following the instructions
| |
| will not lead to a happy running VM instance on GCE. Hopefully someone can figure out
| |
| how to fix it because there are updates in the source code. You may want to use [https://github.com/nix-community/nixos-generators nixos-generators] to create google cloud images.
| |
|
| |
|
| This is a recipe for creating a NixOS machine on Google Compute Engine (GCE) which is part of [https://cloud.google.com/ Google Cloud Platform]. This tutorial assumes you have already set up and account and project under Google Cloud Platform.
| | This tutorial assumes you have already set up and account and project under Google Cloud Platform. We also assume that you have [https://nixos.org/download nix-shell] and KVM virtualization support, the latter is not available in Google Cloud Shell. |
|
| |
|
| == Preparing a NixOS image ==
| | There are no publicly provided images of recent releases of NixOS. There are some old releases at [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/gce-images.nix <nixpkgs/nixos/modules/virtualisation/gce-images.nix>] and in the <code>gs://nixos-images</code> and <code>gs://nixos-cloud-images</code> public buckets, but these have not been updated in years. Instead, it is recommended you build your own image. |
| To bootstrap the process, first start any Linux instance and log into it.
| |
| # From the Google Developers Console select Compute - Compute Engine - VM instances
| |
| # Press "Create an Instance" or "New instance" button
| |
| # Click on "Show advanced options"
| |
| # Select a ZONE close to you and any MACHINE TYPE (f1-micro is adequate)
| |
| # Select any linux image such as centos-7
| |
| # Make sure "Enable Compute Engine service account." is checked
| |
| # Under COMPUTE select "Read Write"
| |
| # Press "Create" button and wait until the VM is ready
| |
| # From the Google Developers Console select Compute - Compute Engine - VM instances press the "SSH" button on your new instance (usually named "instance-1") to log in
| |
|
| |
|
| Next we will create a NixOS image. A source for NixOS images can be found the google storage bucket named [https://storage.cloud.google.com/nixos-images gs://nixos-images].
| | This guide is for people who really need NixOS... not just the Nix package manager. If Nix is all you need, you can install it automatically with a [https://cloud.google.com/compute/docs/instances/startup-scripts/linux startup script]. Debian 12 startup script example: |
| <syntaxhighlight lang="console">
| |
| $ gsutil ls -l gs://nixos-images
| |
| 256556736 2014-12-17T10:51:00Z gs://nixos-images/nixos-14.10pre-git-x86_64-linux.raw.tar.gz
| |
| 290985235 2014-12-19T12:45:58Z gs://nixos-images/nixos-14.12.542.4c9ef9f7-x86_64-linux.raw.tar.gz
| |
| </syntaxhighlight>
| |
|
| |
|
| You can find more up-to-date images at [https://storage.cloud.google.com/nixos-cloud-images gs://nixos-cloud-images], but the bucket itself is not public- <u>the objects inside it are</u>.
| | <syntaxhighlight lang="bash"> |
| | | #!/bin/bash |
| This means that <code>gsutil ls gs://nixos-cloud-images</code> will yield an <code>AccessDeniedException</code>, but the following works:
| | HOME="/root" sh <(curl -L https://nixos.org/nix/install) --daemon --yes |
| <syntaxhighlight lang="console">
| | source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh |
| $ gsutil ls -l gs://nixos-cloud-images/nixos-image-18.09.1228.a4c4cbb613c-x86_64-linux.raw.tar.gz
| | nix-env -i cowsay |
| 344473043 2018-11-16T09:16:51Z gs://nixos-cloud-images/nixos-image-18.09.1228.a4c4cbb613c-x86_64-linux.raw.tar.gz
| | cowsay 'nix is fully operational' |
| TOTAL: 1 objects, 344473043 bytes (328.52 MiB)
| |
| </syntaxhighlight> | | </syntaxhighlight> |
|
| |
|
| You can find a full list of available nixos-cloud-images at [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/gce-images.nix <nixpkgs/nixos/modules/virtualisation/gce-images.nix>].
| | == Bootstrapping a NixOS image from the build of your choice == |
|
| |
|
| In this tutorial we will use gs://nixos-images/nixos-14.12.542.4c9ef9f7-x86_64-linux.raw.tar.gz and follow the [https://cloud.google.com/compute/docs/images?_ga=1.122328651.1179090775.1417532639#creating_an_image_from_a_tar_file documentation on how to create an image from a .tar.gz file].
| | This assumes you have created a Google Cloud project and a Google Cloud Storage bucket that allows public access and uses Fine-grained access control (as opposed to Uniform) in that project. Set them as variables: |
|
| |
|
| <syntaxhighlight lang="console"> | | <syntaxhighlight lang="bash"> |
| $ gcloud compute images create nixos-14125424c9ef9f7-x86-64-linux --source-uri gs://nixos-images/nixos-14.12.542.4c9ef9f7-x86_64-linux.raw.tar.gz
| | PROJECT_ID=my-project-id |
| $ gcloud compute images describe nixos-14125424c9ef9f7-x86-64-linux
| | BUCKET_NAME=my-bucket-name # Set the bucket name without the gs:// prefix |
| ...
| |
| status: READY
| |
| </syntaxhighlight> | | </syntaxhighlight> |
|
| |
|
| You can now log out of your VM and delete the instance | | You'll need <code>gsutil</code> installed. See the [https://cloud.google.com/sdk/docs/install-sdk Google Cloud SDK documentation] for full instructions, or simply use: |
| # $ logout
| |
| # From the Google Developers Console select Compute - Compute Engine - VM instances check your instance and then press the "Delete" button at the top of the page.
| |
|
| |
|
| == Starting a NixOS image == | | <syntaxhighlight lang="bash"> |
| Now that the image is made we can start as many NixOS VMs as we like.
| | $ nix-shell -p google-cloud-sdk |
| | $ gcloud auth login |
| | [ ... this opens a webpage to authenticate your gcloud SDK, follow the authentication prompt in your browser] |
| | $ gcloud config set project $PROJECT_ID |
| | </syntaxhighlight> |
|
| |
|
| === (Strongly Recommended) Generate SSH host keys ===
| | Prepare a local copy of the nixpkgs repository in the state you want to build from. If you want to build a released version, this means checking out one of the release branches from the nixpkgs repository. Make sure you haven't left any unwanted local changes in it. These examples assume you've checked it out at <code>./nixpkgs</code>. |
| Before beginning we should generate unique SSH host keys for our new VM so that when will be able to authenticate our SSH connection. The example below generates keys <code>/dev/shm/ssh_host_ecdsa_key</code> and <code>/dev/shm/ssh_host_ecdsa_key.pub</code> files.
| |
|
| |
|
| <syntaxhighlight lang="bash"> | | <syntaxhighlight lang="bash"> |
| $ ssh-keygen -N '' -C '' -t ecdsa -f /dev/shm/ssh_host_ecdsa_key | | $ git clone --depth=1 --branch 23.11 https://github.com/NixOS/nixpkgs.git |
| Generating public/private ecdsa key pair.
| | $ BUCKET_NAME=my_bucket_name nixpkgs/nixos/maintainers/scripts/gce/create-gce.sh |
| Your identification has been saved in /dev/shm/ssh_host_ecdsa_key.
| |
| Your public key has been saved in /dev/shm/ssh_host_ecdsa_key.pub.
| |
| The key fingerprint is:
| |
| 92:2a:e9:28:1a:cd:43:71:31:36:f2:8e:6e:fa:13:c4
| |
| The key's randomart image is:
| |
| +--[ECDSA 256]---+
| |
| | . = |
| |
| | + + |
| |
| | .. o |
| |
| | E= . |
| |
| | .o . o S |
| |
| | =.. . . |
| |
| |. O.. |
| |
| |.*.o |
| |
| |*.o. |
| |
| +-----------------+
| |
| </syntaxhighlight> | | </syntaxhighlight> |
|
| |
|
| '''Take note of your unique key fingerprint.'''
| | This will create an image and upload it to the bucket. It will also create a GCE image that VMs can use. |
|
| |
|
| === Create your VM ===
| | Warning: this script makes the GCS object and the GCE image world-readable. If you are building from a custom configuration that embeds secrets, you should instead read the contents of the script and build and upload manually, setting your own ACLs. |
| Now we are ready to create a NixOS VM.
| |
| # From the Google Developers Console select Compute - Compute Engine - VM instances
| |
| # Press "Create an Instance" or "New instance" button
| |
| # Click on "Show advanced options"
| |
| # (Strongly Recommended) Under METADATA, create a Key called ssh_host_ecdsa_key and copy the contents of your <code>/dev/shm/ssh_host_ecdsa_key</code> to the corresponding Value if you generated SSH host keys.
| |
| # (Strongly Recommended) Under METADATA, create a Key called ssh_host_ecdsa_key_pub and copy the contents of your <code>/dev/shm/ssh_host_ecdsa_key.pub</code> to the corresponding Value if you generated SSH host keys.
| |
| # Under SSH KEYS, copy the contents of your desktop computer's <code>~/.ssh/id_rsa.pub</code> file into the box that says "Enter entire key data"
| |
| # Under BOOT SOURCE, make sure "New disk form image" is selected
| |
| # Under IMAGE, select your nixos image, e.g. "nixos-14125424c9ef9f7-x86-64-linux".
| |
| # Set any additional options as you see fit.
| |
| # Press the "Create" button and wait until the VM is ready
| |
|
| |
|
| === Connect to your VM ===
| | Note: If you build an image from a commit later then [https://github.com/NixOS/nixpkgs/commit/b894dd8b821d74b25911f63762c24024107d9372 this one], you will need to add <code>enable-oslogin = "TRUE"</code> to the instance metadata, to be able to login. |
| Notwithstanding the username associated with your SSH, you must log in as root to the NixOS machine. Also, you will need to ssh in from your computer's terminal. Below replace <code>XXX.XXX.XXX.XXX</code> with the EXTERNAL IP address listed for your VM instance. (Strongly Recommended) Verify the ECDSA key fingerprint is the same as the one you generated.
| |
|
| |
|
| <syntaxhighlight lang="bash">
| | == Create a VM instance == |
| $ ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=ask root@XXX.XXX.XXX.XXX
| | |
| The authenticity of host '130.211.149.218 (130.211.149.218)' can't be established.
| | #In [https://console.cloud.google.com/compute/instances the GCE console], select <code>CREATE INSTANCE</code> |
| ECDSA key fingerprint is 92:2a:e9:28:1a:cd:43:71:31:36:f2:8e:6e:fa:13:c4.
| | ## <b>Boot disk</b> : <i>Change</i>, then <i>Custom images</i> |
| Are you sure you want to continue connecting (yes/no)? yes
| | ### <b>Image</b> : pick the image recently created |
| Enter passphrase for key '.../.ssh/id_rsa':
| | ## You do not need to add SSH keys, NixOS is set up for [https://cloud.google.com/compute/docs/instances/managing-instance-access Google OS Login] |
| | ## <b>Metadata</b> |
| | ### <b>key</b> : <i>enable-oslogin</i> |
| | ### <b>value</b> : <i>"TRUE"</i> |
| | # Click <b>Create</b> |
| | # Wait until your VM instance is ready |
| | # Under <i>Connect</i>, click <b>SSH</b> |
| | |
| | == Optional: add user account == |
|
| |
|
| [root@instance-1:~]#
| | Once you are logged into your NixOS machine, you can create a user account for yourself with administrator privileges: |
| </syntaxhighlight>
| |
|
| |
|
| Once you are logged into your NixOS machine, I recommend that you create an user account for yourself with administrator privileges.
| | 1. chmod u+w /etc/nixos/configuration.nix |
|
| |
|
| # chmod u+w /etc/nixos/configuration.nix
| | 2. nano -w /etc/nixos/configuration.nix |
| # nano -w /etc/nixos/configuration.nix
| |
| #Add the follow to the configuration:
| |
|
| |
|
| | 3. Add the following to the configuration: |
| <syntaxhighlight lang="nix"> | | <syntaxhighlight lang="nix"> |
| security.sudo.wheelNeedsPassword = false; | | security.sudo.wheelNeedsPassword = false; |
Line 123: |
Line 81: |
| </syntaxhighlight> | | </syntaxhighlight> |
|
| |
|
| After you save this file run <code>nixos-rebuild switch --upgrade</code>. Once that is complete reboot and log back in with your user account. (Strongly Recommended) Again verify the ECDSA key fingerprint is the same as the one you generated. If you plan to keep this instance running for a long time you may removed the <code>-o UserKnownHostsFile=/dev/null</code> option to SSH to add the host to your <code>~/.ssh/known_hosts</code> file, however be advised that IP addresses for GCE instances are often reused, so when you create and destroy instances you may end up with stale keys in your known_hosts file.
| | 4. Save this file and run <code>nixos-rebuild switch --upgrade</code> |
| | |
| <syntaxhighlight lang="bash">
| |
| $ ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=ask <your-username>@XXX.XXX.XXX.XXX
| |
| The authenticity of host '130.211.149.218 (130.211.149.218)' can't be established.
| |
| ECDSA key fingerprint is 92:2a:e9:28:1a:cd:43:71:31:36:f2:8e:6e:fa:13:c4.
| |
| Are you sure you want to continue connecting (yes/no)? yes
| |
| Enter passphrase for key '.../.ssh/id_rsa':
| |
| | |
| [<your-username>@instance-1:~]$
| |
| </syntaxhighlight>
| |
|
| |
|
| At this point you may want to snapshot this image and use this snapshot to make future VMs. You should also delete the <code>/dev/shm/ssh_host_ecdsa_key</code> and <code>/dev/shm/ssh_host_ecdsa_key.pub</code> files from your home machine if you generated them.
| | 5. Reboot and log back in with your user account |
|
| |
|
| == Bootstrapping a NixOS image from the build of your choice == | | == Snapshots == |
| You normally can use a preexisting NixOS image, such as the ones found in gs://nixos-images and gs://nixos-cloud-images. Sometimes the images there haven't been updated in a long time, or you need a newer one to work around a bug. If you need to make a new one, here's how. Perform these steps from any Linux machine.
| |
|
| |
|
| You'll need setup the credentials for Google Cloud (via <code>gsutil config</code> or similar- if you have a GCE service account and have created an associated access key, the following is nice and quick: <code>readlink -f ./relative/path/to/key | gsutil config -e</code>, though it will prompt you for permission to lock down the file permissions on the key if they currently allow others to read the key- Note: the service account must be configured with a Role that allows it to write to your storage bucket), with the Cloud Storage and Compute Engine APIs enabled. You'll also need a Storage bucket. These steps will assume you've already made a bucket and it's named <code>gs://example</code>.
| | At this point you may want to snapshot this image and use this snapshot to make future VMs. |
| Prepare a local copy of the nixpkgs repository in the state you want to build from. If you want to build a released version, this means checking out one of the release branches from the nixpkgs-channels repository. Make sure you haven't left any unwanted local changes in it. These examples assume you've checked it out at <code>/home/example/nixpkgs-clean</code>.
| |
|
| |
|
| <syntaxhighlight lang="bash">
| | [[Category:Server]][[Category:Cookbook]] |
| $ BUCKET_NAME=example /home/example/nixpkgs-clean/nixos/maintainers/scripts/gce/create-gce.sh
| | [[Category:Deployment]] |
| </syntaxhighlight>
| |
| | |
| This will create an image and upload it to the bucket <code>example</code>
| |
| | |
| Note: If you build an image from a commit later then [https://github.com/NixOS/nixpkgs/commit/b894dd8b821d74b25911f63762c24024107d9372 this one], you will need to add <code>enable-oslogin = "TRUE"</code> to the instance metadata, to be able to login.
| |
This is a recipe for creating a NixOS machine on Google Compute Engine (GCE) which is part of Google Cloud Platform.
This tutorial assumes you have already set up and account and project under Google Cloud Platform. We also assume that you have nix-shell and KVM virtualization support, the latter is not available in Google Cloud Shell.
There are no publicly provided images of recent releases of NixOS. There are some old releases at <nixpkgs/nixos/modules/virtualisation/gce-images.nix> and in the gs://nixos-images
and gs://nixos-cloud-images
public buckets, but these have not been updated in years. Instead, it is recommended you build your own image.
This guide is for people who really need NixOS... not just the Nix package manager. If Nix is all you need, you can install it automatically with a startup script. Debian 12 startup script example:
#!/bin/bash
HOME="/root" sh <(curl -L https://nixos.org/nix/install) --daemon --yes
source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
nix-env -i cowsay
cowsay 'nix is fully operational'
Bootstrapping a NixOS image from the build of your choice
This assumes you have created a Google Cloud project and a Google Cloud Storage bucket that allows public access and uses Fine-grained access control (as opposed to Uniform) in that project. Set them as variables:
PROJECT_ID=my-project-id
BUCKET_NAME=my-bucket-name # Set the bucket name without the gs:// prefix
You'll need gsutil
installed. See the Google Cloud SDK documentation for full instructions, or simply use:
$ nix-shell -p google-cloud-sdk
$ gcloud auth login
[ ... this opens a webpage to authenticate your gcloud SDK, follow the authentication prompt in your browser]
$ gcloud config set project $PROJECT_ID
Prepare a local copy of the nixpkgs repository in the state you want to build from. If you want to build a released version, this means checking out one of the release branches from the nixpkgs repository. Make sure you haven't left any unwanted local changes in it. These examples assume you've checked it out at ./nixpkgs
.
$ git clone --depth=1 --branch 23.11 https://github.com/NixOS/nixpkgs.git
$ BUCKET_NAME=my_bucket_name nixpkgs/nixos/maintainers/scripts/gce/create-gce.sh
This will create an image and upload it to the bucket. It will also create a GCE image that VMs can use.
Warning: this script makes the GCS object and the GCE image world-readable. If you are building from a custom configuration that embeds secrets, you should instead read the contents of the script and build and upload manually, setting your own ACLs.
Note: If you build an image from a commit later then this one, you will need to add enable-oslogin = "TRUE"
to the instance metadata, to be able to login.
Create a VM instance
- In the GCE console, select
CREATE INSTANCE
- Boot disk : Change, then Custom images
- Image : pick the image recently created
- You do not need to add SSH keys, NixOS is set up for Google OS Login
- Metadata
- key : enable-oslogin
- value : "TRUE"
- Click Create
- Wait until your VM instance is ready
- Under Connect, click SSH
Optional: add user account
Once you are logged into your NixOS machine, you can create a user account for yourself with administrator privileges:
1. chmod u+w /etc/nixos/configuration.nix
2. nano -w /etc/nixos/configuration.nix
3. Add the following to the configuration:
security.sudo.wheelNeedsPassword = false;
users.extraUsers.<your-username> = {
createHome = true;
home = "/home/<your-username>";
description = "<your-name>";
group = "users";
extraGroups = [ "wheel" ];
useDefaultShell = true;
openssh.authorizedKeys.keys = [ "<contents of your ~/.ssh/id_rsa.pub>" ];
};
4. Save this file and run nixos-rebuild switch --upgrade
5. Reboot and log back in with your user account
Snapshots
At this point you may want to snapshot this image and use this snapshot to make future VMs.