Install NixOS on Amazon EC2

This article has some notes on installing NixOS on Amazon's Elastic Compute Cloud (EC2). EC2 support is work in progress.

Below we assume that the following environment variables are set:

• $EC2_CERT: path to your AWS X.509 certificate (PEM file).
• $EC2_PRIVATE_KEY: path to the corresponding private key (PEM file).
• $AWS_ACCOUNT: AWS account number (e.g. 1234-5678-9012).
• $AWS_ACCESS_KEY_ID: AWS access key ID (e.g. AKIAJM...).
• $AWS_SECRET_ACCESS_KEY: corresponding secret access key (e.g. klG5...).
• $AWS_CALLING_FORMAT may need to be set to SUBDOMAIN.
• $EC2_URL should be set to the desired region, e.g. https://ec2.eu-west-1.amazonaws.com/.

Public NixOS AMIs

The list of current NixOS AMI's are available at https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/ec2-amis.nix

Running NixOS instances

1. To create instances from the command line, you need to install the EC2 API tools:

   $ nix-env -i ec2-api-tools
   

   Alternatively, you can create NixOS instances using the AWS Management Console.

2. Start a small instance running NixOS:

   $ ec2-run-instances -k gsg-keypair ami-c9f2d8bd
   RESERVATION  r-10ca4167  516444698777  default
   INSTANCE     i-f6d3b781  ami-c9f2d8bd  pending gsg-keypair  0 m1.small ...
   

   gsg-keypair denotes an SSH key pair created with ec2-add-keypair. To run a 64-bit instance, you must specify a 64-bit-capable instance type, e.g., -t m1.large. If your requirements allow it, you may prefer using a cheap spot instance:

   $ ec2-request-spot-instances -t m1.large -k gsg-keypair -p 0.3 ami-ecb49e98
   

3. After a while the instance should be running, and you can log in using the SSH key generated by ec2-add-keypair:

   $ ssh -i id_rsa-gsg-keypair root@ec2-79-125-97-89.eu-west-1.compute.amazonaws.com
   

   To get the IP address / hostname of the instance, use ec2-describe-instances. You may need to allow traffic to port 22 (ssh):

   $ ec2-authorize default -p 22 -s 0.0.0.0/0
   

4. To start working with the instance, you may want to do the following to obtain the Nixpkgs and NixOS sources:

   $ nixos-checkout
   $ nixos-rebuild pull
   

   You should now be able to install software, e.g.

   $ nix-env -i emacs
   

   or reconfigure the instance:

   $ nano /etc/nixos/configuration.nix
   $ nixos-rebuild switch
   

   Note: if you're using an ec2 instance based on the HVM virtualization type, you will need to set the

   ec2.hvm = true;
   

   setting in configuration.nix; otherwise GRUB won't pick up changes to your configuration across reboots.

Creating a NixOS AMI

The following steps show how to build your own NixOS Amazon Image (AMI), upload it to S3, and start an instance.

1. Install the EC2 AMI tools:

   $ nix-env -i ec2-ami-tools
   

2. The easy way to build NixOS AMIs is to run the following command:

   $ NIXOS=/path/to/nixos NIXPKGS=/path/to/nixpkgs /path/to/nixos/maintainers/scripts/create-amis.sh
   

   This will build, bundle and upload AMIs to several EC2 regions. Note that the resulting AMIs will be public.

   Alternatively, you can perform the steps below to create the AMI manually.

3. Build the raw disk image:

   $ NIXPKGS=/path/to/nixpkgs NIXOS_CONFIG=/path/to/nixos/modules/virtualisation/amazon-config.nix \
       nix-build /path/to/nixos -A config.system.build.amazonImage \
       --argstr system i686-linux
   

   (Use --argstr system x86_64-linux to build a 64-bit image.) This will produce an ext3 disk image in ./result/nixos.img:

   $ ls -l ./result/nixos.img 
   -r--r--r-- 1 root nixbld 4294967296 Jan  1  1970 ./result/nixos.img
   

4. Bundle the image:

   $ ec2-bundle-image -i ./result/nixos.img --user $AWS_ACCOUNT --arch i386 \
       -c $EC2_CERT -k $EC2_PRIVATE_KEY \
       --kernel aki-c34d67b7
   Bundling image file...
   Splitting /tmp/nixos.img.tar.gz.enc...
   Created nixos.img.part.00
   ...
   Created nixos.img.part.19
   Generating digests for each part...
   Digests generated.
   Creating bundle manifest...
   ec2-bundle-image complete.
   

   The resulting bundle is stored in /tmp/nixos.img.manifest.xml and /tmp/nixos.img.part.*. Use --arch x86_64 for 64-bit images. For the kernel, you should use pv-grub-hd0_1.02-arch (use ec2-describe-images -a to find the AMI ID). This is the PV-GRUB "kernel" that boots the native NixOS kernel in the image.

5. Upload the bundle:

   $ ec2-upload-bundle -b nixos-img -m /tmp/nixos.img.manifest.xml \
       -a $AWS_ACCESS_KEY_ID -s $AWS_SECRET_ACCESS_KEY --location EU
   Creating bucket...
   Uploading bundled image parts to the S3 bucket nixos-img ...
   Uploaded nixos.img.part.00
   Uploaded nixos.img.part.01
   ...
   Uploaded nixos.img.part.19
   Uploading manifest ...
   Uploaded manifest.
   Bundle upload completed.
   

   (Of course you can use any other bucket location as well.)

6. Register the image as a private AMI:

   $ ec2-register nixos-img/nixos.img.manifest.xml -n nixos-img -d '...description...'
   IMAGE   ami-efffd59b
   

   To make it public:

   $ ec2-modify-image-attribute ami-efffd59b -l -a all