Cheatsheet: Difference between revisions

From NixOS Wiki
imported>Mic92
No edit summary
imported>Mic92
No edit summary
Line 260: Line 260:
== Get the store path for a package ==
== Get the store path for a package ==


<source lang="javascript">
<source lang="nix">
$ nix-repl
$ nix-repl
nix-repl> :l <nixpkgs>
nix-repl> :l <nixpkgs>
Line 269: Line 269:
$ nix-build '<nixpkgs>' --no-build-output -A xorg.libXtst
$ nix-build '<nixpkgs>' --no-build-output -A xorg.libXtst
/nix/store/nlpnx21yjdjx2ii7ln4kcmbm0x1vy7w9-libXtst-1.2.3
/nix/store/nlpnx21yjdjx2ii7ln4kcmbm0x1vy7w9-libXtst-1.2.3
</source>
### Adding files to the store
It is sometimes necessary to add files to the store manually.
This is particularly the case with packages that cannot be downloaded automatically,
for example, proprietary software packages.
For most files, it is sufficient to run:
<source lang="bash">
$ nix-store --add-fixed sha256 /path/to/file
</source>
Unfortunately, `nix-store` will try to load the entire file into memory,
which will fail if the file size exceeds available memory.
If we have root access, we can copy the file to the store ourselves:
<source lang="bash">
$ sudo unshare -m bash  # open a shell as root in a private mount namespace
$ largefile=/path/to/file
$ hash=$(nix-hash --type sha256 --flat --base32 $largefile)  # sha256 hash of the file
$ storepath=$(nix-store --print-fixed-path sha256 $hash $(basename $largefile))  # destination path in the store
$ mount -o remount,rw /nix/store  # remount the store in read/write mode (only for this session)
$ cp $largefile $storepath  # copy the file
$ printf "$storepath\n\n0\n" | nix-store --register-validity --reregister  # register the file in the Nix database
$ exit  # exit to the original shell where /nix/store is still mounted read-only
</source>
== Build nixos from nixpkgs repo ==
The following snippet will build the system from a git checkout:
<source lang="bash">
$ nixos-rebuild -I nixpkgs=/path/to/nixpkgs switch
</source>
This method can be used when testing nixos services for a pull request to nixpkgs.
Building nixos from a git is an alternative to using nix channels and set up permanent following this [blog article](http://anderspapitto.com/posts/2015-11-01-nixos-with-local-nixpkgs-checkout.html).
It has a couple of advantages over nixpkgs as it allows back-porting of packages/changes to stable versions
as well as applying customization.
Use the following command to build directly from a particular branch of a repo in github:
<source lang="bash">
$ nixos-rebuild -I nixpkgs=https://github.com/nixcloud/nixpkgs/archive/release-17.03.tar.gz switch
</source>
## Building a service as a VM (for testing)
While `nixos-rebuild build-vm` allows to build a vm out of the current system configuration, there is a more light-weight alternative when only a single service needs to be tested.
Given the following configuration:
<source lang="nix">
# vm.nix
{ lib, config, ... }:                                                                                                 
{                                                                                                                                     
  services.tor.enable = true;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
  users.users.root.initialPassword = "root";                                                                                         
}
</source>
a vm can be build using the following command:
<source lang="bash">
$ nixos-rebuild -I nixpkgs=/path/to/nixpkgs -I nixos-config=./vm.nix build-vm
</source>
where `-I nixpkgs=/path/to/nixpkgs` is optionally depending whether the vm should be build from git checkout or a channel.
On non-nixos (linux) systems the following command can be used instead:
<source lang="bash">
nix-build '<nixpkgs/nixos>' -A vm -k -I nixos-config=./vm.nix
</source>
== Reuse a package as a build environment ==
As packages already contains all build dependencies, they can be reused to a build environment quickly.
In the following a setup for the cmake-based project [bcc](https://github.com/iovisor/bcc) is shown.
After obtaining the source:
<source lang="bash">
$ git clone https://github.com/iovisor/bcc.git
$ cd bcc
</source>
Add the following `default.nix` to the project:
<source lang="nix">
with import <nixpkgs> {};
linuxPackages.bcc.overrideDerivation (old: {
  # overrideDerivation allows it to specify additional dependencies
  buildInputs = [ bashInteractive ninja ] ++ old.buildInputs;
})
</source>
To initiate the build environment run `nix-shell` in the project root directory
<source lang="bash">
# this will download add development dependencies and set up the environment so build tools will find them.
$ nix-shell
</source>
The following is specific to bcc or cmake in general:
(so you need to adapt the workflow depending on the project, you hack on)
<source lang="bash">
$ mkdir build
$ cd build
# cmakeFlags is also defined in the bcc package. autotools based projects might defined $configureFlags
$ eval cmake $cmakeFlags ..
$ make
</source>
== Customizing Packages ==
=== Upgrading individual packages to a different channel ===
One can track multiple channels on NixOS simultaneously, and then declaratively change packages from the default channel to another one.
For example one can have both the unstable and stable channels on system root:
<source lang="nix">
$ sudo nix-channel --list
nixos https://nixos.org/channels/nixos-17.03
nixos-unstable https://nixos.org/channels/nixos-unstable
</source>
and the following in `configuration.nix`:
<source lang="nix">
nixpkgs.config = {
  # Allow proprietary packages
  allowUnfree = true;
  # Create an alias for the unstable channel
  packageOverrides = pkgs: {
    unstable = import <nixos-unstable> {
      # pass the nixpkgs config to the unstable alias
      # to ensure `allowUnfree = true;` is propagated:
      config = config.nixpkgs.config;
    };
  };
};
</source>
which allows you to switch particular packages to the unstable channel:
<source lang="nix">
environment = {
  systemPackages = with pkgs; [
    ddate
    devilspie2
    evince
    unstable.google-chrome
    # ...
    zsh
  ];
};
</source>
== Building statically linked packages ==
<source lang="bash">
$ nix-build -E 'with (import ./. {}); (curl.override { stdenv = makeStaticLibraries stdenv;}).out'
</source>
== Rebuild a package with debug symbols ==
<source lang="bash">
$ nix-build -E 'with import <nixpkgs> {}; enableDebugging st'
$ file result/bin/st
result/bin/st: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/f111ij1fc83965m48bf2zqgiaq88fqv5-glibc-2.25/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped, with debug_info
</source>
</source>

Revision as of 16:25, 22 August 2017

A cheat sheet and rough mapping between Ubuntu and NixOS

This is meant to give you basic ideas and get you unstuck. NixOS being very different from most distributions, a deeper understanding will be necessary sooner or later! Follow the links to the manual pages and browse the wiki to find real NixOS tutorials.

The system-wide column is the equivalent of using apt under Ubuntu.

TODO Provide well-commented sample configuration.nix and ~/.nixpkgs/config.nix files with examples of common tasks.

Task Ubuntu NixOS (system-wide and root) NixOS (user) and Nix in general
Basic concepts
This column will let you do everything you can with Ubuntu and more. This column just isn't possible in Ubuntu.
Who can install packages and who can run them? All packages are always system-wide and only root can install packages. Packages root installs are system-wide. It does so through through /etc/nixos/configuration.nix. If root installs packages the same way users do, through ~/.nixpkgs/config.nix, they are also global. Root's default profile is the system-wide default profile. Users can install their own packages and have their own profiles (environments) through ~/.nixpkgs/config.nix
Package manager apt which is really running on top of dpkg, sometimes wrapped by UIs like aptitude. nix, but many system-wide operations are provided by nixos packages. Just nix without the involvement of nixos.
How do you select your official sources and major releases These are baked into the distribution (e.g. Ubuntu version X). Upgrades are hard and permanent. At any time you select from a collection of channels. They're system-wide when set by root. You can roll back changes or switch channels with ease. Channels are per-user if they're not set by root.
Where are packages installed? apt installs globally into /bin/, /usr/, etc. System-wide packages are in /run/current-system/sw/ (these are installed because of /etc/nixos/configuration.nix) and /nix/var/nix/profiles/default/bin/ (this is the profile managed by root). Note that the files are just symlinks to the real packages managed by nix /nix/store/. User packages are in ~/.nix-profile/. Note that the files are just symlinks to the real packages managed by nix in /nix/store/.
When changes take effect As soon as the command runs. Commands are not atomic and can leave your machine in a bad state. Most of the time you modify the configuration file and apply changes with nixos-rebuild switch

TODO How does one get nixos to do all the work for a switch and separate out the actual switching from fetching/building?

Most of the time you apply changes with nix-env -i all

TODO How does one get nix to do all the work for a switch and separate out the actual switching from fetching/building?

Packages Uniformly referred to as packages Technically called "derivations" but everyone calls them packages. Technically called "derivations" but everyone calls them packages.
Package management
Install a package
sudo apt-get install emacs
In /etc/nixos/configuration.nix:

If it's a program add to systemPackages:

 systemPackages = with pkgs; 
                    [ <other packages...> emacs ];

If it's a service add:

services.openssh.enable = true;
nix-env -i emacs

Or with collections, add the package to your ~/.nixpkgs/config.nix and run

nix-env -i all
Uninstall a package
sudo apt-get remove emacs
remove from /etc/nixos/configuration.nix
sudo nixos-rebuild switch
Uninstall a package removing its configuration
apt-get purge emacs
All configuration is in configuration.nix
Update the list of packages
sudo apt-get update
sudo nix-channel --update
nix-channel --update
Upgrade packages
sudo apt-get upgrade
sudo nixos-rebuild switch
nix-env -u
Check for broken dependencies
sudo apt-get check
nix-store --verify --check-contents
unneeded!
List package dependencies
apt-cache depends emacs
nix-store --query --requisites $(readlink -f /run/current-system)
nix-store -q --tree /nix/var/nix/profiles/system
nix-store --query --references\
  $(nix-instantiate '<nixpkgs>' -A emacs)

For installed packages:

nix-store --query --references $(which emacs)
List which packages depend on this one (reverse dependencies)
apt-cache rdepends emacs
For installed packages (only print reverse dependencies *which are already installed*):
nix-store --query --referrers $(which emacs)
Verify all installed packages
debsums
sudo nix-store --verify --check-contents
nix-store --verify --check-contents
Fix packages with failed checksums Reinstall broken packages
sudo nix-store --verify --check-contents --repair
nix-store --verify --check-contents --repair
Select major version and stable/unstable Change sources.list and apt-get dist-upgrade. A an extremely infrequent and destructive operation. The nix variants are safe and easy to use.
nix-channel --add\
   https://nixos.org/channels/nixpkgs-unstable <name>

Add the unstable channel. At that address you will find names for other versions and variants. Name can be any string.

nix-channel --remove <name>

To eliminate a channel.

nix-channel --list

To show all installed channel.

When run by a user channels work locally, when run by root they're used as the system-wide channels.
Private package repository PPA Define your package tree as in the general column, and include it in configuration.nix, then list your packages in systemPackages to make them available system wide See [1]
Install a particular version of a package
Package configuration
Configure a package
sudo dpkg-reconfigure <package>
edit /etc/nixos/configuration.nix edit ~/.nixpkgs/config.nix TODO More details about how to edit
List package options
Global package configuration Modify configuration file in /etc/
Package configuration
Find packages
apt-cache search emacs
nix-env -qaP '.*emacs.*'
nix-env -qaP '.*emacs.*'
Show package description
apt-cache show emacs
nix-env -qa --description '.*emacs.*'
nix-env -qa --description '.*emacs.*'
Show files installed by package
dpkg -L emacs
readlink -f $(which emacs)
 /nix/store/ji06y4haijly0i0knmr986l2dajffv1p-emacs-24.4/bin/emacs-24.4

then

du -a /nix/store/ji06y4haijly0i0knmr986l2dajffv1p-emacs-24.4

Show package for file
dpkg -S /usr/bin/emacs
follow the symlink follow the symlink
Services
Start a service
sudo service apache start
sudo systemctl start apache
Stop a service
sudo service apache stop
sudo systemctl stop apache
Where your log files live /var/log/ System-wide packages /var/log/ User packages ~/.nix-profile/var/log/
Adding a user sudo adduser alice Add
users.extraUsers.alice =
 { isNormalUser = true;
   home = "/home/alice";
   description = "Alice Foobar";
   extraGroups = [ "wheel" "networkmanager" ];
   openssh.authorizedKeys.keys =
      [ "ssh-dss AAAAB3Nza... alice@foobar" ];
 };
to to /etc/nixos/configuration.nix and then call
nixos-rebuild switch
Misc tasks
List binaries
ls /usr/bin/
ls /run/current-system/sw/bin &&\
ls /nix/var/nix/profiles/default/bin/
ls ~/.nix-profile/bin
Get the current version number
cat /etc/debian_version
nixos-version
nixos-version
Get sources for a package
apt-get source emacs
In Debian, apt-get source gets both the patched upstream source and the recipe for the package. Those need two steps in Nix. To find the package recipe:
grep -r emacs $(nix-instantiate --eval --expr '<nixpkgs>')
To download the source as specified by the package recipe:
nix-build '<nixpkgs>' -A emacs.src
The patched source is usually not a derivation itself, but can be produced for most packages with the following command:
nix-shell '<nixpkgs>' -A emacs\
   --command 'unpackPhase; patchPhase'
Compile & install a package from source
git clone foobar
cat >default.nix <<EOF
with import <nixpkgs> { };
stdenv.lib.overrideDerivation foobar (oldAttrs : {
  src = ./foobar;
})
EOF
nix-build
Install a binary package
Install a .deb
dpkg -i package.deb
Install dpkg with Nix, then
dpkg -i package.deb

Working with the nix store

Get the store path for a package

$ nix-repl
nix-repl> :l <nixpkgs>
Added 7486 variables.
nix-repl> "${xorg.libXtst}"                                                                                                 
"/nix/store/nlpnx21yjdjx2ii7ln4kcmbm0x1vy7w9-libXtst-1.2.3"

$ nix-build '<nixpkgs>' --no-build-output -A xorg.libXtst
/nix/store/nlpnx21yjdjx2ii7ln4kcmbm0x1vy7w9-libXtst-1.2.3


      1. Adding files to the store

It is sometimes necessary to add files to the store manually. This is particularly the case with packages that cannot be downloaded automatically, for example, proprietary software packages. For most files, it is sufficient to run:

$ nix-store --add-fixed sha256 /path/to/file

Unfortunately, `nix-store` will try to load the entire file into memory, which will fail if the file size exceeds available memory. If we have root access, we can copy the file to the store ourselves:

$ sudo unshare -m bash  # open a shell as root in a private mount namespace
$ largefile=/path/to/file
$ hash=$(nix-hash --type sha256 --flat --base32 $largefile)  # sha256 hash of the file
$ storepath=$(nix-store --print-fixed-path sha256 $hash $(basename $largefile))  # destination path in the store
$ mount -o remount,rw /nix/store  # remount the store in read/write mode (only for this session)
$ cp $largefile $storepath  # copy the file
$ printf "$storepath\n\n0\n" | nix-store --register-validity --reregister  # register the file in the Nix database
$ exit  # exit to the original shell where /nix/store is still mounted read-only

Build nixos from nixpkgs repo

The following snippet will build the system from a git checkout:

$ nixos-rebuild -I nixpkgs=/path/to/nixpkgs switch

This method can be used when testing nixos services for a pull request to nixpkgs.

Building nixos from a git is an alternative to using nix channels and set up permanent following this [blog article](http://anderspapitto.com/posts/2015-11-01-nixos-with-local-nixpkgs-checkout.html). It has a couple of advantages over nixpkgs as it allows back-porting of packages/changes to stable versions as well as applying customization.

Use the following command to build directly from a particular branch of a repo in github:

$ nixos-rebuild -I nixpkgs=https://github.com/nixcloud/nixpkgs/archive/release-17.03.tar.gz switch
    1. Building a service as a VM (for testing)

While `nixos-rebuild build-vm` allows to build a vm out of the current system configuration, there is a more light-weight alternative when only a single service needs to be tested.

Given the following configuration:

# vm.nix
{ lib, config, ... }:                                                                                                  
{                                                                                                                                      
  services.tor.enable = true;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
  users.users.root.initialPassword = "root";                                                                                           
}

a vm can be build using the following command:

$ nixos-rebuild -I nixpkgs=/path/to/nixpkgs -I nixos-config=./vm.nix build-vm

where `-I nixpkgs=/path/to/nixpkgs` is optionally depending whether the vm should be build from git checkout or a channel.

On non-nixos (linux) systems the following command can be used instead:

nix-build '<nixpkgs/nixos>' -A vm -k -I nixos-config=./vm.nix

Reuse a package as a build environment

As packages already contains all build dependencies, they can be reused to a build environment quickly. In the following a setup for the cmake-based project [bcc](https://github.com/iovisor/bcc) is shown. After obtaining the source:

$ git clone https://github.com/iovisor/bcc.git
$ cd bcc

Add the following `default.nix` to the project:

with import <nixpkgs> {};
linuxPackages.bcc.overrideDerivation (old: {
  # overrideDerivation allows it to specify additional dependencies
  buildInputs = [ bashInteractive ninja ] ++ old.buildInputs;
})

To initiate the build environment run `nix-shell` in the project root directory

# this will download add development dependencies and set up the environment so build tools will find them.
$ nix-shell

The following is specific to bcc or cmake in general: (so you need to adapt the workflow depending on the project, you hack on)

$ mkdir build
$ cd build
# cmakeFlags is also defined in the bcc package. autotools based projects might defined $configureFlags
$ eval cmake $cmakeFlags ..
$ make

Customizing Packages

Upgrading individual packages to a different channel

One can track multiple channels on NixOS simultaneously, and then declaratively change packages from the default channel to another one.

For example one can have both the unstable and stable channels on system root:

$ sudo nix-channel --list
nixos https://nixos.org/channels/nixos-17.03
nixos-unstable https://nixos.org/channels/nixos-unstable

and the following in `configuration.nix`:

nixpkgs.config = {
  # Allow proprietary packages
  allowUnfree = true;

  # Create an alias for the unstable channel
  packageOverrides = pkgs: {
    unstable = import <nixos-unstable> { 
      # pass the nixpkgs config to the unstable alias
      # to ensure `allowUnfree = true;` is propagated:
      config = config.nixpkgs.config; 
    };
  };
};

which allows you to switch particular packages to the unstable channel:

environment = {
  systemPackages = with pkgs; [
    ddate
    devilspie2
    evince
    unstable.google-chrome
    # ...
    zsh
  ]; 
};

Building statically linked packages

$ nix-build -E 'with (import ./. {}); (curl.override { stdenv = makeStaticLibraries stdenv;}).out'

Rebuild a package with debug symbols

$ nix-build -E 'with import <nixpkgs> {}; enableDebugging st'
$ file result/bin/st
result/bin/st: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/f111ij1fc83965m48bf2zqgiaq88fqv5-glibc-2.25/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped, with debug_info