NixOS Wiki

Nixpkgs/Create and debug packages: Difference between revisions

← Older editNewer edit →
VisualWikitext
imported>Hypnosis2839
m →Packages from source code: minor cleanup, fix command name
imported>Hypnosis2839
→Packages from source code: Remove references to nix3 CLI, fix instructions for phase-by-phase builds, clean up tone, remove repetitive phrasing
Line 28: Line 28:
There are different steps here depending on whether you're building from source or packaging an existing binary.  There are some common steps too.
There are different steps here depending on whether you're building from source or packaging an existing binary.  There are some common steps too.


=== Packages from source code ===
=== Package from source code ===


# Read the repo build instructions. Many times the instructions won't be 100% clear in the README, but there might be some script for CI that will be more precise. This should give you a rough idea of how hard it will be to package it. For example, if it uses a standard, vanilla build (<code>./configure && make && make install</code>), it will probably be easy to package. If on the contrary there are no CI scripts, or no build instructions, or the build instructions are complicated, things could be rough.  
# Read the repo build instructions and CI scripts (for example, on GitHub, these are located in <code>.github/workflows</code>).
# Look in nixpkgs for a package with a similar build process to use as reference. Many times you can just copy/paste a similar package, change the <code>src</code> attribute and things will just work. For example, if you're packaging a project written in Go, find a package for an existing Go application. Each language has its own supporting Nix functions and a more or less standard way of dealing with things. For example, [https://nixos.org/manual/nixpkgs/stable/#sec-language-go Go] has <code>buildGoModule</code>. [https://nixos.org/manual/nixpkgs/stable/#rust Rust] has <code>buildRustPackage</code>. [https://nixos.org/manual/nixpkgs/stable/#python Python] has <code>buildPythonApplication</code>. [https://nixos.org/manual/nixpkgs/stable/#node.js Node.js] has <code>node2nix</code>, <code>yarn2nix</code>, etc. There are also specific functions for wrapping e.g. [https://nixos.org/manual/nixpkgs/stable/#sec-language-gnome GNOME] applications (<code>wrapGAppsHook</code>), or [https://nixos.org/manual/nixpkgs/stable/#sec-language-qt Qt] apps (<code>libsForQt5</code>, <code>wrapQtAppsHook</code>). It pays off to have a look at these functions in the [https://nixos.org/manual/nixpkgs/stable/#chap-language-support language-support table in the NixOS manual].  
# Look in nixpkgs for a package with a similar build process to use as reference. For example, if you're packaging a project written in Go, find a package for an existing Go application. Each language has its own supporting Nix functions and a more or less standard way of dealing with things. For example, [https://nixos.org/manual/nixpkgs/stable/#sec-language-go Go] has <code>buildGoModule</code>. [https://nixos.org/manual/nixpkgs/stable/#rust Rust] has <code>buildRustPackage</code>. [https://nixos.org/manual/nixpkgs/stable/#python Python] has <code>buildPythonApplication</code>. [https://nixos.org/manual/nixpkgs/stable/#node.js Node.js] has <code>node2nix</code>, <code>yarn2nix</code>, etc. There are also specific functions for wrapping e.g. [https://nixos.org/manual/nixpkgs/stable/#sec-language-gnome GNOME] applications (<code>wrapGAppsHook</code>), or [https://nixos.org/manual/nixpkgs/stable/#sec-language-qt Qt] apps (<code>libsForQt5</code>, <code>wrapQtAppsHook</code>). Refer to the [https://nixos.org/manual/nixpkgs/stable/#chap-language-support language support chapter in the nixpkgs manual].  
# If there isn't a specific function for the build/language you'll need to use [https://nixos.org/guides/nix-pills/fundamentals-of-stdenv.html#idm140737319559808 stdenv.mkDerivation] directly but don't despair (yet!). mkDerivation has built-in support for GNU make, CMake, and possibly others too (?) so you probably won't have to spell out every single command necessary to build the project.
# If there isn't a specific builder for the language, use <code>stdenv.mkDerivation</code> directly, which has built-in support for GNU make (and other build systems, provided you add the necessary <code>nativeBuildInputs</code>).
# Figure out at least some dependencies from the project repo. Doesn't have to be perfect, but something to get you started. See if they're available in nixpkgs (<code>nix search some-library</code> or <code>nix-locate --top-level lib/somelibrary.so</code>). If any dependency is missing you'll need to package that as well, but common libraries are usually available.
# Figure out at least some dependencies from the project repo. See if they're available in nixpkgs (<code>nix search some-library</code> or <code>nix-locate --top-level lib/somelibrary.so</code>). If any dependency is missing you'll need to package that as well.
# Prototype / iterate the build with nix-shell. Run <code>nix-shell -p dep1 dep2 dep3...</code> then run the build manually on the command line. Hopefully this will let you quickly identify missing dependencies and maybe some build quirks. Again this doesn't need to be perfect, don't spend too much time on this.
# Create your derivation in <code>default.nix</code> in some empty local directory.
# Write the default.nix file, save it in some empty local directory.
# At the top of the derivation, temporarily add <code>with import <nixpkgs> {};</code>. For now, don't worry too much about declaring every dependency as a parameter<!-- TODO clarify. "parameter of the default build function"? -->, to save time.
# At the top of default.nix add <code>with import <nixpkgs> {};</code>. For now, don't worry too much about declaring every dependency as a parameter<!-- TODO clarify. "parameter of the default build function"? -->, as it will only slow you down. Again, use some similar package as reference.
# Build the package with <code>nix-build</code>. Iterate on tweaking the derivation and rebuilding until it succeeds.
# Build the package with <code>nix build -L</code>. Iterate on tweaking default.nix and running nix build until it builds.
# For large projects with long compile times, you can use [[Development environment with nix-shell|nix-shell]] with <code>nix-shell</code>. Inside the development shell, you can [https://ryantm.github.io/nixpkgs/stdenv/stdenv/#sec-building-stdenv-package-in-nix-shell run the individual phases].
# Instead of nix build, for large projects with long compile times, you can use [[Development environment with nix-shell|nix-shell]] with <code>nix develop -L</code> (same arguments as nix build). Inside the nix-shell, run something like <code>cd $(mktemp -d) && unpackPhase && cd * && configurePhase && buildPhase && checkPhase && installPhase && fixupPhase</code> (these are the phases of [https://nixos.org/guides/nix-pills/fundamentals-of-stdenv.html stdenv.mkDerivation])
# At this stage, you may encounter some build quirks of the project. Compile-time errors will hopefully explain what you're missing. For example [https://github.com/NixOS/nixpkgs/blob/643ce4bd0f057bc0b90f0faebeb83a3b14f01674/pkgs/tools/package-management/micromamba/default.nix#L6-L10 micromamba needs a specialized build of libsolv].
# At this stage, you may encounter some build-quirks of the project. Compile-time errors will hopefully explain what you're missing. For example [https://github.com/NixOS/nixpkgs/blob/643ce4bd0f057bc0b90f0faebeb83a3b14f01674/pkgs/tools/package-management/micromamba/default.nix#L6-L10 micromamba needs a specialized build of libsolv].
# Read on below for further steps.
# Read on below for further steps.


Retrieved from "https://wiki.nixos.org/wiki/Nixpkgs/Create_and_debug_packages"