NixOS Wiki

Nixpkgs/Create and debug packages: Difference between revisions

← Older editNewer edit →
VisualWikitext
imported>Fadenb
m Syntaxhighlight
imported>Fadenb
m Syntax highlighting all the code
Line 78: Line 78:
This will put you into a branch with all the latest changes. Hydra, the build farm, regularly creates binaries, but, since people are constantly contributing to the nix repositories, it is usually the case that there are changes in the master branch which have not yet made it into the binary channel. To take advantage of available binaries you can switch to the revision which produced the binaries in your current system and apply your changes from there. You can use `nixos-version` to see the relevant short revision hash:
This will put you into a branch with all the latest changes. Hydra, the build farm, regularly creates binaries, but, since people are constantly contributing to the nix repositories, it is usually the case that there are changes in the master branch which have not yet made it into the binary channel. To take advantage of available binaries you can switch to the revision which produced the binaries in your current system and apply your changes from there. You can use `nixos-version` to see the relevant short revision hash:


  $ nixos-version  
<syntaxhighlight lang="bash">
  14.11pre52727.5d97886 (Caterpillar)
$ nixos-version  
  ${NixOS release}.${nixpkgs revision}  
14.11pre52727.5d97886 (Caterpillar)
(since the git-repo called nixos was merged into nixpkgs)
${NixOS release}.${nixpkgs revision}  
 
(since the git-repo called nixos was merged into nixpkgs)
</syntaxhighlight>
    
    
 
<syntaxhighlight lang="bash">
  $ nixos-version  
$ nixos-version  
  13.07pre4871_18de9f6-3c35dae (Aardvark)
13.07pre4871_18de9f6-3c35dae (Aardvark)
  ${NixOS release}_${NixOS revision}-${nixpkgs revision}
${NixOS release}_${NixOS revision}-${nixpkgs revision}
</syntaxhighlight>


This string shows the Nixos release number (13.07pre4871) followed by the nixos revision used to produce your current system (18de9f6) followed by the nixpkgs revision (3c35dae).
This string shows the Nixos release number (13.07pre4871) followed by the nixos revision used to produce your current system (18de9f6) followed by the nixpkgs revision (3c35dae).
 
<syntaxhighlight lang="bash">
  $ git branch
$ git branch
  upstream-master
upstream-master
  $ git checkout -b nixpkgs-channel 3c35dae
$ git checkout -b nixpkgs-channel 3c35dae
  Switched to a new branch 'nixpkgs-channel'
Switched to a new branch 'nixpkgs-channel'
  $ git checkout -b my-new-pkg
$ git checkout -b my-new-pkg
  Switched to a new branch 'my-new-pkg'
Switched to a new branch 'my-new-pkg'
</syntaxhighlight>


After making some changes you can commit them into your local repo:
After making some changes you can commit them into your local repo:
  $ git add foo
<syntaxhighlight lang="bash">
  $ git commit
$ git add foo
$ git commit
</syntaxhighlight>


Then you push your changes to your fork:
Then you push your changes to your fork:
 
<syntaxhighlight lang="bash">
  $ git push origin my-new-pkg
$ git push origin my-new-pkg
</syntaxhighlight>


You can use this to open a pull request on github.
You can use this to open a pull request on github.


If some time has passed since you have created your fork, you will want to merge your changes with upstream and test that it still works.
If some time has passed since you have created your fork, you will want to merge your changes with upstream and test that it still works.
  git fetch upstream
<syntaxhighlight lang="bash">
  git merge upstream
git fetch upstream
git merge upstream
</syntaxhighlight>


If your merge then fails because someone else has made the same change (for example, someone else also packaged a library you have just packed for the program you want to get into nixpkgs), then you can do this:
If your merge then fails because someone else has made the same change (for example, someone else also packaged a library you have just packed for the program you want to get into nixpkgs), then you can do this:


  git rebase -i HEAD~10
<syntaxhighlight lang="bash">
git rebase -i HEAD~10
</syntaxhighlight>


there select the edit mode for your commit and remove the your code which added the library. **Warning: only use 'git rebase' on your commits, which have not been pushed and nobody else is working with already!**
there select the edit mode for your commit and remove the your code which added the library. **Warning: only use 'git rebase' on your commits, which have not been pushed and nobody else is working with already!**
Line 120: Line 130:
Next you have to test if your program works with the library packaged from someone else, then do:
Next you have to test if your program works with the library packaged from someone else, then do:


  git checkout master
<syntaxhighlight lang="bash">
  git log --stat
git checkout master
git log --stat
</syntaxhighlight>


and pick the commit where the library was added. Finally cherry-pick that commit into your branch:
and pick the commit where the library was added. Finally cherry-pick that commit into your branch:


  git checkout my-new-pkg
<syntaxhighlight lang="bash">
  git cherry-pick 5d97886a6a545fb20495e0837cc50fa63d2a80e1
git checkout my-new-pkg
git cherry-pick 5d97886a6a545fb20495e0837cc50fa63d2a80e1
</syntaxhighlight>


Afterwards do your usual tests and if needed also make modifications to the library but keep in mind that this might break the other use-case of that library and if in doubt check that as well.
Afterwards do your usual tests and if needed also make modifications to the library but keep in mind that this might break the other use-case of that library and if in doubt check that as well.
Line 134: Line 148:
nix-shell is a command which drops you into the build environment for a package. This is convenient for writing and debugging nix expressions. Nix-shell requires nix-1.6.x although running nix-build --run-env produces a similar environment.
nix-shell is a command which drops you into the build environment for a package. This is convenient for writing and debugging nix expressions. Nix-shell requires nix-1.6.x although running nix-build --run-env produces a similar environment.


  mkdir -p /tmp/nix-shell-bc
<syntaxhighlight lang="bash">
  cd /tmp/nix-shell-bc
mkdir -p /tmp/nix-shell-bc
  nix-shell $NIXREPOS/nixpkgs -A bc
cd /tmp/nix-shell-bc
  export out=/tmp/foo/out
nix-shell $NIXREPOS/nixpkgs -A bc
export out=/tmp/foo/out
</syntaxhighlight>


now we have find out which phases are specified for this package:
now we have find out which phases are specified for this package:
    typeset -f genericBuild | grep 'phases='
<syntaxhighlight lang="bash">
    phases="$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase installCheckPhase $preDistPhases distPhase $postPhases";
typeset -f genericBuild | grep 'phases='
phases="$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase installCheckPhase $preDistPhases distPhase $postPhases";
</syntaxhighlight>


The phases can be defined either as a string to be eval'ed or as a shell function, [https://github.com/NixOS/nixpkgs/blob/5a0b79/pkgs/stdenv/generic/setup.sh#L818 this is how] Nix invokes it.
The phases can be defined either as a string to be eval'ed or as a shell function, [https://github.com/NixOS/nixpkgs/blob/5a0b79/pkgs/stdenv/generic/setup.sh#L818 this is how] Nix invokes it.


so when developing you need to run these phases in a row:
so when developing you need to run these phases in a row:
  unpackPhase
<syntaxhighlight lang="bash">
  patchPhase
unpackPhase
  configurePhase
patchPhase
  buildPhase
configurePhase
  checkPhase
buildPhase
  installPhase
checkPhase
  fixPhase
installPhase
  installCheckPhase
fixPhase
  installPhase
installCheckPhase
  distPhase
installPhase
distPhase
</syntaxhighlight>


{{Note|you do not need to run $preConfigurePhase explicitly as it is run, when running configurePhase already.}}
{{Note|you do not need to run $preConfigurePhase explicitly as it is run, when running configurePhase already.}}


list all functions which are declared in '''set''':
list all functions which are declared in '''set''':
  typeset -F
<syntaxhighlight lang="bash">
  declare -f addCVars
typeset -F
  declare -f addToCrossEnv
declare -f addCVars
  declare -f addToNativeEnv
declare -f addToCrossEnv
  declare -f addToSearchPath
declare -f addToNativeEnv
  declare -f addToSearchPathWithCustomDelimiter
declare -f addToSearchPath
  declare -f buildPhase
declare -f addToSearchPathWithCustomDelimiter
  declare -f checkPhase
declare -f buildPhase
  declare -f closeNest
declare -f checkPhase
  declare -f command_not_found_handle
declare -f closeNest
  declare -f configurePhase
declare -f command_not_found_handle
  declare -f distPhase
declare -f configurePhase
  declare -f dumpVars
declare -f distPhase
  declare -f ensureDir
declare -f dumpVars
  declare -f exitHandler
declare -f ensureDir
  declare -f findInputs
declare -f exitHandler
  declare -f fixLibtool
declare -f findInputs
  declare -f fixupPhase
declare -f fixLibtool
  declare -f genericBuild
declare -f fixupPhase
  declare -f header
declare -f genericBuild
  declare -f installBin
declare -f header
  declare -f installCheckPhase
declare -f installBin
  declare -f installPhase
declare -f installCheckPhase
  declare -f patchELF
declare -f installPhase
  declare -f patchPhase
declare -f patchELF
  declare -f patchShebangs
declare -f patchPhase
  declare -f runHook
declare -f patchShebangs
  declare -f showPhaseHeader
declare -f runHook
  declare -f startNest
declare -f showPhaseHeader
  declare -f stopNest
declare -f startNest
  declare -f stripDirs
declare -f stopNest
  declare -f stripHash
declare -f stripDirs
  declare -f substitute
declare -f stripHash
  declare -f substituteAll
declare -f substitute
  declare -f substituteAllInPlace
declare -f substituteAll
  declare -f substituteInPlace
declare -f substituteAllInPlace
  declare -f unpackFile
declare -f substituteInPlace
  declare -f unpackPhase
declare -f unpackFile
declare -f unpackPhase
</syntaxhighlight>


If the phase has been defined as a function, to list a particular function type:
If the phase has been defined as a function, to list a particular function type:
  typeset -f unpackPhase
<syntaxhighlight lang="bash">
typeset -f unpackPhase
</syntaxhighlight>


Otherwise, if it was a string, simply echo the variable related to it
Otherwise, if it was a string, simply echo the variable related to it


  echo $unpackPhase
<syntaxhighlight lang="bash">
echo $unpackPhase
</syntaxhighlight>


In either case, you can see the code that is about to be executed for each phase:
In either case, you can see the code that is about to be executed for each phase:


  typeset -f unpackPhase
<syntaxhighlight lang="bash">
  unpackPhase ()  
typeset -f unpackPhase
  {  
unpackPhase ()
      runHook preUnpack;
{
      if [ -z "$srcs" ]; then
    runHook preUnpack;
          if [ -z "$src" ]; then
    if [ -z "$srcs" ]; then
              echo 'variable $src or $srcs should point to the source';
        if [ -z "$src" ]; then
              exit 1;
            echo 'variable $src or $srcs should point to the source';
          fi;
            exit 1;
          srcs="$src";
        fi;
      fi;
        srcs="$src";
      local dirsBefore="";
    fi;
      for i in *;
    local dirsBefore="";
      do
    for i in *;
          if [ -d "$i" ]; then
    do
              dirsBefore="$dirsBefore $i ";
        if [ -d "$i" ]; then
          fi;
            dirsBefore="$dirsBefore $i ";
      done;
        fi;
      for i in $srcs;
    done;
      do
    for i in $srcs;
          unpackFile $i;
    do
      done;
        unpackFile $i;
      if [ -n "$setSourceRoot" ]; then
    done;
          runHook setSourceRoot;
    if [ -n "$setSourceRoot" ]; then
      else
        runHook setSourceRoot;
          if [ -z "$sourceRoot" ]; then
    else
              sourceRoot=;
        if [ -z "$sourceRoot" ]; then
              for i in *;
            sourceRoot=;
              do
            for i in *;
                  if [ -d "$i" ]; then
            do
                      case $dirsBefore in  
                if [ -d "$i" ]; then
                          *\ $i\ *)
                    case $dirsBefore in
 
                        *\ $i\ *)
                          ;;
 
                          *)
                        ;;
                              if [ -n "$sourceRoot" ]; then
                        *)
                                  echo "unpacker produced multiple directories";
                            if [ -n "$sourceRoot" ]; then
                                  exit 1;
                                echo "unpacker produced multiple directories";
                              fi;
                                exit 1;
                              sourceRoot="$i"
                            fi;
                          ;;
                            sourceRoot="$i"
                      esac;
                        ;;
                  fi;
                    esac;
              done;
                fi;
          fi;
            done;
      fi;
        fi;
      if [ -z "$sourceRoot" ]; then
    fi;
          echo "unpacker appears to have produced no directories";
    if [ -z "$sourceRoot" ]; then
          exit 1;
        echo "unpacker appears to have produced no directories";
      fi;
        exit 1;
      echo "source root is $sourceRoot";
    fi;
      if [ "$dontMakeSourcesWritable" != 1 ]; then
    echo "source root is $sourceRoot";
          chmod -R u+w "$sourceRoot";
    if [ "$dontMakeSourcesWritable" != 1 ]; then
      fi;
        chmod -R u+w "$sourceRoot";
      runHook postUnpack
    fi;
  }
    runHook postUnpack
}
</syntaxhighlight>


you can also modify the configureFlags prefix:
you can also modify the configureFlags prefix:
  export configureFlags="--prefix=$out --with-readline"
<syntaxhighlight lang="bash">
export configureFlags="--prefix=$out --with-readline"
</syntaxhighlight>


Tip: A git repository can be used for snapshotting attempts at building the package. This also makes it easy to generate patches, should you need to.
Tip: A git repository can be used for snapshotting attempts at building the package. This also makes it easy to generate patches, should you need to.
Retrieved from "https://wiki.nixos.org/wiki/Nixpkgs/Create_and_debug_packages"