Bisecting: Difference between revisions

Latest revision as of 19:20, 5 July 2024

Bisecting is a feature of version control systems such as Git and Mercurial to easily pinpoint regressions. Owing to their reproducibility, Nix and NixOS are well-suited to this. As a result, we will list a few tips for using tools like this in the Nix context.

Commit selection

There are different ways to tweak git bisect's commit selection to reduce the required builds:

git bisect start flag --first-parent: select merge commits, which depending on the repository can help for caching as well as commit stability.
~~hydrasect: select cached commits cached by Hydra~~ - broken
nixpkgs-staging-bisecter (nix-build only): reduce number of derivations to be built.

Automating bisects

git bisect run: runs the selected command until the culprit is found.
nix-bisect: helps better judge outcome (git bisect's good vs bad skip) and gives nicer outputs than git bisect run.

Flakes

Bisecting changes in a flake itself can just follow the regular bisecting process. Bisecting regressions in flakes' inputs is a bit different in the sense the repositories you are building is not the repository you are bisecting. In this case you can use:

--override-input: nix flag that enables bisecting flakes' inputs
--flake flag on nix operations: allows overriding flake directory so you can use nix flake operations from e.g. git bisect run

@@ Line 1: / Line 1: @@
-[https://git-scm.com/docs/git-bisect Bisecting] is a feature of version control systems such as [[Git]] and [[Mercurial]] to easily pinpoint regressions. Owing to their reproducibility, Nix and NixOS are well-suited to this. As a result, there are a few common use-cases for Nix.
+[https://git-scm.com/docs/git-bisect Bisecting] is a feature of version control systems such as [[Git]] and [[Mercurial]] to easily pinpoint regressions. Owing to their reproducibility, Nix and NixOS are well-suited to this. As a result, we will list a few tips for using tools like this in the Nix context.
-== nixpkgs ==
+== Commit selection ==
-either:
+There are different ways to tweak <code>git bisect</code>'s commit selection to reduce the required builds:
-* directly bisect, say from the root of the nixpkgs repository run <code>git bisect start --first-parent master nixos-24.05 && git bisect run nix build -f . lolcat</code> if package <code>lolcat</code> broke since <code>nixos-24.05</code>. This uses [https://git-scm.com/docs/git-bisect#_bisect_run git bisect run] to run the test until the culprit is found, while the [https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent <code>--first-parent</code>] flag selects merge commits, which in nixpkgs should be cached in Hydra.
+* <code>git bisect start</code> flag [https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent <code>--first-parent</code>]: select merge commits, which depending on the repository can help for caching as well as commit stability.
-* [https://github.com/timokau/nix-bisect nix-bisect]: smartly pick <code>bisect bad/skip</code> in automated bisects and give nicer outputs
+* <s>[https://git.qyliss.net/hydrasect/about/ <code>hydrasect</code>]: select cached commits cached by Hydra</s> - [https://github.com/NixOS/nixpkgs/issues/323985#issuecomment-2210893775 broken]
-* [https://git.qyliss.net/hydrasect/about/ hydrasect]: prioritize cached commits in nixpkgs bisects - unfortunately still lacks nix package/flake
+* [https://github.com/symphorien/nixpkgs-staging-bisecter <code>nixpkgs-staging-bisecter</code>] (<code>nix-build</code> only): reduce number of derivations to be built.
-** after each checkout run <code>git checkout $(hydrasect-search | head -1)</code>
-* [https://github.com/symphorien/nixpkgs-staging-bisecter nixpkgs-staging-bisecter]: like hydrasect but minimize how many derivations you will build even in staging
+== Automating bisects ==
+* [https://git-scm.com/docs/git-bisect#_bisect_run <code>git bisect run</code>]: runs the selected command until the culprit is found.
+* [https://github.com/timokau/nix-bisect <code>nix-bisect</code>]: helps better judge outcome (<code>git bisect</code>'s <code>good</code> vs <code>bad</code> <code>skip</code>) and gives nicer outputs than <code>git bisect run</code>.
 == Flakes ==
-Bisecting changes in a flake itself can just follow the regular bisecting process.
+Bisecting changes in a [[flake]] itself can just follow the regular bisecting process.
-Bisecting regressions in [[flake]]s' inputs typically involves updating the dependency's <code>rev</code> and <code>narHash</code> in the <code>flake.lock</code> file. Using tools like <code>nix flake prefetch</code>, <code>jq</code> and <code>sed</code>, this process [https://gist.github.com/KiaraGrouwstra/70bf11002032b3c265512f4e17607631#file-bisect-sh can be scripted].
+Bisecting regressions in flakes' inputs is a bit different in the sense the repositories you are building is not the repository you are bisecting. In this case you can use:
+* <code>--override-input</code>: nix flag that enables bisecting flakes' inputs
+* <code>--flake</code> flag on nix operations: allows overriding flake directory so you can use nix flake operations from e.g. <code>git bisect run</code>
+[[Category:Version control]]