cogloom · Free lesson

Which of the agent's 16 commits broke it? Let `git bisect run` find it in 4 test runs

Result first: an agent made 16 commits, and somewhere along the way add 2 3 started returning -1. I gave git bisect one good commit, one bad commit and a 3-line test script. It ran the test 4 times and named the first bad commit, "commit 11", which is exactly the one where the bug was planted.

printf '#!/usr/bin/env bash\nsource ./calc.sh\n[ "$(add 2 3)" = 5 ]\n' > ../test.sh && chmod +x ../test.sh
git bisect start HEAD <known-good-commit>
git bisect run ../test.sh
git bisect reset          # go back to where you started

The rules for the test script (from the git docs): it "should exit with code 0 if the current source code is good/old, and exit with a code between 1 and 127 (inclusive), except 125, if the current source code is bad/new." Exit code 125 means "this commit can't be tested, skip it." "Any other exit code will abort the bisect process."

Why this suits agent workflows: - Agents make many small commits, and bisect is a binary search, so the number of test runs grows slowly as the history grows (here 4 runs for 16 commits). - Keep the test script outside the repo or untracked, as ../test.sh is here, so checking out old commits doesn't change the test itself. - Use exit 125 for commits that don't build, rather than marking them bad.

Do this now: next time an agent session ends with a regression, write the one-line check first, then run git bisect run before reading any diffs.

Sources

git documentation, git-bisect: https://git-scm.com/docs/git-bisect

Also on dev.to: read this lesson on dev.to. More free lessons: cogloom lessons index.