Automated bisection repeatedly checks a revision and turns the result into a history decision. It can replace many manual checkouts with roughly logarithmic probes on a simple range, but the speedup depends on a trustworthy test. A wrapper that misclassifies missing tools or broken infrastructure can confidently identify the wrong commit. Designing the verdict contract is therefore part of debugging, not merely setup around the search.

Establish a reproducible question

Choose one observable regression and verify it on the proposed good and bad endpoints using the same runner. A release name believed to be healthy is weaker evidence than a successful reproduction check. Keep inputs, toolchain, dependencies, and relevant environment consistent enough that the source revision is the variable under investigation.

Start with the bad revision followed by the known good revision. Git selects intermediate commits and invokes the runner from the checked-out worktree. Keep the runner at a stable path outside the history being changed, or otherwise ensure every tested revision contains a usable version. A relative script that disappears halfway through the search is an infrastructure failure, not evidence that the product became bad.

Map results to bisect’s actual contract

For git bisect run, zero means good. Codes one through 127 mean bad, except 125, which means the revision cannot be tested and should be skipped. Other status values abort the automated search. This is not equivalent to the usual summary that every nonzero code means a regression.

The documented bad range includes 126 and 127, commonly associated with an unexecutable or missing command. Preflight required tools and translate recognized infrastructure failures into an abort status, such as 200. Do not pass through a test tool’s every nonzero result without checking what it means. Likewise, a shell exit value outside the representable range can be truncated; choose a deliberate supported code.

Skipping is appropriate for a specific untestable revision whose classification is unknown, not for every failed build. If the regression being searched is itself a build failure, skipping that outcome would discard the evidence. Conversely, an obsolete dependency unrelated to the symptom must not automatically label an old commit bad. Record why the wrapper distinguishes those cases.

When it breaks

An output pipeline can lose the test’s status when its final logging command succeeds. A cleanup command can overwrite it for the same reason. Save the test result before recording diagnostics, and explicitly exit with the intended mapped verdict. Producing an error message on standard error does not stop Git from accepting an accidental zero.

Tests that change source files, reuse stale build output, or depend on mutable remote services can make successive revisions incomparable. Isolate generated files and retain enough logs to explain each decision. Do not clean a broad user workspace just to make a runner repeatable; use a disposable test environment whose owned paths are explicit.

Skips near the transition can leave several possible first-bad commits. A flaky or nonmonotonic predicate can also invalidate the simple boundary assumption. More probes cannot repair incorrect labels. Inspect the search log and retest the reported boundary and its relevant parent before attributing the root cause. Commit graphs, skipped revisions, and variable build costs also mean logarithmic probe counts are not a wall-clock guarantee.

Check it quickly

Run this with Python 3 and Git 2.28 or newer. The isolated temporary repository excludes personal Git configuration and routing variables, disables hooks and signing, and has no remote. Nine synthetic revisions encode good, bad, or untestable fixture states. The first bad fixture is revision seven, while revision four exercises the skip path.

import os
import subprocess
import sys
import tempfile
from pathlib import Path

env = {key: value for key, value in os.environ.items() if not key.startswith("GIT_")}
env.update(GIT_CONFIG_NOSYSTEM="1", GIT_CONFIG_GLOBAL=os.devnull)
with tempfile.TemporaryDirectory(prefix="git-bisect-run-") as directory:
    root = Path(directory)
    repo, probe = root / "repo", root / "probe.py"
    repo.mkdir()
    probe.write_text('''from pathlib import Path
import sys
try:
    state = Path("state.txt").read_text().strip()
except OSError:
    sys.exit(200)
code = {"good": 0, "bad": 1, "skip": 125}.get(state, 200)
print(f"probe={state} code={code}")
sys.exit(code)
''')
    def git(*args, check=True):
        return subprocess.run(
            ["git", "-c", "user.name=Bisect Example",
             "-c", "user.email=bisect@example.invalid",
             "-c", "commit.gpgsign=false", "-c", "core.hooksPath=/dev/null", *args],
            cwd=repo, env=env, text=True, capture_output=True, check=check,
        )
    git("init", "-q", "-b", "main")
    commits = []
    for number in range(9):
        state = "skip" if number == 4 else ("bad" if number >= 7 else "good")
        (repo / "state.txt").write_text(state + "\n")
        git("add", "state.txt")
        git("commit", "--allow-empty", "-qm", f"revision {number}")
        commits.append(git("rev-parse", "HEAD").stdout.strip())
    codes = []
    for revision in (commits[0], "main"):
        git("switch", "-q", "--detach", revision)
        codes.append(subprocess.run([sys.executable, str(probe)], cwd=repo,
                                    text=True, capture_output=True).returncode)
    git("switch", "-q", "main")
    print("endpoints-verified:", codes == [0, 1])
    git("bisect", "start", commits[-1], commits[0])
    try:
        run = git("bisect", "run", sys.executable, str(probe))
        print("skip-observed:", "probe=skip code=125" in run.stdout)
        print("first-bad-correct:", git("rev-parse", "refs/bisect/bad").stdout.strip() == commits[7])
    finally:
        git("bisect", "reset")
    git("bisect", "start", commits[-1], commits[0])
    try:
        before = git("bisect", "log").stdout
        fatal = git("bisect", "run", sys.executable, "-c", "raise SystemExit(200)", check=False)
        print("fatal-did-not-classify:", fatal.returncode != 0 and git("bisect", "log").stdout == before)
    finally:
        git("bisect", "reset")
    print("original-checkout-restored:", git("symbolic-ref", "--short", "HEAD").stdout.strip() == "main"
          and git("rev-parse", "HEAD").stdout.strip() == commits[-1]
          and not git("status", "--porcelain").stdout)

All five observations should print True. The runner confirms both endpoints, encounters the skip code, and still identifies revision seven. A second search deliberately exits 200; it must fail without adding a classification to the bisect log. Both searches reset to the original branch and tip afterward.

This fixture verifies automation semantics, including an unknown revision and a fatal harness result. Its state file is deliberately synthetic, not a claim to test an application’s behavior. For a real regression, replace that predicate with the independently verified reproduction while preserving the distinctions among good, bad, untestable, and broken test infrastructure.

Further reading

Sources