The Four Deaths

An unattended loop dies four ways. Three are engineering bugs the log catches and brakes cure. The fourth is your own degradation as an engineer, and no code fixes it - but loopexec can at least force you to look.

This is a diagnostics page. For each death it names the symptom, the mechanism that detects it, and the halt reason loopexec computes - because a loop that dies without a computed reason is not a halt, it is a crash you have to autopsy by hand.

Every reason is computed, never declared

A loop runs until an external check passes, a guard trips, a budget is spent, or a human is required - never because an agent reports it is done. Every halt emits a reason computed from observed state. The --halt-reason flag is a hidden test fixture, not a control. See halt reasons for the full map.

The taxonomy at a glance

Death	Detected by	halt_reason	Exit class
Runaway	iteration fuse, run-total budget, cost-rate anomaly	max_iterations_reached, budget_exceeded, cost_anomaly	12, 18
Silent death	external watchdog over the heartbeat; every-exit-through-the-logger	heartbeat_stale, execution_failure	19, 40
Random walk	set-based progress, no-regression ratchet, oscillation detection	no_progress_detected, same_failure_repeated, oscillation_detected, same_test_regressed, unsatisfiable_constraints, infeasible_suspected	17
Understanding debt	diffs_merged_unread counter + comprehension gate	comprehension_debt_exceeded	19

Capability honesty

loopexec implements Slices 0-5. The fuses, the typed receipt, the external watch (staleness detection), the no-regression ratchet, and the comprehension gate all ship. The rolling-sigma cost-anomaly detector and the SIGKILL-the-wedged-PID actuator remain Planned - specified here, not yet shipping. Where a mechanism is Planned, this page describes the contract, not current behavior. Status is tracked normatively in SPEC section 11.

Runaway

The symptom. The bill and the iteration count climb, no green ever arrives. In the log: many agent calls in a row, no convergence. The loop is doing work; the work is going nowhere; the meter is running.

Why the naive brake is not enough. A per-turn budget bounds nothing. A $10 per-call cap across twenty iterations is a $200 run wearing a $10 label. The only true bound on total spend is a fuse on the run as a whole.

The mechanism. Two hard fuses plus an anomaly detector:

• Iteration fuse - a maximum-iterations ceiling. On exhaustion, halt max_iterations_reached (exit 12).
• Run-total budget - a hard cap on cumulative cost, accumulated from parsed provider usage (agent tokens + judge cost + per-iteration check-execution cost), not a per-turn limit. On breach, halt budget_exceeded (exit 18).
• Cost-rate anomaly - a rolling-sigma detector on the spend rate, distinct from the absolute cap. A sudden context blow-up trips cost_anomaly (exit 18) before the cap does, telling you the rate changed, not merely the total is high.

explain-halt is specified to then separate the two iteration-cap meanings the 3am operator actually needs to tell apart: max_iterations_reached while the failing set was still strictly shrinking (“raise the limit, retry”) from a stalled run (“do not retry - it burns budget forever”). That distinction belongs to the random walk, below.

Status. Iteration fuse: Shipped. Run-total budget enforcement, the rolling-sigma cost_anomaly detector, and the composite cost model: Planned.

Silent death

The symptom. The loop “works” but stands still. No crash, no error, no new log events - the heartbeat simply stops advancing. The usual cause is a wedged model call: an agent invocation that hangs forever, or a non-zero exit swallowed by set -e that kills the process mid-iteration with no halt record.

Why a heartbeat alone does not help. A heartbeat is a dead-man’s switch written by the very process whose death it must detect. Nothing reads it. A heartbeat nobody polls is just a file.

The mechanism. Two independent guarantees:

• An EXTERNAL watchdog. loopexec watch is a separate supervisor that polls the heartbeat’s age and emits heartbeat_stale (exit 19) when it goes stale (the SIGKILL-the-wedged-PID actuator is Planned). The detector must not share a fate with the thing it watches - that is the whole point of putting it outside the loop process.
• Every-exit-through-the-logger. Every exit - including a non-zero agent exit from an API error or a rate limit - MUST pass through the typed logger and resolve to a computed reason; a raw failure emits execution_failure (exit 40). A set -e-style silent death, where the process vanishes without a typed event, is non-conformant. The runtime captures the return code explicitly and branches on it; it never lets shell semantics be the exit path.

Tip

The heartbeat must be updated around long operations, not only at iteration start - otherwise a call that wedges for an hour inside one iteration looks alive until the iteration would have ended. The watchdog records the worker PID and the current phase so the kill is precise and the receipt says what it was doing when it stalled.

Status. Typed JSONL receipt / every-exit-through-the-logger: Shipped. External watch staleness detection: Shipped; the SIGKILL actuator: Planned.

Random walk - the missing Lyapunov function

The symptom. The loop spins but moves away from the goal. Agent calls fire; the failing test changes to a new one each iteration; green never arrives. It is the most expensive death because it looks like progress - work is happening, the failure keeps “changing” - right up until the budget is gone.

This is the death the original essay named and then mis-cured. It prescribed “a deterministic fixpoint check” as the fix. But the loop in question already had a fixpoint check and walked anyway. That is the proof the cure is necessary but not sufficient.

Why a bare fixpoint check cannot catch it

A fixpoint check is an arrival detector. It tells you when the state has reached the fixpoint. It exerts zero force pulling the state toward the fixpoint. There is no potential function - no quantity guaranteed to decrease - so there is nothing to forbid a step that moves the state backward. The check answers “are we there?” every iteration and is content to answer “no” forever. A check_fixpoint loop with no acceptance gate is a fixpoint detector wrapped around an undirected walk.

What is missing is a Lyapunov function: a scalar potential the loop is structurally forbidden from increasing. Without it, “progress” is undefined, and a thing you cannot define you cannot halt on.

Why a string-equality breaker cannot catch it

The folk cure is a circuit breaker that compares the last failure line to the current one and trips after N identical repeats. It cannot catch the random walk it was built to catch. Concretely:

# ANTI-PATTERN - do not ship this. It cannot catch the random walk.
current_failure=$(run_tests | grep -m1 "FAIL")
if [ "$current_failure" = "$last_failure" ]; then
  repeat=$((repeat + 1))   # only ever fires on IDENTICAL consecutive strings
fi
last_failure="$current_failure"

Four structural defects:

1. Oscillation is invisible. A walk A -> B -> A -> B never produces two consecutive identical failures, so “same failure 3x” never trips. The exact pathology the breaker names slips through it.
2. It carries one bit per iteration. Green/red - or one grepped line - cannot encode “this diff fixed test A but regressed eight previously-passing tests.” The frontier silently slides from 48/50 to 40/50 and the breaker sees a “different failure,” i.e. apparent movement.
3. A file-level FAIL header masks intra-file motion. Matching the first FAIL line hides changes within a file’s failing assertions - progress and regression both disappear behind the same header.
4. The empty match collides with the initial value. An empty grep result equals the last_failure="" sentinel, false-firing “stuck” on iteration 2 before the loop has done anything.

“Progress” was never rigorously defined, so the breaker compared the wrong thing.

The cure: progress over a set, with a ratchet

Define progress over the failing-test-ID set F_i, parsed from a structured reporter (go test -json, jest --json, TAP) - not a grepped line. Then install the potential function the bare check lacked.

progress:
  failure_identity: [test_id, assertion_location, message_hash]
  init_sentinel: UNSET          # distinct from "" - empty is not the initial value
  no_progress:
    halt_if_set_not_shrinking_for_iters: 5   # |F_i| no strict decrease over K
    halt_if_passed_test_reenters: true       # oscillation / regression signature
  oscillation:
    detect_contradictory_pairs_over_cycles: 2

acceptance:
  no_regression:
    passing_set_superset_required: true   # no previously-green test may go red
    on_violation: revert_to_last_accepted # git restore; re-prompt
  best_so_far_checkpoint: true            # the potential function: 48/50 -> 40/50 is recorded

The pieces, and the halt reason each produces:

• No-regression ratchet (the Lyapunov function). Accept an iteration’s diff only if its passing-test set is a superset of the last accepted set. A diff that fixes A while breaking B is rejected and reverted to the last accepted commit, then re-prompted. best_so_far is the monotone frontier that makes “progress” definable and termination provable for a feasible target. Acceptance runs before any green branch can fire, inside the same ordered transaction where guards dominate success.
• Set-based stall detection. No strict decrease in |F_i| over K iterations halts no_progress_detected; an identical failing set repeating halts same_failure_repeated (both exit 17).
• Oscillation detection. A previously-passing test re-entering F halts same_test_regressed or oscillation_detected; a contradictory pair recurring across >=2 cycles halts unsatisfiable_constraints (all exit 17).
• Feasibility. Refuse to start without a known-feasible reference - a full-green commit that proves a green state exists. When none is reachable, halt infeasible_suspected (exit 17) rather than burning the iteration cap pretending the task is merely hard. explain-halt is specified to report which case you are in.

Note

The acceptance ratchet is what converts a check_fixpoint loop from an undirected search into a monotone one. Inspect or advance the frontier with ratchet. Determinism of the check itself - a separate prerequisite - is covered under determinism.

Status. Set-based progress, oscillation/feasibility detection, and the no-regression ratchet: Shipped (via --failures-cmd). A standalone ratchet command and git revert-to-best: Planned.

Understanding debt

The symptom. The repository grows; you understand less. The loop ships code faster than any human reads it, and diffs get stamped blind. This death leaves no trace in the log - which is exactly what makes it the most dangerous. There is no failing test, no budget breach, no stalled set. The system is “healthy” and the operator is going blind.

The original framing called this “not a loop bug but your degradation as an engineer” and left the only safeguard as “your discipline.” For a runtime that promises auditability, that is a cop-out. Understanding debt is in fact the most instrumentable of the four, because it is a counter.

The mechanism. Instrument it, then gate on it:

comprehension:
  track_diffs_merged_unread: true   # (diffs merged) - (diffs a human acked reading)
  review_every_n_diffs: 10
  on_debt_exceeded: comprehension_debt_exceeded
  ack_command: "loopexec ack --through <sha> --reviewer <id>"

• Measure the debt. diffs_merged_unread = (diffs merged) - (diffs a human signed off as read). A real number, tracked in durable state.
• Gate on it. When the counter crosses its threshold, halt comprehension_debt_exceeded (exit 19). The loop stops shipping until a human catches up.
• Clear it with a signed ack. loopexec ack --through <sha> --reviewer <id> writes a signed line into the receipt and resets the counter. The signature makes the acknowledgment attributable and part of the replayable verdict.

What this gate is, and is not

This is a forcing and visibility gate, not proof of comprehension. loopexec cannot verify that a human understood a diff - only that a human was required to look and signed to say they did. The honesty is in the labeling: it converts an invisible, untracked liability into a counter with a hard stop and an auditable ack. That is the most a runtime can do; the rest is, still, your discipline - but now it is a discipline the system enforces instead of one it hopes for.

Status. diffs_merged_unread tracking, the comprehension gate, and signed ack: Planned.

Reading the log: a death is a diagnosis, not a guess

The point of typing every death is that the autopsy takes a second, not an afternoon. After a loop halts you read one field - the computed halt_reason - and you know which death you are looking at and what to do next:

• climbing cost, no green -> budget_exceeded / cost_anomaly -> runaway: lower the cap or split the task.
• heartbeat frozen, no events -> heartbeat_stale -> silent death: the watchdog already killed the wedge; inspect the last phase.
• failing set churning, no shrink -> no_progress_detected / oscillation_detected -> random walk: check feasibility before raising the limit.
• everything green, counter tripped -> comprehension_debt_exceeded -> understanding debt: read the backlog, then ack.

A loop that stops, names the death with a computed reason, and leaves a receipt you can verify offline - that is the contract. Without the log it is guessing. With it, it is a diagnosis.

See also: halt reasons for the full reason -> exit-code map, determinism for the check stability bound the random-walk cure depends on, and loop engineering for the discipline these four deaths motivate.