loop.yml
What loop.yml is
Section titled “What loop.yml is”loop.yml is to an agent loop what a Compose file is to a service graph: one declarative artifact that pins every fuse, oracle, guard, and budget, so a run is reproducible and a halt is explainable. It is the missing artifact of loop engineering - the file you read to know how a loop will behave before you start it, and the file a receipt is verified against after it stops.
A loop runs until an external check passes, a guard trips, a budget is spent, or a human is required - never because an agent reports it is done. loop.yml is where each of those stop conditions is declared. The top-level loop.mode selects which of the two machines is running: check_fixpoint (iterate until an external oracle passes) or task_list (iterate a SMALL plan). check_fixpoint is the degenerate one-task plan whose acceptance criterion is the external check.
Block status at a glance
Section titled “Block status at a glance”The status column is the status of the mechanism each block configures, mapped to SPEC section 11. The loop.yml file itself is a Planned input format - run takes these as CLI flags today - so read each row as: the capability ships (or ships at its core) via run’s flags, even though the declarative key is not yet the input. Shipped (core) means the core ships with a named sub-part still Planned (named inline below).
| Block | What it defends | Mechanism status (SPEC section 11) |
|---|---|---|
loop | iteration fuse + topology | Shipped (iterating run; task_list mode reserved) |
context | token-budget slice | Shipped (core) (import_closure / dep_graph tiers Planned) |
check | the stop oracle | Shipped (core) (probe + capture + integrity core ship; adequacy mutation canary ships; coverage-delta + hermeticity Planned) |
acceptance | anti-random-walk ratchet | Shipped (core) (git revert-to-best Planned) |
progress | stuck / oscillation detection | Shipped |
feasibility | feasible vs. infeasible | Shipped (core) (explain-halt distinction ships; known-feasible-ref start-gate Planned) |
guards | the metric the check measures | Shipped (core) (assertion-count / manifest-hash / coverage-floor Planned) |
models | verdict reproducibility + judge | Shipped (core) (executor pin ships; cross-lab judge Planned) |
isolation | host + credentials | Shipped (core) (orchestration ships; runtime / egress-proxy / key API are operator hooks) |
budget | runaway spend | Shipped (core) (cap recorded; live metering + cost_anomaly Planned) |
escalation | human handoff | Shipped (core) (file/stdout; github/slack channels Planned) |
comprehension | understanding debt | Shipped (--comprehension-every + ack) |
observability | audit trail + liveness | Shipped (core) (typed log + heartbeat staleness; kill-PID actuator Planned) |
receipt | replayable verdicts | Shipped (core) (pin + attest + replay + reexecute ship; live cost metering Planned) |
state | resumability | Shipped |
loop - identity, the iteration fuse, and topology
Section titled “loop - identity, the iteration fuse, and topology”What it defends. Total spend and topology honesty. max_iterations is the one true bound on a run; mode records which of the two machines (check_fixpoint or task_list) is executing, so every receipt stays attributable. Leaving mode implicit forks the product and makes verdicts non-replayable.
Status: Shipped - the real iterating run loop (SPEC section 4). run runs --exec then the --check oracle each iteration, computes its halt from observed state, and writes a typed receipt; max_iterations ships as --max-iterations. The loop.yml keys that declare this (mode, max_iterations) are the Planned config form; run takes them as flags today, and task_list mode is reserved.
version: 1
loop: name: fix-failing-tests # mode picks the control topology. The repo ships TWO machines # (an external-check loop vs a task_list loop); leaving it implicit # forks the product and makes verdicts non-replayable. # check_fixpoint = iterate until the external oracle passes / a guard # trips; task_list = iterate SMALL plan tasks. check_fixpoint is the # degenerate one-task plan whose acceptance IS the check. mode: check_fixpoint max_iterations: 20 # FUSE 1: the only true bound on total spend stateless: true # fresh context per iter; cures context rot + quadratic cost commit_each_iteration: true # gives HEAD~1 a stable meaningcontext - a narrow slice with a real ceiling
Section titled “context - a narrow slice with a real ceiling”What it defends. The token budget as a true ceiling. The context builder narrows the prompt to a relevant slice, widens through escalation tiers before declaring stuck, and never silently evicts a relevant file or dies mid-build. A slice that cannot fit even after widening emits context_budget_unsatisfiable.
Status: Shipped (core) - build-context (SPEC section 10) ships the budgeted relevant-file slice (stacktrace + last-diff + untracked), workdir-confined and symlink-safe, with a true token ceiling. The import_closure / dep_graph escalation tiers are Planned.
context: builder: ./build_context.sh max_tokens: 8000 token_estimator: tokenizer # chars/4 is a prose ratio; code is ~25-40% denser token_divisor_fallback: 3.3 # code-calibrated, used only if no real tokenizer count_bytes_as: utf8_chars # wc -c counts BYTES; multibyte source miscounts include: [.loop_state.json, STATUS.md] relevant_files: strategy: stacktrace_last_diff diff_base: worktree_vs_head # not HEAD~1 on a never-committing tree track_untracked: true # git add -N so new files are visible escalation_tiers: # widen before declaring stuck - stacktrace_last_diff - test_import_closure - one_hop_dependency_graph widen_after_no_progress_iters: 3 fallback: # build_context must never be fatal on_no_relevant_files: emit_minimal_context on_overflow: split_task # split | summarize | escalate - never silent evict never_fatal: truecheck - the stop oracle
Section titled “check - the stop oracle”What it defends. The stop condition itself. The check is run once per iteration (single capture), determinism is maintained as a confidence bound rather than probed once, and a green that the check never exercised is rejected as a false fixpoint. This block bundles four distinct properties the roadmap collapsed into “deterministic”: determinism, hermeticity, adequacy, and metric integrity.
Status: Shipped (core) - mixed by sub-part. The single-capture execution rides the Shipped run loop; deterministic_probe is the Shipped (core) probe-check confidence bound (SPEC O2; adversarial perturbation + the in-loop sequential monitor Planned); metric_integrity is the Shipped (core) metric-integrity gate via run --integrity-cmd (collected-set monotonicity ships; assertion-count / manifest-hash / coverage-floor Planned, SPEC section 6). The adequacy mutation canary ships via doctor --mutate-cmd (halts check_inadequate); the coverage-delta tier of adequacy and the hermeticity preconditions of the doctor gate remain Planned (O3-O5).
check: command: npm test --silent success_exit_code: 0 single_capture_per_iteration: true # run ONCE; derive verdict AND failing-set capture: [exit_code, stdout, stderr]
phases: # targeted each iter, full before halt targeted: { command: "npm test -- --findRelatedTests", run: every_iteration } full: { command: "npm test --silent", run: before_halt, required_before: [success_condition_met, no_progress_detected] }
deterministic_probe: # determinism as a maintained confidence bound max_flake_rate: 0.00017 # keep whole-loop false-flake < 1% over ~60 runs confidence_target: 0.99 derived_runs: "ceil(3 / max_flake_rate)" # rule-of-three; 10 is a FLOOR, not a target report: confidence_bound # report achieved bound, not pass/fail fingerprint: [exit_code, stdout_normalized, stderr_normalized] adversarial: # surface order/seed/clock/load flakes shuffle_test_order: true randomize_seed: true induce_cpu_load: true max_parallelism: true perturb_clock: true report_broken_dimension: true sequential_monitor: # in-loop runs are free Bernoulli samples enabled: true reprobe_affected_subset_each_iter: true ci_method: wilson_lower_bound halt_when_stability_below: 0.99 # -> check_not_deterministic hermetic_required: true # determinism must be engineered, not lucky side_effect_probe: # determinism != idempotency run_twice_same_workspace: true # detect side effects -> check_has_side_effects run_each_hermetic_reset: true # isolate determinism -> check_flaky
adequacy: # determinism != adequacy (false fixpoint) coverage_delta_required: true # every changed/added line must be exercised mutation_probe_on_diff: true # a mutant in changed lines MUST go red on_violation: check_inadequate
metric_integrity: # the gate that supersedes `git diff test/` baseline_captured_at: t0 # immutable loop-start ref (NOT HEAD~1) min_test_count: baseline # count MUST NOT decrease coverage_floor: baseline # coverage MUST NOT decrease assertion_count_nondecreasing: true # AST-level; catches gutted bodies collected_set_monotonic: true # pytest --collect-only / jest --listTests / go -list protected_manifest_sha: # hash the FULL test-determining surface - runner.config - coverage.config - ci.yaml - test-deps.lockfile canary_mutation: true on_violation: metric_integrity_violationacceptance - the anti-random-walk ratchet
Section titled “acceptance - the anti-random-walk ratchet”What it defends. Convergence. The superset gate is the potential function that makes “progress” provable: a diff is accepted only if its passing-test set is a superset of the last accepted set. A diff that fixes test A while regressing test B is reverted to the last accepted commit, not kept. The numeric ratchet is opt-in with an acceptance band; the superset gate is the hard guarantee.
Status: Shipped (core) - the no-regression ratchet ships inside run via --failures-cmd (SPEC section 3.2): the superset / no-regression gate halts same_test_regressed, oscillation_detected, and no_progress_detected. Git revert-to-best is Planned.
acceptance: no_regression: passing_set_superset_required: true # no previously-green test may go red on_violation: revert_to_last_accepted # git restore; re-prompt ratchet: enabled: false # opt-in; the superset gate is the hard guarantee metric: tests_passing accept_if: ">= best_so_far" acceptance_band: 0.0 allow_worse_escape_hatch: true # transient regressions during refactors are legit best_so_far_checkpoint: true # records 48/50 -> 40/50progress - define “stuck” over a set, not a string
Section titled “progress - define “stuck” over a set, not a string”What it defends. The iteration cap, against being burned on a stalled or oscillating loop. “Stuck” is defined over the failing-test set F_i parsed from a structured reporter, not a grepped line. A sentinel init value keeps “empty failure set” distinct from “initial,” which kills the false-stuck bug.
Status: Shipped - these feed the computed halt reasons (no_progress_detected, oscillation_detected, same_test_regressed) that run derives from the failing set today (SPEC section 5). unsatisfiable_constraints stays reserved.
progress: failure_identity: [test_id, assertion_location, message_hash] # via jest --json / go -json reporter_format_required: true # unparseable -> check_not_deterministic, not no_progress init_sentinel: UNSET # distinct from "" so empty != initial track_failing_set_per_iteration: true repeat_failure_limit: 3 no_progress: halt_if_set_not_shrinking_for_iters: 5 # |F_i| no strict decrease over K halt_if_passed_test_reenters: true # oscillation / regression signature on_no_progress: no_progress_detected oscillation: detect_contradictory_pairs_over_cycles: 2 on_detect: unsatisfiable_constraintsfeasibility - “no green exists” vs. “green exists but unfound”
Section titled “feasibility - “no green exists” vs. “green exists but unfound””What it defends. The 3am operator’s one decision. A recorded full-green reference is an existence proof; without it the runtime refuses to start. On halt, the runtime distinguishes max_iterations_reached while |F_i| was still strictly decreasing (“raise the limit”) from infeasible_suspected (“never retry - it burns budget forever”).
Status: Shipped (core) - explain-halt ships the operator distinction: max_iterations_reached while the failing set was still strictly decreasing (“raise the limit”) versus a stalled or infeasible halt (“do not retry”). The require_known_feasible_ref start-gate (set by doctor / probe-check) and the infeasible_suspected reason are Planned.
feasibility: require_known_feasible_ref: true # Step-1 full-green commit = existence proof known_feasible_ref: null # set by doctor/probe; absent -> refuse to start on_unsatisfiable: infeasible_suspected distinguish_exhausted_improving_vs_stalled: true # tells operator if more budget is rationalguards - guards dominate success
Section titled “guards - guards dominate success”What it defends. The metric the check measures. Guards dominate success: the integrity gate is evaluated against an immutable baseline t0 before any green branch can fire (G-before-S). A check that went green because a guard was violated - for example a weakened test - is classified as the guard reason, never success_condition_met. The diff is examined across working-tree, staged, committed, and untracked changes, because the cheapest cheats hide in the index and in untracked overrides.
Status: Shipped (core) - the metric-integrity gate ships inside run via --integrity-cmd with guards-dominate-success ordering (G-before-S) evaluated before any green branch (SPEC section 6); collected-set monotonicity ships, while the assertion-count, manifest-hash, and coverage-floor layers are Planned. The guards block and check.metric_integrity are two faces of the same gate; this block fixes the evaluation order.
guards: blocked_paths: [test/, infra/, .env, .git/] # .git/ writes escape the worktree sandbox evaluation_order: guards_before_success # gate BEFORE green; guards dominate success reward_hacking: test_diff_forbidden: true baseline_ref: t0 # diff vs immutable t0, not index/HEAD (git add bypasses) cover: [working_tree, staged, committed, untracked] # untracked overrides never show in diff protected_globs: [":(top)test", ":(top)tests"] tests_readonly_mount: true # agent physically cannot edit/commit tests revert_with: ["git reset --hard <t0>", "git clean -fd"] on_violation: blocked_path_modifiedmodels - pin the executor, isolate the judge
Section titled “models - pin the executor, isolate the judge”What it defends. Reproducibility of the verdict and independence of the judge. The executor’s full identity tuple and recorded sampling params are pinned, or the run fails closed (workspace_invalid). The judge must differ in model and lab - same-lab models share correlated reward-hack blind spots - and stays advisory: it can veto a green iteration, never promote a red one.
Status: Shipped (core). The executor pin (model-identity tuple + recorded sampling params) ships in the typed receipt (SPEC section 8); the cross-lab advisory judge is Planned (SPEC section 6). The values below are the spec’s illustrative example, not a hardcoded default.
models: executor: provider: anthropic model_id: claude-opus-4 model_version_pinned: true # weights/build hash, not just a label runtime_version_pinned: true endpoint: "https://api.anthropic.com" sampling: { temperature: 0, top_p: 1, seed: 0, max_tokens: 4096 } # RECORDED values fail_closed_if_unresolvable: true # no resolvable version/usage -> workspace_invalid judge: enabled: true must_differ_from_executor: true must_differ_lab: true # same-lab models share reward-hack blind spots run_on: expensive_iterations # judge doubles the bill; cheap gate stays always-on advisory_only: true # NEVER promotes red->green; only vetoes green inputs: [check_log, test_result_delta, collected_set_delta, coverage_delta, mutation_signal, diff] diff_as_sandboxed_data: true # the diff is UNTRUSTED prompt-injection-bearing input on_reject: reviewer_rejectedisolation - two-zone isolation
Section titled “isolation - two-zone isolation”What it defends. The host and the credentials. A single --network none container cannot both run a cloud agent and isolate untrusted code, so two-zone isolation splits the network-bearing agent zone from the network: none exec zone. They share only a detached clone - a worktree is ergonomic, not a boundary - and the credential is minted per run, short-TTL, spend-capped, never a ~/.claude bind-mount.
Status: Shipped (core) - isolate ships the two-zone orchestration: detached-clone sandbox, per-run minted/revoked credential (0600 env-file, never on the argv), and a rendered/launched network:none exec zone plus egress-allowlist agent zone (SPEC section 7). The container engine, the auditing egress proxy, and the provider key API are operator-provided hooks (--runtime, --egress-proxy, --mint-cmd / --revoke-cmd). See isolation for the threat model.
isolation: boundary: detached_clone # worktree is ERGONOMIC, not a boundary detached_clone: clone_flags: ["--no-hardlinks", "--no-local", "--single-branch"] strip_credential_helper: true core_hooks_path: /dev/null # block .git/hooks host code execution push_gate_remote: "file://...bare" # quarantined ref namespace; a PR bot promotes to origin agent_zone: # reasoning surface: needs narrow model egress network: egress_allowlist egress_allowlist: ["api.anthropic.com:443"] # via auditing forward proxy; default deny proxy_logs_in_receipt: true fs: { rw: ["/work"], default: ro, no_home: true } # NO $HOME, NO ~/.claude exec_zone: # untrusted code-execution surface: needs NO network network: none # correct ONLY here, not around the agent fs: { rw: ["/work"], default: ro, tmpfs: ["/tmp"] } ephemeral: reset_each_invocation secrets: # ~/.claude:ro is theater (:ro blocks writes, not reads) bind_mount_credential_home: false ephemeral_credential: true # minted per run; TTL <= loop budget; revoked at end provider_side_spend_cap_usd: 10 # enforced on the KEY, not by the agent process on_credential_home_mount: credential_scope_invalid # doctor rejects -v ...:/root/.claude hermetic: env_allowlist: [PATH, HOME, LANG] pinned_clock: true seed: 0 timezone: UTC locale: C.UTF-8 tool_versions_pinned: true ephemeral_ports: true # OS-assigned :0; reject checks needing fixed ports/real DB on_egress_outside_allowlist: hermeticity_violationbudget - total, per-turn, and anomaly
Section titled “budget - total, per-turn, and anomaly”What it defends. Runaway spend. The hard cap is run-total, not only per-turn - a per-turn cap alone bounds nothing across twenty iterations. The cost model is composite (agent tokens + judge cost + per-iteration check execution), and a rolling-sigma anomaly detector catches spend-rate spikes distinct from the absolute cap.
Status: Shipped (core). The run-total budget is recorded by run --budget-usd and pinned in the receipt, and inspect-cost ships the cost model over a supplied ledger: a run-total cap (budget_exceeded) and a sigma cost_anomaly detector. Live/auto cost metering (parsing provider usage) and in-loop enforcement during run are Planned.
budget: per_turn_usd: 10 total_usd: 60 # RUN-level hard cap track_cumulative: true # persist cumulative_usd on_total_exceeded: budget_exceeded cost_model: include_terms: [agent_tokens, judge_cost, checks_per_iter_exec_cost] # not token-only baseline_from: warmup_iterations # >=3-5 samples, not n=1 ceiling_metric: p95 # a quantile, not a mean mislabeled "upper bound" anomaly: detector: rolling_sigma threshold: "mean + 3*sigma" # spend-RATE spike, distinct from the absolute cap on_anomaly: cost_anomalyescalation - a packet that blocks until ack
Section titled “escalation - a packet that blocks until ack”What it defends. The human handoff. Escalation writes a structured packet to a real channel and blocks resume until a human acks - an unacked escalation is otherwise indistinguishable from a crash. The packet carries exactly what replay consumes, so the reviewer can verify the verdict without re-running the agent.
Status: Shipped (core) - escalate writes the structured packet (--channel file|stdout) and marks the run paged, cleared by ack; it surfaces the escalation_pending halt reason (SPEC section 9). The github_issue / slack channels and the kill-the-PID actuator are Planned.
escalation: channel: github_issue # github_issue | slack | musketeer_auditor (NOT stdout) packet: [run_id, halt_reason, iteration, last_green_commit, last_n_jsonl_events, working_diff, check_log, heartbeat_ts, cumulative_usd] # == what `replay` consumes block_resume_until_ack: true # unacked escalation is indistinguishable from a crash state: none # none | paged | acked (persisted; resume won't re-page) acked_by: nullcomprehension - understanding debt as a tracked gate
Section titled “comprehension - understanding debt as a tracked gate”What it defends. Understanding debt. The runtime tracks diffs_merged_unread and halts comprehension_debt_exceeded past a threshold, cleared only by a signed ack. This is honestly a forcing and visibility gate, not proof of comprehension.
Status: Shipped - the comprehension gate ships via run --comprehension-every, halting comprehension_debt_exceeded, cleared only by a signed ack (SPEC section 9).
comprehension: review_every_n_diffs: 10 track_diffs_merged_unread: true # (diffs merged) - (diffs a human acked reading) on_debt_exceeded: comprehension_debt_exceeded ack_command: "loopexec ack --through <sha> --reviewer <id>" # signed ack -> receipt line note: "ack is a visibility/forcing gate, NOT proof of comprehension"observability - typed logs and a watchdog that reads the heartbeat
Section titled “observability - typed logs and a watchdog that reads the heartbeat”What it defends. The audit trail and liveness. The JSONL log is built with a typed encoder, never shell string interpolation (which produces invalid JSON on the first FAIL containing a quote). The heartbeat is read by an external watchdog that times out a wedged agent and emits heartbeat_stale - a heartbeat nobody polls is just a file. Every exit, including a non-zero agent exit, passes through the typed logger.
Status: Shipped (core). The typed JSONL log ships as part of the receipt work (SPEC section 8), and the external watch watchdog ships heartbeat-staleness detection, emitting heartbeat_stale (SPEC section 9). The SIGKILL-the-wedged-PID actuator is Planned.
observability: log: .loopexec/run.jsonl log_encoder: typed_json # never shell string-interpolation (invalid JSON) heartbeat: .loopexec/heartbeat watchdog: enabled: true reader: loopexec_watch # external poller; SIGTERM/SIGKILL on stall stall_timeout_s: 600 record_worker_pid: true record_phase: true # heartbeat updated AROUND long ops, not only at iter start on_stall: heartbeat_stale agent_exit_handling: capture_rc_then_branch # never let `set -e` be the exit path every_exit_passes_through_log: truereceipt - pin everything, or fail closed
Section titled “receipt - pin everything, or fail closed”What it defends. Replayable verdicts. The receipt pins everything that determines output - the model-identity tuple, recorded sampling params, an ordered context manifest, and parsed cost actuals - or it fails closed (workspace_invalid). replay verifies the recorded verdict against the recorded end-state; it does not re-run the agent. A live-LLM trajectory is not reproducible; only the verdict is.
Status: Shipped (core). The pinning fields ride the typed receipt + state work, and attest (HMAC-sign + --verify), replay (verify offline), and reexecute (live re-run) all ship today (SPEC section 8). cost_actuals is recorded from flags; live cost metering is Planned. See receipts.
receipt: model_identity_tuple: true # {provider, model_id, version, quant, runtime, endpoint} record_sampling_params: true # temperature/top_p/seed/max_tokens as values context_manifest: true # ordered [{path, content_sha256}] -> reconstructable prompt cost_actuals: true # {prompt_tokens, completion_tokens, usd} from provider usage determinism_confidence: emit # the achieved probe bound store_prompt_response: content_plus_sha fail_closed_on_unresolvable: workspace_invalid verify_not_rerun: true # "replay" VERIFIES the verdict; it does not re-run attest: sign # sign receipts so provenance is checkablestate - durable, resumable fields
Section titled “state - durable, resumable fields”What it defends. Resumability. .loop_state.json carries the fields a run needs to resume: phase, iteration, last green commit, open failures, cumulative spend, the determinism probe result, the metric-integrity snapshot, the model pin, and escalation state. On resume the runtime revalidates the determinism probe and metric integrity before the first agent call and halts on drift.
Status: Shipped - durable, resumable state (SPEC section 8).
state: schema_version: 1 fields: phase: null iteration: 0 last_green_commit: null open_failures: [] cumulative_usd: 0.0 baseline: { model_calls: null, tokens: null, top_error_type: null } determinism_probe: { runs: null, stable: null, confidence: null, checked_at: null } metric_integrity: { check_cmd_sha: null, test_tree_sha_at_last_green: null } model_pin: null escalation: { state: none, channel: null, ref: null, acked_by: null } revalidate_on_resume: [determinism_probe, metric_integrity] # halt on drift before first agent callPutting it together
Section titled “Putting it together”These fifteen blocks compose into a single loop.yml. loopexec is specified to scaffold a versioned loop.yml alongside its durable state via init (SPEC section 10); init ships today and creates the .loopexec/ directory, but the versioned loop.yml scaffold is Planned. The mechanisms these blocks declare are enforced by run today through its CLI flags (--check, --exec, --failures-cmd, --integrity-cmd, the receipt-pinning flags, and the rest) - the loop.yml file is the Planned declarative form of that same surface, not yet the input format. Each on_violation / on_* field names a halt reason - the string is the stable integration contract, and the exit code is the coarse class CI branches on. Every reason is computed from observed state; --halt-reason is a hidden test fixture only.