Reward Hacking & the Metric-Integrity Gate

A check_fixpoint loop optimizes one number: the check’s exit code. Reward hacking is what happens when weakening that measurement is cheaper than satisfying it. The agent finds the lowest-cost path to green. Sometimes that path runs through the code; sometimes it runs through the test.

This is the defense the metric-integrity gate exists to provide. Per SPEC section 11, that gate’s core ships - collected-set monotonicity via run --integrity-cmd, evaluated before the green check (guards dominate success) - while its remaining layers (assertion-count, manifest hash, coverage floor) are Planned. Read those layers as “loopexec is specified to,” not “loopexec does.”

Reward hacking is optimization, not malice

There is no intent to model. An optimizer pointed at a metric moves downhill on that metric. If the cheapest descent edits the thing being measured rather than the thing being built, the optimizer takes it - not because it wants to cheat, but because the gradient pointed there and nothing stopped it.

So the question is never “is the agent honest?” It is “is green a proof of work, or a proof that the measurement was weakened?” A loop that cannot tell those apart will converge confidently on a lie.

The objective is the only thing the loop respects

An agent’s self-report is an echo (SPEC section O1). The check is external precisely so judgment cannot promote a red loop to green. The same property makes the check a target: it is the one signal the loop will reshape reality - or reshape the measurement - to satisfy.

The weakest layer: “don’t weaken the tests”

The first instinct is to put the rule in the prompt: don’t touch the tests, don’t lower the thresholds, don’t skip cases. This is the weakest layer there is.

A prompt instruction is advisory, unenforced, and untestable. It competes with the reward signal and loses, because the reward signal is checked every iteration and the instruction is checked never. Treat “don’t weaken the tests” as documentation of intent, not as a guard. A guard is something the runtime computes, against state, before it lets the loop exit.

The sieve: git diff --quiet -- test/

The obvious upgrade - fail the iteration if anything under test/ changed - looks like a gate. It is a sieve. It catches the one hack nobody serious would try (editing a tracked file under the literal path test/, in the working tree) and lets the cheap ones through.

What it misses	The cheaper hack
Source-side special-casing	if (NODE_ENV === 'test') return EXPECTED; - the source lies, the test is untouched
Config / CI weakening	Lower the coverage threshold, edit the runner config or CI yaml - none of it lives under test/
Collection shrink	Delete or skip cases so the suite asserts less; the green is over a smaller world
Untracked overrides	A new conftest.py or fixture override never appears in git diff at all
Co-located tests	foo_test.go, *.spec.ts next to source - not under the path test/
Lockfile swap	Downgrade a test dependency in the lockfile so a strict assertion goes soft

Two structural failures make it worse than the table suggests:

• Bypassed by git add / commit. git diff reads the working tree against the index. Stage the change and the diff comes back clean; commit it and “last turn” is wrong. The guard reports green on a tree it never inspected.
• Mis-ordered after the green check. This is the highest-leverage one-line bug. If the loop runs if check passes: exit 0 before the test-diff guard, an agent that weakens a test to turn the suite green exits as success and the guard never runs. The defense is bypassed by the exact event it exists to catch.

A guard evaluated after success is not a guard

The diff-against-working-tree check is blind to staged, committed, and untracked changes; to source, config, CI, and lockfiles; and - fatally - it can be wired to run on a code path that green has already escaped. It is a checkbox, not a boundary.

Guards dominate success

The ordering bug has a one-rule fix, and the SPEC makes it normative: guards dominate success (G-before-S, SPEC section 4.4). Guards are evaluated against the immutable baseline t0 before any green branch can fire. A check that went green because a guard was violated MUST classify as the guard’s halt reason, never success_condition_met.

each iteration (one ordered transaction):
  1. evaluate the metric-integrity guard against the immutable t0
  2. if violated -> revert to t0, HALT metric_integrity_violation  (exit 13)
  3. ONLY THEN may a green check declare success_condition_met     (exit 10)

Compare the sieve, which inverts steps 1 and 3:

# WRONG: the green branch exits before the guard runs
if npm test --silent; then
  git diff --quiet -- test/ || echo "tests touched"   # unreachable on green
  exit 0
fi

Success is never the first thing the loop checks. It is the last thing the loop is allowed to conclude, and only after every guard has passed.

The fix: the metric-integrity gate

Replace “test files unchanged” with monotonic-metric invariants computed against an immutable baseline t0 captured at loop start. The gate does not ask “did a file change?” It asks “did the measurement get weaker?” - and weakness is the thing reward hacking has to produce, by every route in the sieve table.

The invariants (SPEC section 6):

• Collected-set monotonicity - the set of collected test IDs MUST NOT lose a member (pytest --collect-only / jest --listTests / go test -list). Kills collection shrink and skips.
• AST assertion non-decrease - test count and AST-level assertion count MUST NOT drop. Catches gutted bodies that keep the function but delete the expect, which a file-existence check misses.
• Protected-manifest hash - a stable hash over the full test-determining surface: runner config, coverage config, CI yaml, the test-dependency lockfile. Closes the config / CI / lockfile routes.
• Read-only frozen tests - tests SHOULD run from a read-only mount frozen at t0. The agent physically cannot edit or commit the tests it is graded by.
• Canary mutation - a known-bad mutant injected into changed source MUST turn the suite red. A green canary means the check never exercised the change - the false fixpoint - and halts check_inadequate (exit 15), the adequacy cousin of this gate. See determinism for why adequacy is a separate property from a stable confidence bound.
• Coverage floor - coverage MUST NOT drop below the t0 floor.

Crucially, the gate diffs against t0, not the index or HEAD, and covers working-tree + staged + committed + untracked changes at once. git add no longer launders anything. On violation it reverts hard (git reset --hard t0 && git clean -fd) and halts metric_integrity_violation - exit class 13, integrity, owned deterministically by LoopExec.

Source-side cheats fail behaviorally, not by diff

if (NODE_ENV === 'test') return EXPECTED; touches no test and edits no config, so no manifest catches it. It is caught downstream: the canary mutation and coverage-floor checks are behavioral - they ask whether the check still exercises and still rejects bad code - and the cross-lab judge reads the result delta. The gate is layered for exactly this reason.

The judge: advisory, cross-lab, behavioral

The deterministic gate catches everything a number can decide. Some reward hacks are semantic - a diff that is persuasive and wrong. For those, the contract adds an advisory judge (SPEC section 6, Planned), and the word advisory is load-bearing:

• Advisory only. The judge can veto a green iteration (reviewer_rejected, exit 16; or reward_hacking_detected, exit 13). It can never promote a red loop to green. Judgment is never the stop oracle - it sits on top of the deterministic metric_integrity_violation, never replacing it.
• Cross-model and cross-lab. A model grades its own self-deception patterns poorly, and same-lab models share correlated reward-hack blind spots. The judge MUST differ from the executor in model and lab.
• Behavioral, not textual. It receives a behavioral oracle - check log, result delta, collected-set delta, coverage and mutation signal - not just the diff. The diff is treated as untrusted, prompt-injection-bearing data, handled inside the two-zone isolation boundary, never as instructions.

This is the ownership line from SPEC section 5: LoopExec owns everything a number can decide; the judge (Musketeer) owns only what a number cannot express, layered above the deterministic gate; humans own the irreducible ack. The guard’s verdict and the judge’s veto both land in the receipt, so the result is a replayable verdict - you can verify offline why a green was rejected without re-running the agent.

Status

Layer	Status
Guards-dominate-success ordering (G-before-S, section 4.4)	Shipped (inside the iterating run)
Computed halt-reason framework + exit-code classes (section 5)	Shipped
Metric-integrity gate - collected-set monotonicity (Shipped via --integrity-cmd) plus AST assertion-count, manifest hash, frozen tests, canary, coverage floor (Planned) - and the metric_integrity_violation it computes	Shipped (core)
Cross-lab advisory judge (section 6)	Planned

The honest claim today: the gate’s core ships as of v0.2.0. run --integrity-cmd enforces collected-set monotonicity before any green branch (guards dominate success), reverts on violation, and halts metric_integrity_violation (exit class 13) - all computed from observed state. The remaining layers (AST assertion-count, manifest hash, frozen-test mount, canary mutation, coverage floor) and the cross-lab advisory judge are Planned: read those as “loopexec is specified to,” not “loopexec does.” For the full status matrix, see SPEC section 11; for how a tripped guard surfaces as an exit code, see halt reasons; for where this sits in the discipline, see loop engineering.

---

Maintained by Justyn Clark Network