A green test should mean your agent fixed the code — not that it edited the test to match the bug. PromptWheel is the referee that tells the difference: it reverts the test and config edits, re-runs your gate, and sees if the win survives. Deterministic, no LLM — and the gate's own files count as evaluator edits, never as wins.
npx promptwheel@latest run
PromptWheel re-proves the win from the agent's source edits alone. The green came from the test file, not the code — so it evaporates, and the verdict is GAMED.
# code is still broken; the "agent" edited the TEST to expect the bug $ npx promptwheel@latest run --base HEAD~1 --head HEAD PromptWheel base → head ▲ tests_pass 0 → 1 (+1, improved) [guard✓, high] 🚩 GAMED — the "win" changed zero production-source files — only test/config/grader/golden VERDICT: GAMED — a metric "improved" by editing the goalposts, not the source $ echo $? 2
A real fix that survives the source-only re-run returns PASS / exit 0. Copy-paste the full self-contained repro from the README.
Deterministic, explainable, and free — every flag is reproducible in seconds with a human-readable reason.
For any base→head, it re-runs your metric commands — tests, lint, tsc, coverage, bundle, evals — before and after, in isolated git worktrees. It never touches your working tree, and refuses to trust a delta inside the measurement noise band.
It splits the diff into production source vs. {test, config, grader, golden}, rebuilds a clean base worktree with only the source slice applied, and re-runs the gate. If under half the gain survives — or the win touched zero source files — it was gamed.
You get a structured verdict and an exit code your harness can gate on: a real win, a guarded regression, a gamed metric, or an inconclusive guard that measured nothing. It prints exactly which edits it reverted and how much of the gain held.
It never reads the agent's trajectory and never calls an LLM — so the verdict can't be obfuscated, costs nothing, and is identical on every re-run. And it catches what a read-only test dir or a PreToolUse hook can't: a win moved into the source — a hardcoded value, a mocked grader, a swallowed error, a relaxed config — on already-committed diffs, on any metric.
Hardened against a 13-scenario labeled benchmark, a sweep across ~1,000 public TS/JS/Python/Go/Rust repos, and a 136-commit usage replay — all rerunnable in corpus/ and bench/.
PromptWheel is the verifier, not the loop driver — a contestant can't referee itself.
while promptwheel improve \
--attempt "$AGENT"; do :; done
# stops on plateau or regression;
# only keeps turns that earned itnpx promptwheel init
npx promptwheel run --working
# exit 0 real · 1 regression · 2 GAMED
# the audit the agent can't skipLong-time web dev sharing what I build and learn. PromptWheel included. No spam.