The model says the task is done.
TASK.md checks the real system.
A task's completion is decided by code against a live system — Acceptance Criteria ∧ Definition of Done — not by the executor's word. Same contract, same verdict, whatever model ran it.
"Done" from an agent tells you nothing
Delegate a task to an AI agent and it will report success. Nothing in that report tells you whether the page actually loads, the endpoint returns the right value, or the tests actually pass — the agent is judging its own work.
So a human re-checks by hand, every time: opening the browser, re-running the tests, reading the real output. TASK.md replaces that re-check with a contract a machine verifies against the running system.
A user story with machine-checkable acceptance
Each Given/When/Then step binds to a typed gate from a closed catalog — no free-form assertions. The contract declares the facts to hold, never the algorithm, so the verdict is independent of the model that implements it.
It reads the running system, never the executor
Gates check the live world: the browser DOM over CDP, an HTTP endpoint's JSON, a command's exit code. The verdict comes from what the system actually does — never from asking the agent "did you do it?".
On a FAIL, the executor gets the gap (expected vs observed), never the assertion itself. It retries with that feedback; a deterministic stop and an escalation ladder (cheap → capable model) close the loop.
You can't hide the gate from a shell
The sharpest empirical result — found by breaking it. A gate only verifies if the executor can't reach it. An executor with a shell defeats every attempt to hide the test by concealment:
A held-out test in a sibling directory — the executor located it and copied the expected values, passing without implementing.
Four rounds of whack-a-mole: on a shared filesystem a shell always reaches the gate material (test, env, audit logs, source).
A container/WSL with no shared mount. The contract must declare its isolation posture — and the lint catches the incoherent ones.
By the numbers, as of this file
Spec v0.1 with a working PoC engine. It's a demonstrator, honestly labeled — see the repo for what's real vs. mock.
What the gate decides, what stays out
The honest boundary is part of the method — and the leverage is only as good as the gate.
The gate decides
- Whether the observable outcome matches the acceptance criteria, against the live system
- Whether a command, endpoint or page reaches the declared state
- Whether the contract itself is well-formed and its isolation posture is coherent (lint)
- The same verdict for any executor model — cheap or capable
Stays out, declared
- Whether the oracle is well-designed — a weak gate lets weak work pass; strength is a separate concern
- Whether a result "reads well" — tone and taste stay a human's call
- The executor's reasoning. TASK.md verifies the outcome a model produces, never how it thinks.
One DNA, six planes
Same DNA across the family — verifiable layer, judgeable layer, deterministic validation — each on a different plane; I verify the dynamic result against the live system.
Read the contract yourself
MIT-licensed, PoC v0.1. Every claim on this page is checked into the repository that backs it — including what's still a mock.