Agents that sign off their own work

Haiqing Li, Junzhou Huang and eight co-authors argue that language-model agents share a structural flaw: the model that does the work is also the one that decides the work is finished. Task instructions, guidelines, output schemas and reusable skills all sit in the context of that same model, so nothing independent decides whether a specification has actually been met.

They name two resulting gaps. In the understanding–execution gap, a requirement is understood but not satisfied when the work is done. In the state–authority gap, an agent's claim of completion does not establish that the required state exists. To measure both, the authors extracted 509 task directions from SkillsBench using only what the agent itself could see. Across seven models, only 79.6% to 86.4% of those directions were satisfied, and completion-claim rates exceeded the official evaluator's pass rates by 28.7 to 37.9 percentage points.

Proposals versus state

Their fix, SpecHarness, separates what an agent proposes from what counts as established. Agents may plan, act and request completion, but only admissible evidence from qualified providers can commit specification-governed state. Visible specifications are compiled into source-linked obligations with versioned state; verifiable requirements are checked at runtime, while ambiguous or subjective ones stay advisory.

Across all 87 SkillsBench tasks, the macro pass rate rose from 61.1% to 73.1%, the understanding–execution gap fell from 17.4% to 9.3% and the state–authority gap from 32.8% to 12.8%. Every model improved: GPT-5.6 Sol, Claude Fable 5, Gemini 3.1 Pro, Kimi K3, GLM-5.2, Qwen3.7-Max and DeepSeek-V4-Pro. The largest pass-rate gain, 17.2 points, went to Gemini 3.1 Pro.

The authors are explicit about the limit: the guarantee covers grounded mandatory obligations, not the full natural-language specification. The harness holds an agent to what can be checked and leaves the rest as guidance.