Most AI benchmarks treat success as a destination: did the model produce the accepted answer, pass the tests, or complete the task? That framing made sense when models generated isolated responses. It is dangerously incomplete for agents that edit repositories, invoke tools, alter databases, and make decisions across dozens of steps.
An agent can arrive at a passing result through a brittle sequence that no engineering team would accept in production. It may overwrite a configuration file, install an unnecessary dependency, suppress a failing test, or leave temporary credentials in a log. If the final grader examines only the expected output, the agent receives credit while the organization inherits the mess.
The research community has already moved toward more realistic work. SWE-bench replaced toy code exercises with issues drawn from real repositories, requiring systems to understand codebases and coordinate changes across files. Research from METR on task-completion horizons asks how reliably models can perform work that takes human professionals increasing amounts of time. Both directions matter. But duration and completion still leave an essential property undermeasured: recovery.
Long tasks are chains of recoverable mistakes
Competent engineers rarely execute a substantial task without a wrong turn. They misread an interface, discover an undocumented constraint, or learn from a test that their mental model was incomplete. What distinguishes reliable work is not the absence of error. It is the ability to notice error before it spreads, preserve evidence, return to a known state, and revise the plan.
Agents need the same discipline. A realistic evaluation should introduce recoverable disturbances: a flaky test, a tool response that is technically valid but stale, a dependency whose latest release breaks compatibility, or an ambiguous instruction that conflicts with repository conventions. The benchmark should then observe whether the agent recognizes the inconsistency and responds proportionately.
This changes what counts as a successful trajectory. An agent that immediately guesses the intended fix may look efficient, but an agent that checks assumptions, creates a reversible patch, runs a narrow test, detects a regression, and rolls back may be far safer. The latter could consume more tokens and time while demonstrating more deployable intelligence.
That distinction is especially important because agent behavior is path-dependent. A poor early decision changes the state encountered later. Once an agent has modified files or external systems, subsequent reasoning occurs inside a world partly created by its own actions. Static question-answer grading cannot represent this feedback loop. As Anthropic’s discussion of multi-turn agent evaluations emphasizes, useful assessments must include tools, environments, state changes, and the agent loop itself.
A recovery score needs more than a final test suite
We would evaluate recovery across at least four dimensions. First is detection latency: how many actions occur between introducing a problem and recognizing it? Second is blast radius: how much unrelated state changes before containment? Third is reversibility: can the agent restore a known-good state without concealing evidence or damaging valid work? Fourth is adaptation: does it update its plan, or merely repeat the same action with slightly different wording?
These dimensions should be recorded from the trajectory, not reconstructed from the final answer. The harness needs snapshots, tool-call logs, explicit state boundaries, and graders that distinguish productive exploration from reckless mutation. Some tasks should contain traps that a careful agent avoids; others should require an initial failure so the benchmark can observe what happens next.
A useful suite might include scenarios such as:
- an apparently relevant test that passes for the wrong reason;
- a migration that succeeds locally but violates a rollback requirement;
- a permission error that tempts the agent to broaden access unnecessarily;
- conflicting documentation where repository behavior is the stronger evidence;
- an interrupted task resumed from an incomplete checkpoint.
None of these is exotic. They are ordinary features of engineering work. Their absence from evaluation rewards agents optimized for clean rooms that do not exist.
Efficiency should be risk-adjusted
Leaderboards often encourage a single scalar score because rankings are convenient. Recovery resists that compression, and that is a virtue. Two agents with the same completion rate can have radically different operational profiles. One may fail early and visibly. Another may fail rarely but contaminate surrounding systems when it does. A buyer should not have to infer that difference from a demo.
Cost metrics also need reinterpretation. The cheapest successful run is not necessarily the most economical system. Checkpoints, verification calls, and narrow probes add expense, but they can reduce the expected cost of cleanup. Evaluation should expose that trade: compute spent on caution versus damage avoided through caution.
This is not an argument for agents that endlessly ask permission. Excessive hesitation is another failure mode. The target is calibrated autonomy: act freely within reversible boundaries, pause when consequences become asymmetric, and recover intelligently when evidence invalidates the plan.
The industry will eventually stop being impressed that an agent can finish a ticket. Completion is becoming table stakes. The harder and more valuable question is whether the system remains legible and repairable when the task stops cooperating. Benchmarks that measure recovery will tell us far more about which agents deserve access to real work.
Advertisement
