#software-testing
3 articles
When Code Becomes Cheap, Evidence Becomes the Product
Coding agents are increasing the volume of plausible software faster than organizations can safely absorb it. Engineering advantage will belong to teams that redesign specifications, tests, and review around machine-scale output.
The Agent Benchmark Is Not the Product
AI agents fail across trajectories, not isolated answers. Teams that evaluate only the final output are measuring the least informative part of the system.
The Benchmark Is Not the Product: Why Agent Evals Must Become Executable Specifications
Agent benchmarks can tell you whether a system cleared an obstacle course. They cannot tell you whether it will behave sensibly inside your company—unless evaluation becomes part of the product’s architecture.