Testing & Evaluation Articles
6 articles
How Much Testing Is Enough for Your AI Agent?
Code coverage doesn't apply to AI agents. Here's a framework for thinking about evaluation coverage: how many scenarios you need, what distribution to target, and how to know when you've tested enough.
Your Agent Aced the Benchmark. Production Disagreed.
We scored 92% on GAIA. Production failed the same scenarios 25% of the time. Here's which AI agent benchmarks actually predict deployed performance, why most don't, and what to measure instead.
Agent Drift: Why Your AI Gets Worse the Longer It Runs
AI agents silently degrade over long conversations. Research quantifies three types of drift and shows why point-in-time evals miss them entirely.
How MCP Tool Descriptions Break Your Agent
New research shows 97% of MCP tool descriptions have quality issues that hurt agent accuracy. Here's what the smells look like, why they matter, and how to fix them.
MCP tool description drift: the silent failure nobody alerts on
Edit an MCP tool description for clarity, lose 8% routing accuracy, and the eval suite stays green. How to detect, gate, and roll back the drift.
The MCP Marketplace Problem: Why Standardized Integrations Need Standardized Testing
5,800+ MCP servers, 43% with injection flaws. Standardized protocol doesn't mean standardized quality. Why every MCP integration needs automated testing.
MCP changelog and dev notes
Short, occasional updates on the MCP spec, new Protobox features, and patterns we've seen in production. No marketing fluff.
