Testing & Evaluation Articles
6 articles
How Much Testing Is Enough for Your AI Agent?
Code coverage doesn't apply to AI agents. Here's a framework for thinking about evaluation coverage: how many scenarios you need, what distribution to target, and how to know when you've tested enough.
Your Agent Aced the Benchmark. Production Disagreed.
We scored 92% on GAIA. Production failed the same scenarios 25% of the time. Here's which AI agent benchmarks actually predict deployed performance, why most don't, and what to measure instead.
Agent Drift: Why Your AI Gets Worse the Longer It Runs
AI agents silently degrade over long conversations. Research quantifies three types of drift and shows why point-in-time evals miss them entirely.
How MCP Tool Descriptions Break Your Agent
New research shows 97% of MCP tool descriptions have quality issues that hurt agent accuracy. Here's what the smells look like, why they matter, and how to fix them.
MCP tool description drift: the silent failure nobody alerts on
Edit an MCP tool description for clarity, lose 8% routing accuracy, and the eval suite stays green. How to detect, gate, and roll back the drift.
The MCP Marketplace Problem: Why Standardized Integrations Need Standardized Testing
5,800+ MCP servers, 43% with injection flaws. Standardized protocol doesn't mean standardized quality. Why every MCP integration needs automated testing.
Changelog MCP y notas para devs
Actualizaciones cortas y ocasionales sobre la spec MCP, nuevas funciones de Protobox y patrones que vemos en producción. Sin marketing innecesario.
