Source: Autonomous AI Engineering Harnesses — based on Anthropic Labs research
Across multiple production AI deployments, two failure modes appear so consistently they deserve names. Understanding them — and their architectural fix — is probably the most practically useful thing I learned in the engineering-focused portion of this course.
Context Anxiety: as a model approaches the perceived limit of its context window, it begins to rush — truncating reasoning steps, skipping verifications, producing lower-quality outputs to get to a conclusion before it “runs out.” This behaviour is irrational (the model doesn’t actually run out of context in a harmful way) but consistent. Models seem to have learned from training data that running out of space is bad, and they react to approaching context limits by cutting corners. In long-horizon agentic tasks, this is one of the most reliable predictors of output quality degradation.
Self-Evaluation Leniency: when asked to evaluate its own work, an LLM systematically over-rates it. The same training process that makes models helpful and agreeable also makes them generous judges of their own outputs. A model asked “did you complete this task correctly?” will almost always say yes, even when the answer is no. In agentic systems where self-evaluation is used as a stopping criterion, this compounds disastrously — the agent declares success and stops when it should have continued.
The fix for both, based on Anthropic Labs research, is architectural: separate the Generator from the Evaluator. A separate Evaluator agent — specifically tuned to be skeptical rather than generous — checks outputs against original requirements. It turns out it’s much easier to tune a standalone Evaluator to be strictly critical than it is to get a Generator to accurately critique its own work. Specialisation beats self-reflection. This is a principle worth internalising well beyond AI engineering.