Generator-Evaluator Pattern

What is the Generator-Evaluator Pattern?

The generator-evaluator pattern is an agent architecture where the AI system that creates work is separated from the AI system that judges it. Inspired by Generative Adversarial Networks (GANs) in machine learning, this pattern addresses a fundamental limitation of AI agents: they are unreliable self-critics. An agent asked to evaluate its own output will confidently approve mediocre work, but a separate evaluator agent — with fresh context and explicit grading criteria — catches errors the generator misses.

In practice, the generator produces an output (code, design, document), and the evaluator tests it against defined criteria, providing structured feedback. The generator then iterates based on that feedback. This loop typically runs 5-15 cycles before the output meets quality thresholds.

Key Characteristics

• Role separation: The generator never grades its own work — a separate agent with independent context handles evaluation
• Explicit criteria: Quality is broken into gradable dimensions (e.g., design quality, originality, craft, functionality) rather than a single pass/fail
• Active testing: The evaluator doesn’t just read output — it interacts with it (e.g., via Playwright for web applications)
• Hard thresholds: Each criterion has a pass/fail bar, preventing the generator from shipping work that’s “good enough” in aggregate but weak in specifics
• Iterative refinement: Multiple generation-evaluation cycles produce compounding quality improvements

Why the Generator-Evaluator Pattern Matters

This pattern transforms autonomous AI work from “hope it works” to “verified it works.” Anthropic’s research showed a solo agent run produced non-functional features in 20 minutes ($9), while the generator-evaluator harness delivered a working application in 6 hours ($200). The cost increase is significant, but the quality difference is the gap between a demo and a product.

For organizations deploying AI agents in production, this pattern is the clearest path to reliable autonomous output. It also scales naturally — the same evaluator criteria can grade work from any generator, making it possible to swap models or upgrade components independently.

Historical Context

The concept draws from GANs (Goodfellow et al., 2014), where a generator network and discriminator network improve each other through adversarial training. Anthropic’s Applied AI team adapted this principle for agentic software engineering, replacing neural network training with structured feedback loops between LLM-powered agents. The pattern was formalized in Anthropic’s March 2026 engineering post on harness design for long-running applications.

Related Reading

• Multi-Agent Systems - The broader architectural family
• Agentic Coding - Where this pattern is most commonly applied
• Prithvi Rajasekaran - Researcher who formalized this approach