Critic

Anthropic’s engineering team published a blog post this week about how they design long-running agent systems. The main pattern they use is from GANs: separate the agent doing the work from the agent judging it. They call it the generator-evaluator pattern. Two things clicked when I read it.

The first is context anxiety. Their blog mentions it directly. When context gets long, the model starts rushing to wrap up. It gets nervous, like it can feel the walls closing in. I’ve been seeing this a lot since switching to 1M context on Claude Opus. The model has all this room now, but sometimes it still acts like it’s about to run out. Old habits from training, maybe. It starts trying to wrap things up way before it should. The 1M window helps, but the anxiety is still there in the model’s behavior.

The second thing is self-evaluation. I discovered this a long time ago: if a model grades its own work, it almost always scores itself too high. Every time. Even when the grading criteria are enforced by code, the scores still drift upward. The model is both the student and the teacher, and it’s a generous teacher.

Anthropic’s solution comes from the GAN idea. Have a separate model instance act as the evaluator. Don’t let the generator grade itself. In their system, the evaluator uses Playwright to actually test the code like a real user would, clicking through the app, checking if things work. That’s the key part: the critic has to be isolated from the generator, with its own context and its own job.

I tested this on a side project I’ve been tinkering with in my spare time. The setup borrows from Karpathy’s autoresearch pattern, and I added this harness idea of an isolated “critic” subagent. The critic runs separately, doesn’t share context with the main agent, and its only job is to find problems.

I put the agents there overnight. When I checked in the morning, the output was better than what the previous 4 days of work had produced.

Anthropic’s numbers tell the same story: $9 for a broken solo run vs $200 for a working app from the harness. That ratio is going to shrink as token costs drop, and the quality gap is only going to widen as models get better at being critics.

We’re in the agent era now. Figuring out how to control and coordinate agent work well is kind of the main problem. The GAN pattern, autoresearch, critic subagents, these are early discoveries. There will be a lot more.

Next thing I’m going to try is combining all of these and adding more than one critic. Different critics checking different dimensions, like having a small review panel instead of a single reviewer.

If that works, the product I’m working on could get a real upgrade from this.