Automated AI research beat 28 human safety researchers

Anthropic published results on August 28 from an automated AI research loop: Claude reads the literature, proposes a training method, runs it, scores it against a benchmark, and iterates. Pointed at ten categories of alignment failure, it found a fix for every one. It also outscored 28 human safety researchers. The interesting part for anyone running a business is not the safety result. It is what happens when you hand a machine an objective function and walk away.

What actually happened

In Automated researchers can reliably mitigate alignment failures, Anthropic describes targeting ten categories of misaligned behavior, each with roughly three to five benchmarks. The automated researcher improved the target metric on all ten without degrading the capability tests they ran, closing between 26% and 96% of the measured safety gap depending on the failure.

Against people: 28 human safety researchers submitted methods. On deception, where six experienced researchers competed, Claude's best method scored about 20% better than the best human proposal. TechCrunch reported the loop ran at roughly $4 per hour in inference against $150 per hour for a human researcher, and beat the humans within about six hours on average.

The methods also generalized upward — they still worked on models up to 4.7 times larger than the ones used during the search.

Anthropic is direct about the limits. The failures studied are narrow next to production behavior, capability testing covered a predetermined set, the evaluations are proxies, and the human researchers were not allowed to iterate the way Claude did.

Why an automated research loop matters for your business

You already have the shape of this. Propose, run, score, keep the winner. That is a pricing test, a subject-line test, a retrieval-config sweep, a prompt tuned against a labeled set of your own support tickets. What Anthropic demonstrated is that the loop closes without a person in the middle when the scoring step is real.

The scoring step is the whole job. An automated loop will optimize whatever you measure, relentlessly, at $4 an hour. Point it at a metric that is a bad proxy for the outcome you want and it will hit that metric anyway. Writing the eval is now the expensive, human, judgment-heavy work. Running it is not.

Start where you can already grade the answer. Extraction accuracy against invoices you have already processed. Classification against a month of tickets you closed by hand. Those have ground truth sitting in your database. Build the eval before you build the agent.

Sources: Anthropic, TechCrunch.