AI labs wanted a say in evaluator reports: run your own evals

Independent AI safety evaluations are less independent than the press releases say. According to a Reuters exclusive published October 13, Anthropic and OpenAI sought significant control over what outside evaluators can publish about their models. Both companies had publicly endorsed evaluator independence last month. If your team picks AI vendors partly on their safety reports, know who had edit access before you read one.

What actually happened

Two people familiar with the talks told Reuters that the labs were negotiating terms with embedded evaluators, meaning outside reviewers who work inside the lab. The labs' stated worry is trade secrets: an embedded reviewer could carry proprietary information to a rival. Options discussed included lab staff watching evaluation teams as they work, and barring an evaluator embedded at one lab from working at another at the same time.

The timeline matters. Last month, Anthropic CEO Dario Amodei said external evaluators should be able to publish key findings without editorial control. OpenAI followed and called for "editorial independence." Yet as recently as October 5, both labs reportedly still wanted strict control. By October 8, Anthropic's talks with some evaluators had moved on, one source said.

Both companies pushed back on the framing. Anthropic told Reuters that "significant control" does not describe all its discussions. It says evaluators it has worked with before control their methods, conclusions and what they publish. For new evaluator firms, it keeps a right to comment on and approve reports. OpenAI said it does not seek approval of full reports. It redacts confidential information, and evaluators can disclose when major redactions were made.

Reuters also described a real case. METR and Redwood Research edited a report on an incident involving OpenAI-powered agents after OpenAI's feedback. They said nothing important to their conclusions was removed.

Why it matters for your business

There is no agreed standard for how these reviews work. The White House accord signed last month names independent evaluation as one of four safety measures, but it does not say how evaluations should be run. Until a standard exists, a vendor's safety report is a document the vendor helped shape. That doesn't make it false. It makes it marketing-adjacent.

You don't need a lab's permission to test the model you actually deploy. Build a small eval set from your own work: 50 real support tickets, 30 invoices, the 20 edge cases that broke your last automation. Run every model candidate against it before you switch, and run it again when the vendor pushes an update. Your results have one stakeholder, and it's you.

Sources: Reuters via Euronext.