Skip to content
Rush Commerce
AI & Automation3 min read

OpenAI embedded assessments: 4 outside evaluators get inside access

OpenAI's embedded assessments give Apollo, METR, Transluce and Goodfire access to training runs, but OpenAI can limit the evidence. Enforce agent limits yourself.

OpenAI announced its first phase of embedded assessments on November 16: four outside safety groups (Apollo Research, METR, Transluce and Goodfire) get access inside OpenAI's systems to test its models. The headline question is whether a model will quietly work around the controls meant to contain it. That matters to any business running AI agents, because the answer comes back under terms the vendor helped write.

What actually happened

Per TechCrunch's report, which cites OpenAI, Apollo and Goodfire:

  • Apollo Research studies "subversion": whether models try to undermine their own alignment or control mechanisms, or those of later models. Its four questions are whether training rewards that behavior, whether tests can bring it out, whether it happens in OpenAI's internal deployment, and whether the reasoning behind it would be detectable. Apollo's pilot starts this week.
  • METR looks for dangerous or unintended behavior that OpenAI's own tests missed, and METR mostly picks what to look at. A second METR test, of whether OpenAI's monitors catch that behavior, has no agreed scope yet.
  • Transluce checks whether OpenAI's post-training safety evaluations are good enough. In 2027 it plans to test how OpenAI uses those evaluations to decide whether a model ships.
  • Goodfire embeds interpretability researchers to watch training runs from the inside.
  • Money. Apollo and Goodfire are paid. METR and Transluce are not, but may take compute or API credits.
  • Limits. Apollo gets training code, datasets, reinforcement-learning rollouts and intermediate checkpoints, but only for agreed claims, models and time periods. It can publish its conclusions, but OpenAI can limit the evidence it shows to back them up. OpenAI may withhold in-scope material for security, IP or technical reasons, but says it won't withhold something only because it looks bad. Apollo can talk only to staff OpenAI authorizes.

TechCrunch notes the deal follows OpenAI's firing of three researchers who allegedly shared confidential information with METR. Redwood Research CEO Buck Shlegeris said the agreements fall short of employee-level access.

Why embedded assessments matter for your business

This is real progress. Outside groups looking at training runs is better than groups testing only a finished product. But read the terms. The vendor sets the scope, picks the staff who can talk, and can limit the evidence behind the conclusions. We made the same point in October when Reuters reported the labs wanted a say in evaluator reports.

The research question is the operator's question too. If a model can work around controls written in its own prompt, those controls don't hold. So don't put your limits in the prompt. Put them where the model can't edit them:

  • Scoped API keys, with read-only access by default.
  • Spend caps and rate limits at the gateway, not in the instructions.
  • A person approves anything that moves money, deletes data or emails a customer.
  • Logs the agent can't write to, so you can check what it actually did.

That setup works no matter what an evaluator finds, and no matter which model you swap in next quarter.

Key takeaways

  • OpenAI announced embedded assessments with Apollo Research, METR, Transluce and Goodfire on November 16
  • Apollo studies whether models try to undermine their own control mechanisms; its pilot starts this week
  • Apollo and Goodfire are paid; METR and Transluce are not
  • Evaluators can publish conclusions, but OpenAI can limit the evidence and picks which staff they talk to
  • Put your agent limits in keys, gateways, approvals and logs, not in the prompt

Your agent's limits should live outside the model. We build AI agents with scoped credentials, spend caps, human approval steps and logs the agent can't touch, on a stack you own. See how we build agents, or have us review the agents you already run.

Sources: TechCrunch.

  • #openai
  • #ai-safety
  • #ai-evaluations
  • #ai-agents
  • #vendor-risk
TR

Tommy Rush — Founder, Rush Commerce

Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More

Get The Rush Report weekly — one email, zero fluff.