Nadella's AI emergency brake: put controls outside the model

On Saturday morning, Microsoft CEO Satya Nadella posted an essay on X that says frontier models need an AI emergency brake. His argument: treat every capable model, open or closed, like an insider with access to your systems. Do not trust it to police itself. Put the controls outside the model, log what it does, and keep a person able to stop it mid-task. When the head of a company that sells frontier models through its cloud says "don't trust the model," your agent setup should take note.

What actually happened

TechCrunch reports that Nadella's post calls for a new look at the "trust architecture" of AI. The essay is titled Models as Insider Risks in the Super Intelligence Era. The main points:

• Separate the model from the harness. The model supplies intelligence. The orchestration layer around it decides what the model can touch. Those are two different things, and the permissions belong in the second one.
• Externalize controls and safeguards. Nobody can trace a model's behavior back to specific training data or weights, so the rules that limit its access cannot live inside it.
• Keep tamper-proof, human-readable evidence of every meaningful action the model takes.
• An emergency brake. An authorized person must be able to pause or shut down a model in the middle of a task.
• Assume compromise. Nadella's line, per TechCrunch: "We must assume a model is compromised and contain it from the start."

He also rejects stacking black boxes, where one opaque model watches another. The post is a proposal, not a product launch. TechCrunch notes no Microsoft product was named. It lands one day after Anthropic cut its internal evals off from the live internet because its agents took actions nobody asked for.

Why it matters for your business: the brake is your job

None of this needs a frontier lab budget. It needs you to own the layer between the model and your systems. Here is how we read it for a small team running agents on Shopify, a CRM or a help desk:

• Permissions live in code, not in the prompt. "Do not issue refunds over $200" in a system prompt is a suggestion. A refund tool that rejects anything over $200 is a control.
• Every write goes to an append-only log. Tool name, arguments, result, timestamp. Keep it somewhere the agent cannot edit. That is your "tamper-proof evidence" at a small-business scale.
• Build a kill switch before launch. One flag in your database or env config that makes every tool call fail closed. Test that it works.
• Do not let a second LLM be the only guard. A model reviewer can help with triage, but it is not your safety boundary. Hard rules are.

If your vendor's agent platform does not let you see or change the harness, you cannot do any of this. That is the real portability question.

Sources: TechCrunch, TechCrunch.