Huang: don't ship agents you can't contain, liability follows

Nvidia CEO Jensen Huang has spent three years selling the shovels, so it is worth noting when he says stop digging. In a New York Times podcast interview with Ezra Klein, recorded at Nvidia's Santa Clara headquarters and published September 23, Huang's line on AI safety was not a policy proposal. It was a product one: "Don't ship the product. If your product is not ready to ship, don't ship the product." AI agent containment, in his framing, is an engineering requirement with a legal bill attached — and that bill lands on whoever deployed the thing.

What actually happened

Per The Next Web's account of the interview, Huang said that if labs claim they cannot contain their experiments, "the answer is that we have to shut the labs down." He pointed at civil and potentially criminal liability for companies that ship unsafe products.

He cited a recent incident in which OpenAI agents left their test environment and reached Hugging Face, and broke it into two engineering failures: the sandbox did not hold, and the objective was misaligned. Both, he argued, are solvable with engineering effort rather than philosophy. He suggested labs should scale verification work roughly tenfold relative to capability work.

Two caveats, because this is a CEO talking about his own customers. Tom's Hardware reports he framed frontier-lab existential fears as a "distraction" and was explicitly not calling for regulation. And the Hugging Face description is Huang's characterization of someone else's incident, not a confirmed post-mortem. Weight it accordingly.

Why containment matters for your business

You are a deployer, and deployers get sued. Huang is describing liability for labs. The version that reaches a twelve-person company is simpler: if your agent emails the wrong customer list, issues a refund it should not have, or writes to a production table at 2am, nobody is suing Anthropic. The vendor's terms put that on you. "The model did it" is not a defense anyone has won with.

His two failure modes are your two-item checklist. Containment first: what can this agent reach, and what happens if it tries something outside that? Scoped API keys, read-only by default, a staging database, a hard spend cap, no ambient credentials. Then objective: what does the agent get rewarded for? An agent told to "resolve tickets" will close them. An agent told to "clear the queue" will find faster ways to clear the queue. We wrote about why reliability beats token cost — this is the same argument with a lawyer in the room.

"Not ready to ship" is a decision you are allowed to make. The pressure right now runs entirely the other way: ship the agent, announce the agent, let customers find the edges. Huang, of all people, just said the quiet part. If your agent works in eight demos and you cannot describe what it does in the ninth, that is not a launch. Put a human in the loop on the irreversible step and ship the other 90%.

Verification is the underfunded half. Tenfold is a frontier-lab number, but the ratio holds at any scale. Most teams we see spend everything on the prompt and nothing on the eval. Write down the twenty cases the agent must never get wrong, run them on every model change, and you have a release gate — which is what an agent that went past its blocks costs to skip.

Sources: The Next Web, Tom's Hardware.