Google TPU 8tx and 8i Max: Confidential AI Inference Goes Hardware

Google just added two more chips to its eighth-generation lineup. On November 9, Google SVP Amin Vahdat introduced the TPU 8tx and TPU 8i Max. The 8tx is the interesting one for small businesses: it moves confidential AI inference into the hardware, so data and model weights stay protected even while the model is working on them. Google says this is the first time it has shipped four TPU systems in one year.

What actually happened

Per Google's announcement:

• TPU 8tx uses the same silicon as the TPU 8t training chip, paired with AMD CPU hosts (5th Generation EPYC, per Google's main blog). It targets regulated enterprises and governments, and Google says it keeps sensitive data and model weights isolated during active use.
• Google pitches the 8tx for confidential inference plus small-scale work on private data: fine-tuning, distilling big models into small ones, and reinforcement-learning training for agents, without copying sensitive data between clusters.
• TPU 8i Max puts two TPU 8i inference chips on one package. Google says that doubles compute, SRAM and HBM bandwidth compared with the 8i, in the same rack space. The target is long-context work, like many agents working across a whole codebase or hours of video.
• Google also released Agent Compute Engine, an open-source layer that automatically suspends idle agent sessions, and made Cloud Pathways generally available on GKE.
• Google gave no pricing, regions or absolute memory figures. Bloomberg reports both systems launch in 2027.

Why confidential AI inference matters for your business

You will not rent a TPU 8tx. Your AI vendors might. And once one big cloud sells "encrypted while in use" as a product, your customers will start asking whether your AI tools have it.

That matters most if you handle data people care about: patient notes, tax returns, legal files, payment disputes. Today, most AI privacy promises are contract language: we don't train on your data, we delete it after 30 days. Confidential computing is a different claim. It is a hardware guarantee that the cloud operator can't read the data mid-computation. Google's post doesn't mention attestation, the proof step that lets you verify that guarantee. That is the question to ask when this ships.

Here is what we would do now:

Map where sensitive data meets a model. List every AI tool that sees customer records. Note the vendor, the model provider underneath, and what their contract actually promises.

Ask the in-use question. "Is my data encrypted while your model processes it, and can you prove it?" Expect a lot of "not yet." Write down the answer anyway. It becomes a renewal lever in 2027.

Kill idle agent time. Google building a tool that suspends idle agent sessions tells you idle sessions cost real money. Check whether your agents hold compute while they wait on a human or an API.

Sources: Google Cloud Blog: Introducing TPU 8tx and TPU 8i Max, Google: Our newest TPUs, 8tx and 8i Max, Bloomberg.