AMD and Cerebras split inference: prefill vs decode
AMD Helios handles prompt processing, Cerebras wafer-scale handles token generation. Up to 5x tokens per watt. Why disaggregated inference changes your AI bill.
Every AI request you make is actually two different computing problems wearing one trench coat, and the industry just admitted it. AMD and Cerebras announced a partnership that runs the two halves on different silicon: AMD chews through your prompt, Cerebras spits out the tokens. Disaggregated inference is a plumbing story, but it's the plumbing that sets what you pay per request and how long your users stare at a spinner.
What actually happened
At Advancing AI 2026 on July 23, AMD and Cerebras announced a technical partnership pairing AMD Helios rackscale systems with the Cerebras Wafer-Scale Engine into a single inference workflow. The split follows the actual shape of the work: Helios handles prefill — processing your prompt and large context windows, which is throughput-bound and loves lots of GPUs. Cerebras handles decode — generating tokens one at a time, which is memory-bandwidth-bound and is exactly what a wafer-sized chip is built for.
The claim is up to 5x higher tokens per second per watt compared to a Cerebras WSE-only configuration. Cerebras will deploy AMD Helios in its own data centers, with the combined offering expected in the second half of 2026 through Cerebras Cloud. Axios framed it as two challengers pooling their strengths against Nvidia rather than each fighting it alone.
AMD's Lisa Su called inference "one of the largest infrastructure opportunities." Cerebras CEO Andrew Feldman pointed at demand for ultra-fast inference growing "at an unprecedented pace." Translation for everyone else: the people selling tokens have figured out that one chip architecture serving both phases of a request is leaving margin on the table.
Why this matters for your business
You will never buy a Helios rack. You will absolutely feel this, because it lands in the two numbers that decide whether an AI feature is worth shipping: cost per request and time to first token.
Prefill and decode having different economics is why prompt caching exists, why long-context calls price differently, and why your "fast" model sometimes isn't. As providers get better at running each phase on hardware suited to it, the pricing you see gets more granular — more separate line items for input tokens, cached input, and output, priced further apart. Budget for that instead of assuming a single blended per-token rate.
The practical move is the same one we push on every model story: don't hard-code a provider. Route your AI calls through one internal abstraction so that when a cheaper or faster endpoint shows up in the second half of the year, switching is a config change and an eval run, not a refactor. Measure cost per completed task and latency at the p95, not benchmark scores. The chip vendors are competing on your behalf right now — you only capture the savings if your code can move.
Key takeaways
- AMD and Cerebras announced a disaggregated inference partnership July 23: Helios does prefill and long-context prompt processing, Cerebras WSE does token generation
- Claimed up to 5x higher tokens per second per watt vs Cerebras WSE alone; available 2H 2026 via Cerebras Cloud
- Prefill and decode have different hardware economics — expect provider pricing to keep splitting input, cached input, and output further apart
- Route AI calls through your own abstraction layer so a cheaper endpoint is a config change, not a rewrite
- Track cost per completed task and p95 latency; those move with infrastructure shifts like this one, benchmark scores don't
Hard-coded to one AI provider? We build the routing layer that lets you swap models and endpoints as the price war plays out — and the evals that tell you whether the swap actually held up. See how we build portable AI systems or run the numbers on what your AI features cost.
Sources: AMD Newsroom, Axios.
- #amd
- #cerebras
- #inference
- #ai-costs
- #infrastructure
Tommy Rush — Founder, Rush Commerce
Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More
Get The Rush Report weekly — one email, zero fluff.
Keep reading
Pilot Protocol's $4.5M: a directory isn't a standard
A seed-stage AI agent network wants to be where agents discover each other. Useful, but know the difference between an open spec and someone else's front door.
Read itNvidia backs Safe Superintelligence, discloses no terms
Nvidia's Safe Superintelligence partnership names no dollar figure and no term length. Reported at $5B. Here's how to read AI deals that omit the numbers.
Read it