Gimlet Labs raises $300M for multi-silicon inference

Multi-silicon inference just got a $300 million vote of confidence. Gimlet Labs announced a Series B on September 4 led by Andreessen Horowitz, valuing the company at $3 billion and bringing total funding to $392 million. The pitch is not another GPU cloud. It is software that chops a single model into stages and runs each stage on whatever chip serves it best — and that has direct consequences for what you pay per token.

What actually happened

Per Gimlet's announcement, Arm and Microsoft's M12 fund both joined the round. The company says its platform spans NVIDIA, AMD, Intel, Arm, Cerebras, and d-Matrix silicon in one system, claims up to 10x throughput and interactivity gains, and is scaling to several hundred megawatts of managed heterogeneous infrastructure.

The mechanism is disaggregation. SiliconANGLE reports the most common split is prefill and decode — the phase that reads your prompt and the phase that writes the answer have very different memory and compute profiles, so Gimlet runs them on different hardware. A finer mode carves the decode phase into sub-workflows, or pairs a lightweight drafter model with a frontier model that refines its output. AI agents plus a custom compiler do the porting and validation work per chip.

Gimlet emerged from stealth in October 2025 and raised an $80 million Series A in March, per SiliconANGLE. CEO Zain Asgar and his co-founders previously built Pixie Labs, the Kubernetes observability company New Relic acquired in 2020. Customer claims — a top-three frontier lab, a top-three hyperscaler, "billions of dollars" in contracted revenue — are the company's own and name no one.

Why multi-silicon inference matters for your business

We covered the AMD–Cerebras prefill/decode split in July. This is the same physics with a funding round attached, and the pattern is now clear enough to plan around: the unit of inference is stopping being "a GPU hour" and starting to be "a stage on the right chip."

For you, that cuts one way. Inference pricing gets less correlated with one vendor's supply, which is good, and more opaque, which is not. A provider that quietly routes your decode phase to a cheaper accelerator can pass savings along or keep them. You will not see the difference in an API response.

So instrument the two things a router cannot hide. Log time to first token and output tokens per second separately for every production call, tagged by provider and model. Those two numbers are the prefill and decode phases respectively, and when a vendor changes the silicon underneath you, they move independently — that is your early warning that something changed. Then keep your prompts and tool definitions provider-neutral so switching is a config change, not a rewrite. Disaggregation is an argument for portability, not against it: the more places a model can run, the less any one of them can charge you.

Sources: Gimlet Labs press release, SiliconANGLE.