Gemini API Tool Retrieval: stop paying for tool schemas
Gemini API Tool Retrieval is in preview: agents load tool schemas on demand with defer_loading, at no extra charge. Here's when it saves tokens and when it won't.
Every tool you give an agent costs you tokens on every single call, whether the agent uses it or not. On November 19, Google shipped Tool Retrieval in the Gemini API as a preview in the Interactions API. Tool Retrieval lets an agent keep a big tool library on standby and pull in a tool's full parameter schema only when the task needs it.
What actually happened
From Google's announcement:
- Two changes to your code. Add a
tool_searchtool to the request. Setdefer_loading: trueon the tools you want to hold back. - What the model still sees. The name and description of every deferred tool. The parameter schemas stay out of context until the model asks for one by name.
- One request, three steps. Search, schema load and the function call happen inside the same API call. All three show up in
interaction.steps. - MCP servers too. Set
defer_loading: trueon anmcp_servertool and every tool on that server loads on demand. - Bring your own search. Declare
tool_searchwith"execution": "client"to run lookups against your own registry or per-user permission filter. That costs one extra round trip per search, but you can add new tools mid-conversation. - Price. No separate charge. Turning it on adds a small fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation.
- Models. Gemini 3.6 and later, including all Gemini 4 models, in the Interactions API.
Why it matters for your business
Here's the pattern we see in client builds. The agent starts with six tools. Then someone connects the CRM, the help desk, Shopify and an accounting MCP server, and suddenly every "where's my order?" message ships 80 schemas to the model. You pay for all of them. And the more tools the model has to pick from, the more often it picks the wrong one.
Tool Retrieval fixes the token bill for that long tail. Google is upfront that it's not free money: you save only when the schemas you never load outweigh the ones you do. A five-tool agent gains nothing.
How we'd roll it out:
- Keep the hot path loaded. Order lookup, refund and handoff-to-human stay undeferred. Defer the rarely used admin tools.
- Rewrite your tool descriptions. With schemas hidden, the description is the only thing the model searches. "Refund an order to the original payment method" beats "refund_v2."
- Measure on real traffic. Replay a week of logged conversations with and without deferral. Compare input tokens and wrong-tool calls.
Key takeaways
- Tool Retrieval is in preview in the Gemini Interactions API as of November 19
- Add
tool_searchand setdefer_loading: trueto load schemas on demand - Works on whole MCP servers, with an optional client-side search for your own registry
- No extra fee, but loaded schemas stay billed for the rest of the conversation
- Pays off on large tool libraries; measure it on your own traffic before you rely on it
Agent bill growing faster than its job list? We audit tool sets, trim the token waste and build agents with permission filters you control. Run the numbers, then see how we build.
Sources: Google: Introducing Tool Retrieval in the Gemini API.
- #gemini-api
- #tool-retrieval
- #ai-agents
- #mcp
- #token-costs
Tommy Rush — Founder, Rush Commerce
Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More
Get The Rush Report weekly — one email, zero fluff.
Keep reading
OpenRouter drops 3 providers, pulls ZDR from 125 endpoints
OpenRouter off-boarded three inference providers and removed 125 endpoints from zero data retention after Dragos canary research. What to check in your stack.
Read itOpenAI Performance Factory: Codex agents debug, engineers merge
OpenAI's Performance Factory ran 1,700+ Codex investigations into ChatGPT slowdowns. 70% of audited root causes held up; about 15 agent PRs shipped as written.
Read it