Skip to content
Rush Commerce
Software & Dev3 min read

Gemini API Tool Retrieval: stop paying for tool schemas

Gemini API Tool Retrieval is in preview: agents load tool schemas on demand with defer_loading, at no extra charge. Here's when it saves tokens and when it won't.

Every tool you give an agent costs you tokens on every single call, whether the agent uses it or not. On November 19, Google shipped Tool Retrieval in the Gemini API as a preview in the Interactions API. Tool Retrieval lets an agent keep a big tool library on standby and pull in a tool's full parameter schema only when the task needs it.

What actually happened

From Google's announcement:

  • Two changes to your code. Add a tool_search tool to the request. Set defer_loading: true on the tools you want to hold back.
  • What the model still sees. The name and description of every deferred tool. The parameter schemas stay out of context until the model asks for one by name.
  • One request, three steps. Search, schema load and the function call happen inside the same API call. All three show up in interaction.steps.
  • MCP servers too. Set defer_loading: true on an mcp_server tool and every tool on that server loads on demand.
  • Bring your own search. Declare tool_search with "execution": "client" to run lookups against your own registry or per-user permission filter. That costs one extra round trip per search, but you can add new tools mid-conversation.
  • Price. No separate charge. Turning it on adds a small fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation.
  • Models. Gemini 3.6 and later, including all Gemini 4 models, in the Interactions API.

Why it matters for your business

Here's the pattern we see in client builds. The agent starts with six tools. Then someone connects the CRM, the help desk, Shopify and an accounting MCP server, and suddenly every "where's my order?" message ships 80 schemas to the model. You pay for all of them. And the more tools the model has to pick from, the more often it picks the wrong one.

Tool Retrieval fixes the token bill for that long tail. Google is upfront that it's not free money: you save only when the schemas you never load outweigh the ones you do. A five-tool agent gains nothing.

How we'd roll it out:

  1. Keep the hot path loaded. Order lookup, refund and handoff-to-human stay undeferred. Defer the rarely used admin tools.
  2. Rewrite your tool descriptions. With schemas hidden, the description is the only thing the model searches. "Refund an order to the original payment method" beats "refund_v2."
  3. Measure on real traffic. Replay a week of logged conversations with and without deferral. Compare input tokens and wrong-tool calls.

Key takeaways

  • Tool Retrieval is in preview in the Gemini Interactions API as of November 19
  • Add tool_search and set defer_loading: true to load schemas on demand
  • Works on whole MCP servers, with an optional client-side search for your own registry
  • No extra fee, but loaded schemas stay billed for the rest of the conversation
  • Pays off on large tool libraries; measure it on your own traffic before you rely on it

Agent bill growing faster than its job list? We audit tool sets, trim the token waste and build agents with permission filters you control. Run the numbers, then see how we build.

Sources: Google: Introducing Tool Retrieval in the Gemini API.

  • #gemini-api
  • #tool-retrieval
  • #ai-agents
  • #mcp
  • #token-costs
TR

Tommy Rush — Founder, Rush Commerce

Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More

Get The Rush Report weekly — one email, zero fluff.