Google AI Previews Tool Retrieval in the Gemini API for Large Tool Libraries

Google has launched Tool Retrieval in the Gemini API, now in preview in the Interactions API. Gemini API Tool Retrieval lets an agent keep a large tool library available. It does not send every parameter schema in every prompt. The model loads a schema only when a task needs that tool. For agents wired to dozens of functions or MCP servers, it targets a real cost: tool definitions. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.

TL;DR

  • Tool: An API feature with no stated tool cap. Google advises static loading under 20 stable tools.
  • Runs on: Google’s servers (server-side search) or your own registry (client-side). Interactions API, v1beta only.
  • Performance: Search, schema load and function call resolve in 1 API request in server mode.
  • Best: 0 extra round trips and no index to host for server-side retrieval.

Tool Retrieval is a Gemini API feature that defers tool parameter schemas until the model asks for them. Google’s documentation states the problem plainly. A few dozen function declarations can consume tens of thousands of tokens on every request.

With deferral, the model sees only each tool’s name and a short description. It does not see the parameter schema until retrieval.

How does defer_loading work in the Gemini Interactions API?

You add {"type": "tool_search"} to the tools array. Then you set defer_loading: true on each function to hold back. The @googledevs launch post shows a deferred issue_refund function on gemini-4-hydrogen. Its code comment mentions 50+ other deferred functions.

When the model needs a deferred tool, the response contains 3 steps:

  • tool_search_call: the model requests a tool by name.
  • tool_search_result: the API returns the full schema.
  • function_call: the model emits the call your code executes.

The API resolves the first 2 steps itself. Your app only runs the function_call and returns a function_result.

Deferred functions can carry an optional short_description. It is shown during retrieval and is valid only when defer_loading is true.

Server-side vs client-side retrieval

Server-side is the default. Gemini runs the search, so there is no index to build or host.

Client-side mode uses "execution": "client". Your app runs its own lookup, such as an embedding index, registry or permission filter. It returns matching declarations in function_result.result. Those tools can join mid-conversation, even if absent from the initial tools array. Each client-side search costs 1 extra round trip.

The 2 modes cannot be combined in 1 request. Only 1 tool_search entry is allowed per request.

Does it work with MCP servers and prefix caching?

Yes on both, with caveats. Setting defer_loading: true on an mcp_server entry defers every tool that server exposes. Deferred MCP tools still execute remotely through the Gemini API MCP support.

On caching, deferred schemas stay out of the prompt prefix. Loaded schemas append inline, so implicit caching still applies. The docs add one catch: deferring tools can shrink a prompt below the minimum cacheable size.

There is no separate fee. Tokens bill at the model’s standard input, cached and output rates. Each server-side search triggers an extra model call over the full context. Those tokens appear in total_tool_use_tokens, billed at the input rate.

Each loaded schema then stays billed as input for the rest of the conversation. The docs say static loading can cost less for small libraries or per-turn searches.

Anthropic and OpenAI already ship similar tool search features, with published savings numbers.

FeatureGemini API Tool RetrievalAnthropic Tool Search ToolOpenAI tool search
StatusPreview, v1betaNot disclosed on docs pageNot disclosed on docs page
API surfaceInteractions API onlyMessages API; Bedrock InvokeModelResponses API
ModelsGemini 4 Carbon, Neon, Hydrogen, Argon; 3.8 Flash; 3.6 FlashClaude 5 series, Opus 4.5+, Sonnet 4.5+, Haiku 4.5gpt-5.4 and later
Search modesServer or clientRegex, BM25 or customHosted or client-executed
Max deferred toolsNot disclosed10,000Not disclosed
Results per searchNot disclosed5 by defaultNot disclosed
MCP deferralPer mcp_server entryPer mcp_toolsetOn MCP server tool
Separate feeNoNoNot disclosed
Published token savingsNone85% (77K to 8.7K tokens)47% on 250 MCP Atlas tasks
Published accuracyNoneOpus 4.5: 79.5% to 88.1%Not disclosed

Anthropic and OpenAI figures are self-reported in their own tests. They are not comparable to any Gemini result.

‘;d+=got[t]?’

‘+t+’ {schema loaded}

‘:’

‘+t+’: short description

‘});R.querySelector(‘#gstatic’).innerHTML=s;R.querySelector(‘#gdef’).innerHTML=d;var c=””;tools.forEach(function(t){c+=’‘});R.querySelector(‘#gchips’).innerHTML=c;R.querySelectorAll(‘.chip’).forEach(function(b){b.onclick=function(){got[b.dataset.tool]=true;draw()}});post()}
draw();
// modes
var M={s:[[‘Who runs the search’,’Gemini, on Google servers’],[‘Retrieval code you write’,’None’],[‘Extra round trips’,’None’],[‘Remote MCP servers’,’Yes, defer a whole mcp_server’],[‘Best for’,’Large known libraries and MCP servers’]],c:[[‘Who runs the search’,’Your app (registry, embeddings, permission filter)’],[‘Retrieval code you write’,’Yes’],[‘Extra round trips’,’1 per search’],[‘Remote MCP servers’,’Not applicable’],[‘Best for’,’Dynamic registries, namespaces, per-user tools’]]};
function mode(m){var h=””;M[m].forEach(function(r){h+=’

‘+r[0]+’‘+r[1]+’

‘});R.querySelector(‘#gmode’).innerHTML=h;R.querySelectorAll(‘.seg button’).forEach(function(b){b.classList.toggle(‘on’,b.dataset.m===m)});post()}
R.querySelectorAll(‘.seg button’).forEach(function(b){b.onclick=function(){mode(b.dataset.m)}});mode(‘s’);
function bars(){var f=R.querySelectorAll(‘.fl’);f.forEach(function(x){x.style.width=”0″});setTimeout(function(){f.forEach(function(x){x.style.width=x.dataset.w+’%’})},80)}
R.querySelector(‘#greplay’).onclick=bars;
R.querySelectorAll(‘.fc’).forEach(function(c){c.onclick=function(){c.classList.toggle(‘flip’)}});
window.addEventListener(‘load’,post);window.addEventListener(‘resize’,post);setTimeout(post,300);
})();

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *