Google Research introduces ToolGrad for answer-first tool-use data generation

Google Research has introduced ToolGrad, a framework that generates successful tool-use workflows before writing the user prompts and model responses paired with them for training. The answer-first method reverses a common synthetic-data process: generating a hypothetical query first, then using a search agent to find a working sequence of tools.

In experiments reported by the researchers, ToolGrad completed 99.8% of generation runs, compared with 63.8% for a depth-first-search method based on ToolBench. The ToolGrad paper also reports 63.9 LLM calls and 20.0 tool-use steps per generated sample, versus 64.5 and 34.3 for the DFS baseline. These are author-reported results, not independently reproduced measurements or currency-denominated costs.

Under the query-first baseline, a system samples APIs and generates a plausible instruction. A depth-first-search agent then explores possible tool calls in search of an answer, consuming resources without necessarily producing a valid training example. ToolGrad constructs its chain from successful executions, then asks a model to write a coherent query and response around the resulting workflow.

The Google Research announcement describes a propose-execute-select-update loop. First, an API proposer narrows a sampled mini-batch to a few candidates that could extend the current workflow. Executors test those candidates in parallel and report the calls and whether they succeeded. A selector reviews successful reports, chooses the most valuable API and identifies where to add it. An updater revises the synthetic user query and AI response to match the expanded workflow. Repeating the loop produces a training triple: a user query, a verified API workflow and a final response.

ToolGrad calls the selector’s discrete API choice a textual “gradient.” This is not a numerical gradient used to update model weights. Instead, it is the directional signal that determines how the workflow grows in the next iteration. The paper says ToolGrad-500 came from 500 seeded generation sessions, each with 10 iterations, mini-batches of 50 APIs and up to three proposals per iteration. Gemini 2.5 Flash-Lite generated the data.

ToolGrad’s successful samples averaged 3.4 ground-truth tool uses, according to the researchers, compared with 2.1 for the DFS samples. In their ToolBench single-turn evaluation, ToolGrad models based on 1-billion, 4-billion and 12-billion-parameter Gemma 3 variants scored 14.1, 17.6 and 19.6. The paper says these results topped the corresponding base Gemma 3 models by 13.1, 6.4 and 9.8 points.

On the researchers’ BFCL v1/v2 single-turn evaluation, fine-tuning with ToolGrad-500 raised scores for the three model sizes by 8.1, 8.0 and 6.3 points. Google reported 83.1 for ToolGrad-12B, 0.1 point behind Gemini 2.5 Pro at 83.2 and ahead of the 82.8 and 74.4 figures it listed for Claude 4.5 Opus and GPT-5. The comparison covers single-turn tool use with largely unseen tools; multi-turn tool use and agent-use tracks were outside the study’s evaluated scope.

The authors identify several limitations: the dataset lacks reasoning examples, its use in reinforcement learning has not been tested, synthetic queries may not fully reflect real human requests, and gains plateaued in a small-data scaling study. The paper appears in Findings of ACL 2026, and the researchers have released the ToolGrad code repository. The work joins Google Research’s MAPL-EMIT release, another research package built around specialized training data, a trained model and code.