Can a geospatial AI agent reliably answer planet-scale questions?
A Google preprint reports large benchmark gains from specialist geospatial tools and, on open-ended tasks, planning with one Gemini model. The evidence is vendor-produced, unreviewed and does not establish reliable field decisions.
The short version
3 key points
- 1.Across 76 held-out open-ended tasks, the full agent with Gemini 3.8 Flash scored 62.7/100, compared with 52.2 for the same model and geospatial tools without upfront planning, and 3.3 with basic tools.
- 2.Specialist tools delivered the largest gain; upfront planning helped the newer Flash model on open-ended tasks but did not improve the Pro model and added little on deterministic benchmarks.
- 3.The 39-page paper is an unreviewed Google-led evaluation of Google's own system. It excludes infeasible queries, relies partly on a Gemini autorater and does not measure field decisions.
The answer: tools helped on benchmarks; field reliability remains unproven
Google's Geospatial Reasoning Agent performed much better than a standalone Gemini model or the same model with only search, maps and code execution on three of four benchmarks. The biggest gains came from specialist geospatial skills and data tools. Upfront planning delivered a further improvement on open-ended problems with Gemini 3.8 Flash, but not consistently with Gemini 3.1 Pro.
That is meaningful systems evidence because configurations are compared while the language model is held fixed. It is not evidence that the agent reliably answers planet-scale questions in practice. The paper is a Google-led preprint about Google's own system, has not been peer reviewed, and evaluates benchmark outputs rather than downstream health, climate, infrastructure or emergency decisions.
Practitioners should read the result as support for a design principle—specialist tools and carefully matched orchestration can outperform a general chatbot—not as permission to remove geospatial experts. Real decisions still depend on data vintage, boundaries, proxy choices, uncertainty and local knowledge.[1][2]
What the system does
The agent connects Gemini to 23 skills and 62 domain or workspace tools. Its capability set includes Earth Engine, Data Commons, Maps data, administrative boundaries, weather and flood forecasts, satellite and terrain data, remote-sensing foundation models, population and socioeconomic representations, time-series forecasting and spatial computation.
Large datasets remain outside the model's context and pass between tools by reference. Skills load on demand so the model does not see every tool definition at once. Before calling a geospatial tool, the full agent produces a plan and then follows guidance about adapting, retrying or substituting a proxy when an ideal dataset is unavailable.
The architecture addresses a real limitation of chatbots: a plausible sentence about a place is not the same as calculating from current satellite, weather or demographic data. Yet the breadth of connected assets creates more ways for a wrong boundary, stale layer or unjustified proxy to enter an analysis.[1]
Four configurations, two models and four benchmarks
Researchers tested four configurations with Gemini 3.1 Pro and 3.8 Flash: the model alone; the model with Search, Maps and code execution; the model with full geospatial skills and tools but no upfront planning; and the complete agent. Holding the model fixed helps isolate the contribution of tools and orchestration.
GeoExpertQueries contains 101 open-ended practitioner problems, with 25 development and 76 held-out test items. Each has a consensus workflow from three independent experts blind to the agent's tools. GeoKTools contains 305 templated queries—85 development and 220 test—covering 24 tool types and requiring one to four types per task.
Two external benchmarks add 44 usable CBGB questions with deterministic Earth-observation answers and 140 UnivEARTH questions based on NASA material. Each evaluation ran three times, with means and standard deviations reported. Scores have different definitions and cannot be compared directly across benchmarks.[1]
Where the largest gains appeared
On the 76 GeoExpertQueries test items, the full agent with Gemini 3.8 Flash scored 62.7 out of 100, with a standard deviation of 2.5. The same model with geospatial tools but no planning scored 52.2, basic tools scored 3.3 and the standalone model scored zero because it could not retrieve required data. Planning and guidance added 10.5 points beyond specialist tools in this configuration.
The pattern did not generalise to Gemini 3.1 Pro. Geospatial tools without planning scored 50.8 while the full agent scored 47.6. On GeoKTools, tool-rich configurations scored around 82 to 86 against roughly 37 to 41 for simple baselines. On CBGB, tool-rich systems scored about 85 to 88 while neither simpler baseline exceeded 20.
The defensible claim is narrower than 'planning makes agents better'. Specialist geospatial tools drove the large, consistent improvement. Planning's value depended on the model and task: it helped Flash on open-ended work, did not help Pro there, and added little where queries had one well-defined answer.[1]
A saturated benchmark exposed a warning
Every configuration scored between 86.9 and 98.8 on UnivEARTH. A standalone model without Earth Engine access scored 86.9 with Pro and 93.6 with Flash. Trace inspection suggested models often answered from published material without performing the intended analysis.
That is not a demonstration of geospatial skill; it means the benchmark no longer distinguishes systems. It shows why a headline score can mislead when answers are recoverable from model memory or the web.
CBGB had a different decay problem. Four reference values no longer matched current data because satellite imagery was reprocessed and a protected-area boundary changed. Benchmarks about the physical world require ongoing revalidation because ground truth and administrative definitions move.[1]
The autorater and reproducibility limits
GeoExpertQueries outputs were scored by a Gemini 3.8 Flash autorater against expert workflows. The paper reports validation with domain experts, but absolute scores depend on the workflow and judge's strictness. The comparative result is more credible than treating 62.7 as an absolute level of professional competence.
The authors release development splits, prompts and scoring code, including reference-solver specifications for GeoKTools. Most held-out test content stays private, and many tools and datasets are proprietary Google assets. Outside teams therefore cannot fully reproduce the production system or audit every scored trajectory.
The evaluation excludes queries that cannot be completed because required data are proprietary or unavailable. That cleans the comparison but removes an operational failure mode. In real work, recognising missing evidence—and refusing to manufacture a result—is part of reliability.[1]
What this means for climate and public-service work
The promising element is combining natural-language questions with data discovery, spatial computation and provenance. A competent tool could speed first-pass flood screening, site comparison or regional analysis. The study also suggests a smaller model can outperform a larger one when tools and orchestration fit the task.
For communities, speed is not enough. A flood-risk result can change insurance or emergency priorities; a health map can direct scarce services; an energy site screen can alter land use. Each requires uncertainty, current data, transparent boundaries, sensitivity to proxies and review by people with local knowledge.
The paper reports language-model inference cost, not the full cost of external data, Earth Engine computation, expert checking or correction. Flash GRA cost an estimated $0.199 per GeoExpertQueries item versus $0.688 for Pro while scoring higher. That is useful design evidence, not a total-cost or public-value analysis.[1][2]
Commercial interests and what would change the assessment
Most authors are affiliated with Google Research or Google Geo, and the system depends on Google models, products and datasets. This gives the team access for deep evaluation while creating an obvious commercial interest. Independent testing against alternative tools and model families is essential.
The assessment would strengthen with peer review, locked test protocols, external replication and head-to-head comparisons using public data. Field studies should measure factual error, missing-data refusals, sensitivity to boundary changes, expert correction time and real consequences.
For now, the preprint supports a bounded conclusion. Specialist tools can transform what a language model accomplishes on controlled geospatial tasks, and planning can help when matched to the model and problem. It does not establish an autonomous or reliable substitute for geospatial professionals, local evidence or accountable decisions.[1][2]
What this means for people
- Climate, health and emergency teams may gain faster first-pass analysis, but high-consequence decisions still require current data, uncertainty and expert review.
- Communities can be harmed by a wrong boundary, stale dataset or unjustified proxy even when an agent's narrative appears coherent.
- Reported inference prices exclude data services, compute, expert checking and correction, so they are not full deployment costs.
Global context
The benchmarks cover global use cases, but the agent relies heavily on Google-controlled models, data and infrastructure. Performance may differ where data are sparse, boundaries contested, connectivity limited or public agencies cannot use proprietary services.
What the evidence does not yet show
- The paper is an unreviewed Google-led evaluation of Google's own models, tools and datasets.
- Open-ended results rely on an LLM autorater, and absolute scores depend on its strictness and reference workflows.
- Only Gemini models were tested, and most held-out items plus the production tool stack are not independently reproducible.
- Queries requiring unavailable or proprietary data were excluded, understating a central real-world failure mode.
- Benchmarks measure outputs and workflows, not downstream decision quality, safety or outcomes for affected people.
What to watch next
- Peer review and independent replication using public data and non-Google models
- Field evaluations measuring factual errors, expert correction time and downstream outcomes
- Tests of missing-data refusal, proxy selection and sensitivity to changing boundaries or datasets
- Transparent comparisons of total operational cost, including data services, compute and human review
Living evidence record
Impact record IAI-0IGX49N
Evidence stage
Studied
Confidence
Supported
Reporting basis
Multi-source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
14 October 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Evidence trail
Sources used for this report
Links checked 14 October 2026
Research question, topics and regions
What controlled benchmarks show about specialist geospatial tools and planning, and why benchmark gains do not establish safe real-world analysis
Topics: Geospatial AI · Climate · Disaster response · AI agents · Benchmarks · Earth observation
Regions: Global · United States
This report is labelled multi-source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Climate & Energy
Can open radar data train better landslide AI?
A peer-reviewed data descriptor releases 4,910 expert annotations from 92 selected radar interferograms across the Central Alps and Northern Apennines. It expands material for training slow-movement detectors, but it does not benchmark a model across all nine classes or operate as an early-warning system.
7 min · 2 sources
Climate & Energy
Can AI forecasts make a microgrid cheaper and cleaner?
Not on the evidence in this study. A peer-reviewed model found a grid-connected solar-and-battery design cheapest under its assumptions and hybrid neural networks forecast one building's net load, but the forecasts never controlled dispatch and the paper's renewable-fraction figures do not reconcile.
9 min · 1 source
Climate & Energy
Mombak targets $150m for Amazon restoration as Salesforce buys carbon removals
The Brazilian company announced a second fund and a multiyear buyer agreement. Carbon credits can finance restoration, but future verified removals should be kept separate from AI infrastructure emissions.
3 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.