Can you trust citations in Google’s AI health answers? A 3,611-question preprint found weak grounding
Google showed an AI Overview for 96.4% of the benchmark questions, yet many factual statements lacked citations and only about half of scorable cited claims were supported. This was one US-English snapshot, not a test of patient outcomes or today’s product.
The short version
- What happened
- Researchers submitted 3,611 English HealthBench questions to Google Search in a US locale on 16–18 September 2026; 3,481, or 96.4%, returned an AI Overview.
- Who it affects
- People may read a cited AI health answer as more authoritative than an uncited one. This study says the presence of citations alone is not enough to establish support, accuracy or completeness; urgent and personal decisions still need qualified clinical advice.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
One Google product snapshot
US-English locale
September 2026 collection
Automated source and answer grading
Limited human validation samples
Unreadable cited pages excluded
Complex benchmark questions
No patient behaviour or clinical outcomes
No causal test of how citations were generated
No separate funding or competing-interests statement
Source analysis · 2 linked sources. Primary analysed source date: 29 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Citations were common, but support was inconsistent
A new preprint found that Google frequently produced AI-generated health summaries without consistently tying their factual claims to supporting pages in this snapshot. Researchers sent 3,611 realistic, problem-focused HealthBench questions to Google Search from 16 to 18 September 2026, fixing the locale to US English. One retrieval failed; 3,481 of the remaining 3,610 questions returned an AI Overview. Of 3,474 usable overviews, 327 provided no source citation at all.
The team split the answers into 24,060 statements. One in five was classified as factual but uncited. Of 12,175 cited statements that the automated system could score against a linked page, 47.1% were supported. After adjusting for the classifier’s measured sensitivity and specificity, the estimate rose to 56.7%, but its 95% interval stretched from 45.3% to 68.6%. That uncertainty matters: neither number is a direct human audit of every claim.[1][2]
A large benchmark was paired with automated checks and physician rubrics
The questions came from 5,000 HealthBench conversations, filtered to 3,611 English health queries. The researchers captured each overview, all its citations and Google’s five highest traditional results. Their source-quality script applied four JAMA disclosure criteria and two HONcode-derived checks; on 174 human judgments it agreed 84% of the time, with Cohen’s kappa of 0.68. The grounding classifier was validated on 100 statements, reaching 81% agreement and kappa 0.61.
For answer quality, 41,180 physician-written rubric criteria were graded by GPT-4.1. On 6,200 physician grades from a random 3,000-example validation set, its macro-F1 was 0.707, compared with inter-physician agreement of 0.681. The overviews earned 38.2% of all available rubric points, including 54.2% for accuracy and 21.6% for completeness. These are fractions of detailed rubric points—not a claim that every answer was simply 38.2% correct. The cited pages also scored slightly lower than traditional results on four of six disclosure and authority checks, although every standardised difference was small and the largest absolute effect was 0.19.[1][2]
The distinction that matters
What the study measured—and what it did not
Three linked tests examine different parts of an AI health answer; none measures patient benefit or harm.
Coverage
3,611 health questions
3,481 returned an AI Overview
Grounding
24,060 extracted statements
Citations checked against linked pages
Answer quality
41,180 physician rubric criteria
Automated grading with physician validation
Not measured
Patients and clinicians
No behaviour, decisions or health outcomes
A citation icon should not be treated as a clinical guarantee
The study also found little relationship between citation grounding and rubric accuracy or completeness: Spearman’s rho was 0.01. Among 2,065 questions whose physician rubric anticipated a particular inaccurate or potentially harmful statement, 22.2% of overviews made at least one such statement. Yet the system referred 94.9% of 413 emergency-referral questions with the expected urgency. The evidence therefore describes uneven performance, not uniform failure.
This is an unreviewed snapshot of one changing product in one language and locale. HealthBench questions can be long and complex, although a shorter single-turn subset produced similar patterns. Automated graders, inaccessible pages and text-only extraction add error; 11.7% of cited pages and 14.0% of traditional results were excluded as unreadable. No patients, clicks, treatment decisions or health outcomes were studied. Independent replication on current versions, clinician-led claim review and user-outcome research would change the assessment. The manuscript gives no separate funding or competing-interests statement.[1][2]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring health information: compare the finding and its evidence limits.
Could hidden AI steering quietly change depression-treatment advice?
A peer-reviewed proof-of-concept shifted one open model's emphasis between medication and self-care without changing its weights or visible prompt. The test used 12 synthetic scenarios—not patients, clinicians or commercial chatbots.
Read the next reportExplore the full source and evidence record
What this means for people
- People may read a cited AI health answer as more authoritative than an uncited one. This study says the presence of citations alone is not enough to establish support, accuracy or completeness; urgent and personal decisions still need qualified clinical advice.
Global context
The questions were submitted in US English and the results cannot be assumed to hold for other countries, languages, search interfaces or later product versions. Health-information regulation and access to clinicians also vary widely.
What the evidence does not yet show
- Unreviewed preprint
- One Google product snapshot
- US-English locale
- September 2026 collection
- Automated source and answer grading
- Limited human validation samples
- Unreadable cited pages excluded
- Complex benchmark questions
- No patient behaviour or clinical outcomes
- No causal test of how citations were generated
- No separate funding or competing-interests statement
What to watch next
- Peer review
- Current-version replication
- Other languages and countries
- Larger clinician-led claim audits
- Patient and clinician behaviour
- Emergency-query performance
- Product citation changes
- Provider transparency on source selection
Living evidence record
Impact record IAI-19TZJ5O
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
31 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 31 October 2026
Research question, topics and regions
Source quality, citation grounding and physician-rubric performance of Google AI Overviews on health questions
Topics: Google Search · AI Overviews · Health information · Citations · Source quality · Medical accuracy
Regions: United States · North America
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.