A local model simplified radiology reports—but 45 contained clinically relevant hallucinations
Gemma-3 made Dutch reports easier to read without sending text to a cloud service. Expert reviewers still found potentially consequential invented content in one in eight reports.
The short version
- What happened
- Researchers randomly sampled 360 Dutch primary-care reports, 60 from each of six X-ray and ultrasound examination types, from a database of 5,534.
- Who it affects
- Plain language could help patients understand a radiology conclusion, but an invented or altered detail may mislead them. A clinician must remain responsible for checking every patient-facing rewrite.
- What remains unproven
- Retrospective proof of concept
Keep in mind
Retrospective proof of concept
One institution
One model and prompt
Dutch reports only
Mostly one expert reviewer
No patient comprehension test
No prospective workflow
Reproducibility not tested
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Readable conclusions performed better than full reports
A locally hosted language model produced simpler Dutch radiology reports without transferring report text to an external cloud, but expert reviewers found potentially clinically relevant hallucinations in 45 of 360 reports. The retrospective proof-of-concept study used Gemma-3 12B on a random sample drawn from 5,534 primary-care reports at one Dutch institution. The sample contained 60 reports for each of six common examinations: chest, knee, lumbar spine and shoulder X-rays, plus shoulder and abdominal ultrasound.
Researchers evaluated 359 rewritten main sections and 350 rewritten conclusions. Seventy-one per cent of main sections and 96% of conclusions met the prespecified expert threshold for accuracy and completeness, both scored at least four on five-point scales. Yet only 48% of combined main-and-conclusion rewrites also passed the readability threshold. Conclusions were the stronger candidate: their median accuracy score was five, compared with four for the main report.[1]
Easier words came with invented details
Readability improved sharply on the Dutch Flesch–Douma scale, with an average increase of 54.2 points, although rewritten reports also became a median 65 words longer. A readability formula counts features such as word and sentence difficulty; it does not show that a patient understood the result, felt less anxious or made a better decision. No patients evaluated the output.
Reviewers documented 69 hallucinations across 58 of 359 main-section rewrites. They judged 54 events in 45 reports—12.5% of the sample—as potentially clinically relevant. The paper's examples include adding an incorrect explanation or changing the meaning of a finding. A patient-friendly version can therefore sound clearer while becoming less faithful to the clinical report. Local hosting limits one privacy risk, but does not solve the accuracy problem.[1]
The distinction that matters
What the local model achieved—and risked
Quality, readability and fidelity were separate tests.
Quality
71% of main sections; 96% of conclusions passed
Conclusions performed better
Readability
48% passed quality plus readability together
Easier language was not universal
Safety
45 reports had clinically relevant hallucinations
Clinician review remains necessary
The evidence is promising, not deployment-ready
Most reports were rated by one expert. Four reviewers independently scored a 36-report subset, and agreement was mixed: moderate for conclusion accuracy, lower for main-report accuracy and poor for completeness. The experiment used one model, one prompt, one institution and one language. It did not formally test whether repeated runs would reproduce the same rewrite, and it did not place the system in clinical care.
For patients, a reliable plain-language conclusion could make a portal result less opaque and give a better starting point for questions. For radiologists and GPs, unchecked invented details could create confusion, extra calls or unsafe reassurance. The study did not measure either effect, so reduced workload and better understanding remain hypotheses rather than outcomes. Confidence would change with patient comprehension testing, independent multi-reader evaluation, repeated-output testing and a prospective workflow that requires clinician approval and records corrections. External studies should include abnormal and high-stakes findings, other languages and comparisons with human-written summaries. They should also report how often clinicians reject or substantially rewrite the model's draft. The present result supports cautious development—especially for conclusion sections—not automatic publication to patients.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring clinical ai: compare the finding and its evidence limits.
Qwen3 caught 320 of 335 radiology-report errors—what did it never see?
The model performed strongly on report text from two hospitals and an error-enriched comparison with three radiologists. It never saw the scans, was not used prospectively and did not measure patient outcomes.
Read the next reportExplore the full source and evidence record
What this means for people
- Plain language could help patients understand a radiology conclusion, but an invented or altered detail may mislead them. A clinician must remain responsible for checking every patient-facing rewrite.
Global context
Patient access to clinical reports is expanding internationally, but language, report style, privacy rules and clinician workflows vary. Local hosting may be attractive where cloud transfer is unacceptable, while every setting still needs local fidelity and safety testing.
What the evidence does not yet show
- Retrospective proof of concept
- One institution
- One model and prompt
- Dutch reports only
- Mostly one expert reviewer
- No patient comprehension test
- No prospective workflow
- Reproducibility not tested
What to watch next
- Patient comprehension
- Multi-reader validation
- Repeated-output stability
- Prospective clinician review
- Correction rates
- High-stakes abnormal findings
- Other languages and institutions
Living evidence record
Impact record IAI-0YEZCZU
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
30 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what European Radiology published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 30 October 2026
Research question, topics and regions
Locally hosted language-model simplification of Dutch primary-care radiology reports
Topics: Clinical AI · Radiology · Health communication · Local models · Hallucinations
Regions: Netherlands · Europe · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.