How often do clinicians catch a wrong AI recommendation?
A review of 46 reports found that correct automated advice was far more likely than wrong advice to produce a correct clinical decision. In four real-world ECG studies, 26.6% of erroneous outputs were not corrected or recognised—but the evidence was highly heterogeneous and often predates modern AI.
The short version
- What happened
- The systematic review included 46 reports mapped to 41 independent cohort clusters; technologies ranged from computerized ECG interpretation to current deep-learning and language-model systems.
- Who it affects
- A clinician may be formally responsible for checking an automated result while working under time pressure and with incomplete information. A safer workflow must help them see why the recommendation could be wrong, not merely place a human name at the end of the process.
- What remains unproven
- The primary meta-analysis contained three adjusted cohorts. One older cohort did not clearly document adjustment for repeated observations; the authors therefore interpret its precision cautiously.
Keep in mind
The primary meta-analysis contained three adjusted cohorts. One older cohort did not clearly document adjustment for repeated observations; the authors therefore interpret its precision cautiously.
The real-world non-correction estimate pooled four computerized-ECG studies with extreme heterogeneity, different definitions and technologies that are not equivalent to current generative AI.
Contemporary AI and language-model studies were too heterogeneous for quantitative pooling and were often controlled or simulated rather than deployed in clinical care.
Sixteen full texts could not be retrieved. Original verbatim search exports and retrieval histories were not retained, and the authors reconstructed the search syntax for the report.
The authors report no funding and no competing interests.
Source analysis · 1 linked source. Primary analysed source date: 23 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Human review did not cancel out a wrong machine answer
A peer-reviewed review found a consistent asymmetry in clinical decision support: clinicians were much more likely to reach the right decision when the automated recommendation was right than when it was wrong. The main meta-analysis covered only three adjusted cohorts, but its pooled odds ratio was 10.19 (95% CI 7.43–13.98). Restricting the analysis to two studies published from 2020 produced a similar point estimate with much wider uncertainty.
Four retrospective studies of computerized ECG interpretation supplied a second, real-world estimate. Across them, 26.6% of erroneous outputs were not corrected or recognised. That average is not a universal failure rate: its 95% confidence interval ran from 6.4% to 65.6%, heterogeneity was extreme, definitions differed and much of this evidence concerns older ECG software rather than today's generative AI.[1]
The review spans 40 core cohorts—but the strongest synthesis is small
The authors searched eleven bibliographic and preprint sources through July 2026, with targeted updating through 17 August. They included 46 reports representing 41 independent cohort clusters, including 40 core cohorts. Thirty-five core cohorts were controlled or experimental and five were real-world studies. Technologies ranged from conventional decision support and ECG interpretation to image models, draft reports and language models.
For patients and clinicians, the practical warning is specific: a human in the loop is not proof that an unsafe recommendation will be intercepted. Systems need visible uncertainty, easy comparison with primary evidence, audit trails and workflows that make disagreement possible. Patient-level adverse events appeared in only one retrospective cohort, with few events and no causal attribution, so this review cannot estimate how many people were harmed by automation errors.[1]
The distinction that matters
Three evidence layers answer different safety questions
The review separates controlled decision effects from deployed error handling and patient outcomes.
Decision accuracy
Pool three adjusted cohorts
Correct advice strongly associated with a correct decision
Error interception
Combine four real-world ECG studies
26.6% uncorrected or unrecognised, with very high uncertainty
Patient harm
Look for downstream adverse events
Only one cohort reported events; causation was not established
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring clinical ai: compare the finding and its evidence limits.
Did clinicians prefer AI discharge summaries after long hospital stays?
In a retrospective 60-case comparison, 12 physicians usually preferred GPT-5.2 summaries and annotated fewer omissions. Reviewers knew which summary was AI-written, one hospital supplied the records, and no patient outcome or time saving was tested.
Read the next reportExplore the full source and evidence record
What this means for people
- A clinician may be formally responsible for checking an automated result while working under time pressure and with incomplete information. A safer workflow must help them see why the recommendation could be wrong, not merely place a human name at the end of the process.
Global context
The review combines evidence across countries, specialties and several generations of decision support. Its strongest general lesson is about conditional safety: the presence of human oversight is not itself an outcome measure. Local deployment evidence must still test error recognition, overrides, workload and patient consequences.
What the evidence does not yet show
- The primary meta-analysis contained three adjusted cohorts. One older cohort did not clearly document adjustment for repeated observations; the authors therefore interpret its precision cautiously.
- The real-world non-correction estimate pooled four computerized-ECG studies with extreme heterogeneity, different definitions and technologies that are not equivalent to current generative AI.
- Contemporary AI and language-model studies were too heterogeneous for quantitative pooling and were often controlled or simulated rather than deployed in clinical care.
- Sixteen full texts could not be retrieved. Original verbatim search exports and retrieval histories were not retained, and the authors reconstructed the search syntax for the report.
- The authors report no funding and no competing interests.
What to watch next
- Prospective studies that deliberately include wrong recommendations and measure whether clinicians detect them
- Real-world monitoring of overrides, delayed corrections and downstream patient outcomes
- Whether uncertainty displays and access to underlying evidence reduce harmful reliance
- Larger syntheses separating current deep-learning and language-model systems from legacy decision support
Living evidence record
Impact record IAI-1RDPD4F
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
24 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what npj Digital Medicine published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 24 October 2026
Research question, topics and regions
Whether clinicians reliably intercept incorrect recommendations from AI and other automated decision-support systems
Topics: Clinical AI · Human oversight · Automation bias · Patient safety · Decision support
Regions: International · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.