Commercial clot-detection AI missed the cases that were easiest to overlook
Across more than 67,000 CT examinations, two FDA-cleared models were highly specific but less sensitive than their clearance studies suggested—especially for smaller, older and incidental clots.
The short version
- What happened
- The retrospective study evaluated 30,678 dedicated CT pulmonary angiograms and 37,191 routine contrast-enhanced CT examinations across 17 facilities from April 2023 to April 2025.
- Who it affects
- For a patient, the practical risk is false reassurance: the commercial system was least dependable for subtle or older clots. Radiologists still need to inspect the scan and clinical context even when the AI returns no alert.
- What remains unproven
- This was retrospective work in one academic health system, albeit across 17 facilities; community and rural performance may differ.
High specificity hid a clinically important miss rate
Researchers at Emory evaluated two Aidoc algorithms already running in clinical practice. On 30,678 dedicated scans, the pulmonary-embolism model reached 86.8% sensitivity and 99.1% specificity. Its sensitivity was below the 94.9% reported in the FDA clearance summary, although the study populations and software versions may not have been identical.[1][2]
Who it affects
For a patient, the practical risk is false reassurance: the commercial system was least dependable for subtle or older clots. Radiologists still need to inspect the scan and clinical context even when the AI returns no alert.[1][2]
Keep in mind
This was retrospective work in one academic health system, albeit across 17 facilities; community and rural performance may differ.
The main reference labels came from radiology reports exposed to the AI output, not an independent blinded image-review panel.
Suboptimal and indeterminate examinations were excluded, and software-version differences make the FDA benchmark illustrative rather than like-for-like.
No causal patient outcome—such as time to treatment, complications or mortality—was measured.
Source analysis · 2 linked sources. Original source date: 29 September 2026. Latest supporting source: 21 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
High specificity hid a clinically important miss rate
Researchers at Emory evaluated two Aidoc algorithms already running in clinical practice. On 30,678 dedicated scans, the pulmonary-embolism model reached 86.8% sensitivity and 99.1% specificity. Its sensitivity was below the 94.9% reported in the FDA clearance summary, although the study populations and software versions may not have been identical.
The gap widened where help may be most valuable. Sensitivity declined from 99.3% for saddle emboli to 72.9% for subsegmental emboli, and from 89.7% for acute clots to 65.3% for non-acute ones. A negative flag therefore could not safely end a radiologist's search for a subtle clot.[1][2]
Incidental clots were a harder test
A separate algorithm examined 37,191 routine contrast-enhanced chest and abdominal CT studies, where pulmonary embolism was not the reason for scanning. It detected 336 of 457 confirmed incidental emboli: 73.5% sensitivity, alongside 99.8% specificity.
In a selected subset later given dedicated imaging, it had flagged 42 of 61 emboli confirmed on follow-up and missed 19. That supports using the tool as a second look, not an autonomous clearance step. The study did not measure whether any flag prevented harm or accelerated treatment.[2]
The distinction that matters
Where the algorithm's sensitivity fell
The same system was much more likely to flag a large central clot than a small peripheral or older one.
Saddle clot
Run the dedicated PE model
99.3% sensitivity
Subsegmental clot
Run the same model
72.9% sensitivity
Incidental clot
Run the routine-CT model
73.5% sensitivity
The real-world label was not fully independent
The team extracted labels from final radiology reports with a validated language-model pipeline that achieved 97% accuracy and a kappa of 0.94 against expert labels. But clinicians could see the AI flag, so the reference standard may favour the deployed system. There was no blinded image re-adjudication of every discordant case.
The work was NIH-funded in part; Aidoc supplied aggregated prediction outputs but the authors say it had no role in design, analysis or writing. Prospective, independently adjudicated studies across unrelated health systems—and patient outcomes rather than detection metrics—would change the assessment most.[2]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Evidence check
This was retrospective work in one academic health system, albeit across 17 facilities; community and rural performance may differ.
The main reference labels came from radiology reports exposed to the AI output, not an independent blinded image-review panel.
Suboptimal and indeterminate examinations were excluded, and software-version differences make the FDA benchmark illustrative rather than like-for-like.
No causal patient outcome—such as time to treatment, complications or mortality—was measured.
Reporting basis: Source analysis. Original source date: 29 September 2026. Latest supporting source: 21 October 2026.
Continue the story
Read next
Continue exploring patient safety: compare the finding and its evidence limits.
AI health assistants often endorsed self-diagnoses that needed caution—what did the test measure?
In 100 matched pairs, 27 models were asked when to challenge or accept a plausible self-diagnosis. Even the best current model cautioned appropriately on only 77% of time-critical condition cases, but the benchmark used written scenarios and one physician reviewer.
Read the next reportExplore the full source and evidence record
What this means for people
- For a patient, the practical risk is false reassurance: the commercial system was least dependable for subtle or older clots. Radiologists still need to inspect the scan and clinical context even when the AI returns no alert.
Global context
Both algorithms were evaluated in a US health system under FDA-cleared use. Hospitals elsewhere may use different scanners, protocols, disease prevalence, reporting practices and regulatory versions; local post-deployment monitoring remains necessary.
What the evidence does not yet show
- This was retrospective work in one academic health system, albeit across 17 facilities; community and rural performance may differ.
- The main reference labels came from radiology reports exposed to the AI output, not an independent blinded image-review panel.
- Suboptimal and indeterminate examinations were excluded, and software-version differences make the FDA benchmark illustrative rather than like-for-like.
- No causal patient outcome—such as time to treatment, complications or mortality—was measured.
What to watch next
- Prospective multi-system validation with independent blinded image adjudication
- Performance on suboptimal scans and software-version-specific reporting
- Whether alerts reduce diagnostic delay without creating unsafe reliance or alert fatigue
- Patient outcomes and subgroup estimates with enough cases for narrow confidence intervals
Living evidence record
Impact record IAI-1COCBSE
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
21 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 21 October 2026
Research question, topics and regions
How two FDA-cleared pulmonary-embolism algorithms performed after deployment across a large academic health system
Topics: Medical imaging · Patient safety · Pulmonary embolism · Post-market surveillance · Commercial AI
Regions: United States
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.