Aneurysm AI kept 78% accuracy—and still was not ready for the clinic
A peer-reviewed study tested five algorithms on 75 patients and a small outside cohort. Treatment prediction transferred best; rare-complication detection and institutional bias remained serious limits.
The short version
3 key points
- 1.Five algorithms were trained on 75 complete cases from 434 patients identified at one Arizona centre between 2002 and 2023.
- 2.Random-forest treatment prediction reached 78.7% cross-validation accuracy and 78.1% in an outside Louisville cohort, but other outcomes transferred less well.
- 3.Only four training patients had a recorded complication; a 94.7% accuracy result came with 0% sensitivity before class balancing.
The outside test preserved one headline result
A machine-learning model predicted whether people with posterior communicating artery aneurysms received open surgery or endovascular treatment with 78.7% accuracy in repeated cross-validation and 78.1% accuracy at a second US centre. That close result is more informative than another high score from the training hospital alone. Yet it measures agreement with clinicians' past treatment choices—not whether the chosen procedure was best for an individual patient.[1]
Who it affects
People facing aneurysm treatment need advice based on anatomy, rupture status, local expertise and their own risks; a model that reproduces one centre's history could narrow rather than improve that discussion.[1]
Keep in mind
Retrospective complete-case cohort of 75; four recorded complications; selected external sample of 20 most complete cases; one centre supplied training data; no prospective clinical or patient-outcome evaluation.
Source analysis · 1 linked source. Original source date: 16 October 2026.
Read more The full report, methods and context
The outside test preserved one headline result
A machine-learning model predicted whether people with posterior communicating artery aneurysms received open surgery or endovascular treatment with 78.7% accuracy in repeated cross-validation and 78.1% accuracy at a second US centre. That close result is more informative than another high score from the training hospital alone. Yet it measures agreement with clinicians' past treatment choices—not whether the chosen procedure was best for an individual patient.
Researchers identified 434 patients treated at Barrow Neurological Institute in Arizona from 2002 to 2023, then retained 75 with complete medical, operative and imaging records. Fifty-nine had open surgery and 16 received endovascular treatment. Five algorithms were compared across seven targets, including treatment type, intraoperative rupture, complications, angiographic obliteration and discharge disability. Models used repeated stratified five-fold cross-validation, and the primary external analysis selected the 20 most complete cases from a Louisville cohort; endpoint-specific results used between 20 and 28 patients.[1]
Accuracy concealed the rare event the model missed
Only four of the 75 training patients had a recorded complication. A random forest therefore reached 94.7% accuracy for complications while detecting none of the positive cases: sensitivity was 0%. After class balancing, minority-class recall improved across several rare targets, but the small denominators leave wide uncertainty. The paper itself says external complication performance fell to 70.3% and treats the outside metrics as directional rather than precise evidence for deployment.
The training centre's practice pattern also shaped what the model learned. Seventy-nine per cent of its patients had open surgery, a distribution the authors say does not reflect most centres. The 21-year period spanned major changes in endovascular options, and complete-case selection reduced 434 identified records to 75. Some feature-selection descriptions also include downstream variables such as final disability, length of stay and rehaemorrhage, which weakens any claim that the full pipeline is a purely preoperative decision aid even when the immediate target was excluded.[1]
The distinction that matters
Why the same accuracy number can mean different things
A useful clinical model must predict the event that matters, transfer to another hospital and improve a real decision.
Treatment
Predict past procedure choice
78.1% outside accuracy
Complications
Find four rare training events
94.7% accuracy, 0% sensitivity
Clinical use
Guide care prospectively
Not tested
The model is a prototype, not a treatment recommender
External validation varied by target: treatment type transferred best, while procedure subtype and complication results were weaker. The study did not compare AI-assisted decisions with standard multidisciplinary care, test prospective use, measure whether recommendations improve disability or survival, or establish performance across hospitals with different treatment philosophies. The authors describe the models as centre-contextual prototypes and call for prospective, multi-institutional cohorts of 200 to 500 patients.
The article is a peer-reviewed accepted version and may still receive editorial corrections. The authors declared no personal, financial or institutional interest in the drugs, materials or devices described. For patients, the safe interpretation is narrow: external testing found some signal in routine clinical variables, but not a validated answer to whether clipping, coiling or another approach is right. Confidence would require larger representative cohorts, strictly pre-treatment inputs, prespecified outcomes, uncertainty intervals and a prospective comparison of clinical decisions and patient outcomes.[1]
Watch the background, then check the evidence
The Royal Society
AI-generated illustration · not a video frame
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
Healthcare, culture & human impact · Background explainer
Published 2026-07-01 · 44 min 28 sec
Is this the AI future we want? | The Royal Society
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
Evidence check
Retrospective complete-case cohort of 75; four recorded complications; selected external sample of 20 most complete cases; one centre supplied training data; no prospective clinical or patient-outcome evaluation.
Reporting basis: Source analysis. Original source date: 16 October 2026.
Explore the full source and evidence record
What this means for people
- People facing aneurysm treatment need advice based on anatomy, rupture status, local expertise and their own risks; a model that reproduces one centre's history could narrow rather than improve that discussion.
Global context
Both cohorts were in the United States, and the training hospital strongly favoured open surgery. The results cannot establish performance in centres, countries or patient groups with different referral and treatment patterns.
What the evidence does not yet show
- Retrospective complete-case cohort of 75; four recorded complications; selected external sample of 20 most complete cases; one centre supplied training data; no prospective clinical or patient-outcome evaluation.
What to watch next
- Prospective multicentre validation, a locked pre-treatment feature set, confidence intervals by endpoint, sensitivity for rare harm and evidence that decision support improves patient outcomes.
Living evidence record
Impact record IAI-0UPG59B
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
17 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 17 October 2026
Research question, topics and regions
Whether machine-learning models trained at one US neurosurgical centre can predict treatment selection and outcomes for posterior communicating artery aneurysms at another centre
Topics: Clinical AI · Aneurysm care · External validation · Small cohorts · Decision support
Regions: United States · North America
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Health & Life Sciences
Can EHR-note AI help distinguish epilepsy from PNES?
EpiScreen separated epilepsy from psychogenic non-epileptic seizures across 13,633 records from two US datasets and improved accuracy in an eight-clinician simulation. It remains retrospective decision support, not a substitute for neurological assessment or video-EEG.
2 min · 2 sources
Health & Life Sciences
Can an ICU forecast model see kidney injury earlier?
A model trained on 13,277 ICU admissions forecast eight physiological targets and improved retrospective discrimination for 45 moderate-to-severe kidney-injury events in an external cohort. That small event count and untested workflow effects keep it far from clinical deployment.
2 min · 1 source
Health & Life Sciences
An Alzheimer’s MRI model scored 94.5%. On new data, it fell to 61.6%
For a patient, the important test is whether a model works beyond familiar images. This classifier’s accuracy dropped sharply on an external dataset, and it remains a research system rather than a clinical diagnostic tool.
2 min · 1 source
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.