When the current symptoms were identical, did a psychiatric history change AI hospital advice?
In one synthetic vignette repeated 20,000 times, adding a prior schizophrenia diagnosis raised continued-hospitalisation recommendations from 4.7% to 22.3% when the current evidence supported discharge. No patients or clinical tools were tested.
The short version
- What happened
- Five general-purpose models answered four versions of one synthetic psychiatric vignette 1,000 times each, producing 20,000 responses.
- Who it affects
- If record history exerts excessive influence, a person whose present condition has improved could face more restrictive advice; this study identifies a testable risk, not an observed clinical event.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
One synthetic vignette and one prompt
No patients, clinicians or clinical tool
One described White male patient
Repeated calls are not independent cases
Researcher-defined references are not clinical ground truth
Possible provider-side model changes
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
One line in the record changed the models’ recommendations
An unreviewed counterfactual experiment found that five general-purpose language models treated otherwise identical current evidence differently after a prior schizophrenia diagnosis was added to a synthetic medical record. When the vignette said the patient’s voices had ceased—an experimental condition prespecified to support discharge—the share recommending continued hospitalisation rose from 4.7% without the diagnosis to 22.3% with it. The pooled risk difference was 17.5 percentage points, with a 95% confidence interval of 16.2 to 18.8.
Researchers created four versions of one vignette by crossing current evidence—voices ceased or persisted—with diagnostic history—no prior diagnosis stated or schizophrenia recorded. GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.6 Flash, Grok 4.6 and Mistral Large 2512 each completed 1,000 stateless calls per condition. That produced 20,000 outputs, but it did not involve 20,000 patients or 20,000 distinct clinical situations.[1]
The label mattered most where the current evidence supported discharge
All five models shifted toward continued hospitalisation in the discharge-supporting condition; model-specific risk differences ranged from 10.4 to 34.4 percentage points. Reference concordance fell from 95.3% to 77.7% while the models’ stated confidence increased. When voices persisted and the experimental reference supported continued hospitalisation, adding the diagnosis produced a smaller rise, from 92.6% to 97.0%.
Those results demonstrate label sensitivity, not necessarily an error mechanism or human-like stigma. Psychiatric history can contain relevant prognostic information, and the study cannot decide how much weight would be clinically appropriate. The prompts instructed general-purpose systems to act as psychiatrists, even though none was designed or validated for disposition decisions. Confidence was self-reported text, not a calibrated probability.[1]
The distinction that matters
What changed in the controlled vignette
The experiment altered diagnostic history and current symptoms separately while keeping all other text fixed.
Current evidence
Voices ceased or persisted
Experimental discharge reference changed
Past record
Schizophrenia diagnosis added or omitted
Current facts stayed identical
Measured output
Model chose discharge or hospitalisation
Label sensitivity, not patient outcome
A controlled stress test cannot establish clinical safety
The design used one White male vignette, one prompt and one prior diagnosis. The four versions isolated the selected record change but cannot show whether the pattern extends to other people, diagnoses, questions or clinical environments. Provider-side model changes during eight days of API calls cannot be excluded. The prespecified discharge categories were research references, not judgments from a clinician panel or observed patient outcomes. The study used no patient data and reports no specific grant funding.
Before any system influences psychiatric disposition, it should be tested on richer longitudinal cases that vary history and current presentation independently, with expert review and outcome evidence. Clinicians also need a way to see which record elements drove an answer. The useful conclusion today is narrow: in this repeated synthetic case, a historical label changed recommendations even when the current facts were held constant.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring mental health: compare the finding and its evidence limits.
A small mental-health model beat larger LLMs on one hospital’s records. Can it travel?
Zhixin reached 68.33% exact-match accuracy on 1,020 held-out Chinese clinical records, compared with 49.88% for the strongest prompted comparator. The test came from the same specialist hospital as training, covered an imbalanced subset of diagnoses and did not measure patient outcomes.
Read the next reportExplore the full source and evidence record
What this means for people
- If record history exerts excessive influence, a person whose present condition has improved could face more restrictive advice; this study identifies a testable risk, not an observed clinical event.
Global context
The authors are based in Israel, Sweden and the United States, while the evaluated commercial APIs serve many regions. Clinical standards, legal safeguards and hospital practice differ, and the study does not establish cross-country validity.
What the evidence does not yet show
- Unreviewed preprint
- One synthetic vignette and one prompt
- No patients, clinicians or clinical tool
- One described White male patient
- Repeated calls are not independent cases
- Researcher-defined references are not clinical ground truth
- Possible provider-side model changes
What to watch next
- Expert-reviewed longitudinal cases
- Tests across diagnoses and demographic groups
- Prospective clinical workflow studies
- Calibrated confidence
- Auditable attribution to current versus historical evidence
Living evidence record
Impact record IAI-1DAL96P
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
27 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 27 October 2026
Research question, topics and regions
Whether adding a documented schizophrenia history changes general-purpose language-model recommendations when the current clinical evidence is otherwise identical
Topics: Mental health · Clinical decision support · Diagnostic labels · Bias · Patient autonomy
Regions: Israel · Sweden · United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.