Did a GPT-4o virtual patient prepare medical students better? This study found no measurable gain
Only 49 of 96 responding students offered the tool used it. Nervousness, preparedness, self-efficacy and perceived competence were no better than in the comparison group, despite positive user ratings.
The short version
3 key points
- What changed
- The University of Amsterdam allocated 383 first-year medical students by class schedule; 180 answered the post-class questionnaire and 49 used the virtual patient.
- Why it matters
- Medical students may value private rehearsal before a stressful consultation. Educators still need evidence of observed skill and safe feedback before using virtual patients to replace scarce human practice.
- What to watch
- Prospectively registered randomised trials with minimum engagement
Positive reactions did not become a measured learning benefit
Offering first-year medical students an optional GPT-4o virtual patient did not make them less nervous or more prepared for their first standardised-patient consultation, according to a peer-reviewed Dutch study. The University of Amsterdam allocated 383 students by class schedule: 182 received the usual e-learning and 201 also received access to a text-based virtual patient built with retrieval-augmented generation. Only 180 students completed the post-class questionnaire—84 controls and 96 in the intervention group—and 49 intervention respondents actually used the tool.[1]
Who it affects
Medical students may value private rehearsal before a stressful consultation. Educators still need evidence of observed skill and safe feedback before using virtual patients to replace scarce human practice.[1]
Keep in mind
Allocation by class schedule was not random, and the study was not prospectively registered in a public trial registry.
Only 180 of 383 eligible students responded; 49 intervention respondents used the virtual patient.
The outcomes were self-reported after one activity, with no blinded assessment of communication performance or longer-term learning.
The study tested one institution, one case and a voluntary text interface; results may not transfer to other curricula or multimodal systems.
Source analysis · 1 linked source. Original source date: 19 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Positive reactions did not become a measured learning benefit
Offering first-year medical students an optional GPT-4o virtual patient did not make them less nervous or more prepared for their first standardised-patient consultation, according to a peer-reviewed Dutch study. The University of Amsterdam allocated 383 students by class schedule: 182 received the usual e-learning and 201 also received access to a text-based virtual patient built with retrieval-augmented generation. Only 180 students completed the post-class questionnaire—84 controls and 96 in the intervention group—and 49 intervention respondents actually used the tool.
Across nervousness, preparedness, self-efficacy and self-perceived competence, the allocation-based effects were close to zero and every confidence interval included no difference. Comparing the 49 users directly with controls did not change the conclusion. The tool was liked: 63%–71% of users gave positive ratings, and 71% called it a valuable addition. That contrast matters for education buyers: satisfaction and plausibility are not evidence that a product improves skill.[1]
What the comparison did—and did not—test
Students could practise history-taking with one simulated case before a class involving a trained standardised patient. Most users tried the chatbot once: 35 of 49 had one qualifying conversation, eight used it twice and only six used it three or more times. The researchers powered the study to detect a moderate effect, used non-parametric comparisons and corrected 19 item-level tests for multiple comparisons. The observed effects were smaller than the pre-specified small-effect threshold.
This was not a randomised trial. Class schedules determined access, use was voluntary, and only 47% of eligible students answered the questionnaire. Outcomes were students' own ratings after the activity; independent assessors did not score consultation recordings, knowledge, empathy or clinical decisions. The study was not prospectively registered in a public trial registry, and the allocation-based analysis was described as post-hoc. Those limits make the null result informative but not proof that higher-dose or better-designed virtual-patient practice cannot work.[1]
The distinction that matters
From offer to evidence
The participant flow shows why access, use and measured benefit are different denominators.
383 allocated
Split by class schedule
182 control; 201 offered AI
180 responded
Complete post-class survey
47% response rate
49 used it
Hold a qualifying conversation
Mostly one session
4 outcomes
Compare self-ratings
No measurable difference
A supplement needs a purpose, feedback and enough practice
For students and educators, the safest interpretation is that an accessible chatbot can provide low-pressure rehearsal, but one optional text session should not displace trained actors, observation or feedback. The tested system did not automatically assess performance, and the study did not show that it reduced teaching costs. A school adopting a similar tool would need privacy controls, transparent case design, staff oversight and a way to detect clinically unsafe or unrealistic responses before students learn from them.
Amsterdam UMC and University of Amsterdam grants funded the work; the authors declared no commercial conflicts. Confidence would rise with a prospectively registered randomised trial that ensures minimum practice, uses several cases, records engagement and assesses blinded communication performance over time. It should also report withdrawals and compare feedback quality, not merely whether students opened the system in practice. Until then, the study supports usability interest—not a claim that GPT-4o-trained rehearsal improves medical communication.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Evidence check
Allocation by class schedule was not random, and the study was not prospectively registered in a public trial registry.
Only 180 of 383 eligible students responded; 49 intervention respondents used the virtual patient.
The outcomes were self-reported after one activity, with no blinded assessment of communication performance or longer-term learning.
The study tested one institution, one case and a voluntary text interface; results may not transfer to other curricula or multimodal systems.
Reporting basis: Source analysis. Original source date: 19 October 2026.
Continue the story
Read next
Also about medical education
Can AI grade medical students without hiding mistakes?
A prospective US deployment processed 72,907 OSCE rubric scores for 222 students and sharply reduced human scoring passes, but validation focused on the low-scoring tail and physician adjudicators could see which score came from AI.
Read the 7 min reportAlso about medical education
How are Iranian medical students using AI?
In a stratified survey of 384 students at one medical university, research, writing and self-directed learning attracted the highest AI-use preference scores. The study measured stated preferences, not competence, accuracy or patient safety.
Read the 6 min reportExplore the full source and evidence record
What this means for people
- Medical students may value private rehearsal before a stressful consultation. Educators still need evidence of observed skill and safe feedback before using virtual patients to replace scarce human practice.
Global context
Standardised-patient access, privacy rules and medical curricula differ across countries. A locally grounded virtual patient may be easier to scale, but every programme needs its own cases, safeguards and objective skill assessment rather than importing satisfaction scores from Amsterdam.
What the evidence does not yet show
- Allocation by class schedule was not random, and the study was not prospectively registered in a public trial registry.
- Only 180 of 383 eligible students responded; 49 intervention respondents used the virtual patient.
- The outcomes were self-reported after one activity, with no blinded assessment of communication performance or longer-term learning.
- The study tested one institution, one case and a voluntary text interface; results may not transfer to other curricula or multimodal systems.
What to watch next
- Prospectively registered randomised trials with minimum engagement
- Blinded ratings of recorded consultations and longer-term retention
- Multiple cases, automated feedback and multimodal interaction
- Privacy, hallucination and cost comparisons with trained standardised patients
Living evidence record
Impact record IAI-018YVH7
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
19 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Education / Amsterdam UMC published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 19 October 2026
Research question, topics and regions
Whether optional text practice with a retrieval-grounded GPT-4o virtual patient improves first-year medical students' experience of their first standardised-patient consultation
Topics: Medical education · Virtual patients · GPT-4o · Communication training · Student evidence
Regions: Netherlands · Europe · Global education
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.