AI drafts sped selected patient-message replies—why didn't after-hours work fall?
A study of 983,154 message responses found faster turnaround when clinicians used an AI draft, but deployment did not reduce after-hours replying overall. Use was selective and observational, so the result does not prove the tool caused the difference.
The short version
- What happened
- The peer-reviewed cohort covered 983,154 first responses from 1,542 clinicians to messages from 272,091 patients at one US academic medical centre between March 2023 and May 2025.
- Who it affects
- Clinicians may find some replies easier to start, but the study offers no evidence that the tool reduced their total inbox workload. Patients may receive selected first responses sooner; the paper did not measure whether those replies were more accurate, understandable or sufficient to resolve the issue.
- What remains unproven
- This was a single-centre retrospective cohort, not a randomised trial. Message-level tool use was selective and residual time-varying confounding may remain.
Keep in mind
This was a single-centre retrospective cohort, not a randomised trial. Message-level tool use was selective and residual time-varying confounding may remain.
The study measured first-response turnaround and after-hours sending, not editing time, cognitive burden, reply quality, clinical safety, downstream messages or patient outcomes.
The message classifier was 86.2% accurate, and the tool's exact underlying model configuration could not be confirmed.
After-hours subgroup results were sensitive to how working hours were defined and need replication.
Source analysis · 1 linked source. Primary analysed source date: 30 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Using a draft and merely having the tool were different exposures
Columbia University researchers separated enterprise deployment from actual message-level use. They analysed every first clinician response to a patient-portal message at Columbia University Irving Medical Center from March 2023 to May 2025. The Epic In Basket tool, which used an OpenAI large language model in the institution's protected environment, was deployed in December 2024. The authors used clinician fixed-effects models to compare timing while accounting for stable differences between clinicians.
Across 797,799 pre-adoption and 185,355 post-adoption responses, adoption was associated with an 8.6% increase in turnaround time. Within the post-adoption period, clinicians used an AI draft for 9,757 messages—about one in 19—and those messages had 29.5% shorter turnaround than messages without the tool. The median was four hours with a draft and six hours without one. This does not mean every message would become faster if the tool were forced on: clinicians may have selected drafts for messages that were easier, more urgent or otherwise different in ways the model could not observe.[1]
Faster selected replies did not remove the wider burden
Neither adoption nor use was associated with an overall reduction in after-hours replies. Laboratory-result messages received during working hours showed 3.67 percentage points fewer after-hours responses when a draft was used, while messages received after hours showed 4.52 points more. Those subgroup findings changed under a broader definition of working hours, so they should not be treated as settled effects. The analysis also counted only the first reply, not editing time, message quality, later exchanges or whether patients needed another contact.
For clinicians, the practical lesson is narrower than 'AI saves time'. A draft may lower the barrier to answering some messages, yet a tool that is rarely used may not change total workload. For patients, quicker first replies may matter, but speed is not the same as accuracy, clarity or resolution. A stronger test would randomise eligible messages, measure editing and total conversation time, audit clinical quality and safety, and repeat the study across health systems. One author disclosed CDC, NIH and PCORI grants during the study; the authors could not verify the exact underlying model configuration.[1]
The distinction that matters
Three comparisons that should not be collapsed into one claim
The study separated access to the tool, actual use on a message and work outside normal hours.
Tool adopted
Compare all messages before and after deployment
Turnaround was 8.6% longer after adoption
Draft used
Compare post-adoption messages with and without a draft
Selected messages using a draft had 29.5% shorter turnaround
After-hours work
Test whether replies moved into normal hours
No overall reduction was detected
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Continue the story
Read next
Continue exploring electronic health records: compare the finding and its evidence limits.
Can clinical AI recognise when the patient record does not support an answer?
A new clinical-agent benchmark raises a practical question for health systems: can an assistant explain what the record cannot establish? Our analysis examines evidence, local testing and the burden on staff.
Read the next reportExplore the full source and evidence record
What this means for people
- Clinicians may find some replies easier to start, but the study offers no evidence that the tool reduced their total inbox workload. Patients may receive selected first responses sooner; the paper did not measure whether those replies were more accurate, understandable or sufficient to resolve the issue.
Global context
Patient portals have shifted a substantial volume of administrative work to clinicians. Health systems are adding draft tools before evidence is mature enough to assume that faster replies translate into less work. This study's value is the contrast between a promising message-level association and the absence of an organisation-wide burden reduction.
What the evidence does not yet show
- This was a single-centre retrospective cohort, not a randomised trial. Message-level tool use was selective and residual time-varying confounding may remain.
- The study measured first-response turnaround and after-hours sending, not editing time, cognitive burden, reply quality, clinical safety, downstream messages or patient outcomes.
- The message classifier was 86.2% accurate, and the tool's exact underlying model configuration could not be confirmed.
- After-hours subgroup results were sensitive to how working hours were defined and need replication.
What to watch next
- Randomised message-level evaluations
- Editing and total conversation time
- Reply accuracy and escalation safety
- Use across specialties and health systems
- Patient understanding and resolution
- Whether higher-quality drafts increase safe uptake
Living evidence record
Impact record IAI-1M3F60M
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
31 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what JAMA Network Open published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 31 October 2026
Research question, topics and regions
Topics: Clinical work · Patient messaging · Generative AI · Workload · Electronic health records · Human review
Regions: North America · United States
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.