Did AI drafting save doctors time? In a 17-doctor trial, not clearly
Physicians editing AI drafts produced higher-rated simulated discharge documents in most quality domains, but the pilot found no statistically significant time difference. It tested six fictional ophthalmology records—not real patients, safety or outcomes.
The short version
- What happened
- Twenty-one Kyoto University Hospital physicians were randomized and 17 completed the pilot, creating discharge summaries and referrals from six simulated ophthalmology records.
- Who it affects
- For clinicians, an AI draft may shift work from writing to checking. This pilot suggests that active review can improve document quality, but it does not show that the total workload falls or that patients receive safer information.
- What remains unproven
- Single ophthalmology department at one Japanese hospital
Keep in mind
Single ophthalmology department at one Japanese hospital
21 physicians randomized and 17 completed the intervention
Six simulated patient records rather than real clinical encounters
No statistically significant time difference for either document type
No patient safety, understanding, follow-up or health outcome
Trial registered after completion
System developer involvement and disclosed related financial interests
Source analysis · 2 linked sources. Primary analysed source date: 26 October 2026. Earliest linked supporting source: 22 September 2025.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Editing an AI draft changed quality more clearly than speed
Researchers at Kyoto University Hospital randomized 21 ophthalmology physicians, with 17 completing the trial. Each worked from six simulated patient records. One group edited drafts generated with Gemini 2.0 Flash-Lite through the CocktailAI system; the comparison group wrote from scratch using matched templates. Unedited model drafts formed a third, non-randomized reference arm.
For discharge summaries, AI-assisted editing was an estimated 59.4 seconds faster than clinician-only drafting, but the 95% confidence interval ranged from 150.6 seconds faster to 31.7 seconds slower (p=.18). For referrals, it was 52.9 seconds slower, with an interval from 7.0 seconds faster to 112.9 seconds slower (p=.08). Neither result established a time saving. Blinded experts nevertheless rated the edited AI drafts higher in most quality domains; unedited AI output was fastest and lowest quality.[1][2]
The pilot supports supervision, not an efficiency claim
The experiment used fictional records in one Japanese specialty, not live electronic records or actual discharge decisions. It did not measure corrections after handover, harmful omissions, patient understanding, clinician burnout or health outcomes. Four randomized physicians did not complete the intervention, and the trial was registered after it finished, although the authors say the protocol and analysis plan were fixed before enrolment.
Fitting Cloud developed the tested system and paid its Gemini API fee. A company board member and employee were co-authors, and several authors disclosed related grants or honoraria; the paper says the company had no organisational role in study conduct or publication decisions. A larger, prospectively registered multi-centre trial using real workflows and downstream error checks would be needed before hospitals could treat this as evidence of saved time or safer care.[1][2]
The distinction that matters
What changed when physicians edited an AI draft
The randomized comparison separated document type, time and expert-rated quality; it did not test patient outcomes.
Summaries
Physicians edited AI drafts rather than starting from a template
59.4 seconds faster; 95% CI crossed no difference
Referrals
The same workflow produced referral letters
52.9 seconds slower; 95% CI crossed no difference
Quality
Blinded experts rated completed documents
Edited drafts led in most domains; raw AI drafts ranked lowest
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring ophthalmology: compare the finding and its evidence limits.
Can an AI agent take a better eye history than a resident?
A randomized trial at a specialist hospital in Guangzhou found that an LLM agent scored higher than ophthalmology residents on structured pre-consultation histories for 172 non-emergency patients. Interviews took much longer, examination findings narrowed the diagnostic gap, and the study did not test patient outcomes or emergency care.
Read the next reportExplore the full source and evidence record
What this means for people
- For clinicians, an AI draft may shift work from writing to checking. This pilot suggests that active review can improve document quality, but it does not show that the total workload falls or that patients receive safer information.
Global context
Clinical-documentation formats, electronic-record systems, staffing and legal responsibility differ across countries. A simulated Japanese ophthalmology workflow cannot establish performance in another specialty, language or health system.
What the evidence does not yet show
- Single ophthalmology department at one Japanese hospital
- 21 physicians randomized and 17 completed the intervention
- Six simulated patient records rather than real clinical encounters
- No statistically significant time difference for either document type
- No patient safety, understanding, follow-up or health outcome
- Trial registered after completion
- System developer involvement and disclosed related financial interests
What to watch next
- Prospectively registered multi-centre trials
- Real-record studies with privacy and safety controls
- Total checking and correction time after handover
- Downstream omission, medication and follow-up errors
- Independent replication without developer involvement
Living evidence record
Impact record IAI-1OU0T66
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
28 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 28 October 2026
Research question, topics and regions
Whether physicians editing large-language-model drafts complete simulated clinical documents faster and to a higher standard than physicians drafting from matched templates
Topics: Clinical documentation · Clinician workload · Large language models · Randomized trials · Ophthalmology · Human oversight
Regions: Japan · Asia
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.