Simpler diabetes advice was easier to read—but no AI prompt preserved every strength
In 384 model answers, a sixth-grade prompt improved readability while doctor-style prompts produced more comprehensive responses. The study tested text, not whether patients understood it or made safer decisions.
The short version
- What happened
- Researchers generated 384 answers to 16 type 2 diabetes-remission questions, crossing four language models with six prompting strategies; two experienced endocrinologists rated every response.
- Who it affects
- People seeking diabetes information need language they can understand without losing qualifications about eligibility, duration and medical supervision. This experiment identifies a design tension but does not show which wording helps a real patient make a better decision.
- What remains unproven
- Sixteen curated questions on one clinical concept
Keep in mind
Sixteen curated questions on one clinical concept
Synthetic responses only
Two clinician raters
No patient comprehension or outcome test
Readability formulas do not measure understanding
Exploratory correlations did not account for question clustering
Model versions will change
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Clearer language came with a measurable trade-off
A peer-reviewed study published on 29 October found no single way to prompt a language model that produced the strongest diabetes-remission advice on every measure. Researchers in China asked four systems—DeepSeek-V3.2, GPT-5.5, Gemini 3.1 Pro and Claude Sonnet 4.6—the same 16 core questions using six prompt styles. That balanced design produced 384 responses, all of which were included in the analysis.
The clearest contrast came from audience framing. The sixth-grade prompt reached a mean Flesch Reading Ease Score of 69.70, the best of the six strategies, while a doctor prompt scored 27.50, the hardest to read. Yet the sixth-grade prompt had the lowest descriptive composite quality score, 4.066 out of five, and the doctor prompt the highest, 4.350. Doctor prompts also had the highest mean comprehensiveness score, 4.609; patient prompts had the highest mean empathy score, 4.359.[1]
The comparison is useful, but it is not a patient trial
Two endocrinologists with more than five years of clinical experience independently scored accuracy, relevance, comprehensiveness, empathy and falsehood avoidance on five-point scales. The authors used mixed-effects models with the question as a random intercept. Model choice affected all five domains, although the reported effect sizes were mostly small or modest. GPT-5.5 had the highest numerical mean for accuracy and falsehood avoidance; DeepSeek led numerically on relevance, comprehensiveness and empathy. Those rankings are descriptive, not proof that one model is safest.
The study did not recruit patients, test understanding, measure treatment choices or observe clinical outcomes. Readability formulas count textual features; they do not establish that someone grasped the difference between remission and cure. The exploratory correlations also analysed responses without accounting for clustering by question, so they are hypothesis-generating. For clinicians and health-information teams, the practical lesson is to test both clarity and medical completeness with real intended readers, rather than treating a single prompt as a safety control.[1]
The distinction that matters
One prompt changed several qualities at once
The same prompt could make an answer easier to read while reducing other qualities in the study's rubric.
Sixth-grade prompt
Mean reading ease 69.70
Easiest to read; lowest composite quality
Doctor prompt
Mean reading ease 27.50
Hardest to read; most comprehensive
Patient prompt
Mean empathy 4.359/5
Highest empathy; not best on every measure
Funding and the next evidence
The work was funded by China's National Science and Technology Major Project, the China Academy of Chinese Medical Sciences and a hospital talent-development programme. The authors declared no competing interests. The publisher labels the paper peer-reviewed and citable, while noting that this accepted early version will receive further editing.
Confidence would rise with blinded evaluations involving more clinical topics, languages and independent patient groups; preregistered comparisons; and direct measures of comprehension, decision quality and harmful misunderstanding. Model versions also change, so this 2026 snapshot should not be treated as a permanent product ranking.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring patient information: compare the finding and its evidence limits.
Can AI explain bowel leakage safely? A 20-answer study found weak next-step guidance
Three clinicians rated answers from four chatbots to five commonly searched phrases. The median actionability score was 40%, and every model produced inaccurate or irrelevant statements.
Read the next reportExplore the full source and evidence record
What this means for people
- People seeking diabetes information need language they can understand without losing qualifications about eligibility, duration and medical supervision. This experiment identifies a design tension but does not show which wording helps a real patient make a better decision.
Global context
The researchers and funding institutions were based in China, while the tested commercial and open models are used internationally. The English-language design does not establish equivalent performance across languages, health systems or literacy contexts.
What the evidence does not yet show
- Sixteen curated questions on one clinical concept
- Synthetic responses only
- Two clinician raters
- No patient comprehension or outcome test
- Readability formulas do not measure understanding
- Exploratory correlations did not account for question clustering
- Model versions will change
What to watch next
- Patient comprehension studies
- Multilingual replication
- Blinded independent raters
- Harmful-misunderstanding measures
- Results after model updates
Living evidence record
Impact record IAI-1ELO5Y1
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
29 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 29 October 2026
Research question, topics and regions
Quality and readability of AI-generated patient information
Topics: Patient information · Type 2 diabetes · Medical AI · Readability · Large language models
Regions: China · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.