Did a grounded medical AI earn more clinician trust than ChatGPT? Yes—but the study measured perceptions, not accuracy
At a German internal-medicine congress, 64 physicians and medical students rated paired answers from AMBOSS's grounded LiSA system and ChatGPT. LiSA scored higher for perceived quality and trust, but the non-blinded evaluation did not independently score factual correctness, clinical decisions or patient outcomes.
The short version
- What happened
- Sixty-four physicians and medical students each rated one paired response set drawn from 15 real-world internal-medicine questions; the design was paired but non-blinded.
- Who it affects
- Clinicians may find grounded answers more usable, but must still verify evidence and patient-specific context before acting.
- What remains unproven
- The study included 64 congress participants, each evaluating one response pair from 15 questions.
Keep in mind
The study included 64 congress participants, each evaluating one response pair from 15 questions.
It was non-blinded and measured perceptions, not independently adjudicated factual accuracy or patient outcomes.
AMBOSS supplied the questions, helped select the final set and provided LiSA, although it did not fund, analyse or write the study.
The comparison bundles grounding with product, interface, content and familiarity differences.
Source analysis · 1 linked source. Primary analysed source date: 4 November 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 8 min The evidence, methods, implications and limitations
The direct answer: clinicians trusted the grounded system more in this small, non-blinded comparison
A peer-reviewed Scientific Reports study published on 4 November found that a knowledge-base-grounded clinical assistant received stronger perceived quality and trust ratings than ChatGPT from 64 physicians and medical students. Participants attended the 2026 congress of the German Society of Internal Medicine. Each saw paired responses from LiSA, a clinical system developed by AMBOSS, and ChatGPT for one of 15 real-world internal-medicine questions. Mean overall quality was 3.96 for LiSA and 3.48 for ChatGPT, while mean trust was 5.11 and 3.67 respectively. The reported differences were statistically significant.
The result is useful because clinicians assessed material resembling questions they may encounter in practice. It does not show that LiSA produced more factually correct diagnoses or safer care. Raters knew what they were evaluating, and the outcomes were perceptions. A structured, clinically familiar answer can feel more dependable even when an unnoticed error remains. The defensible finding is narrow: in this setting and question set, participants preferred and trusted the grounded system's answers. It is not evidence that either tool should autonomously guide a patient or replace clinical verification.[1]
What the paired design measured—and what its denominator limits
Each participant considered both systems' answers to the same question, helping control for question difficulty within that judgement. The study reports overall quality, trust, clarity, completeness, actionability, perceived medical accuracy and safety concerns. LiSA scored higher for completeness, actionability and perceived accuracy; clarity did not differ significantly. The authors interpret that pattern as evidence that the perceived advantage was not simply smoother wording. Yet every person assessed only one of 15 question pairs, so the dataset is not a comprehensive examination of internal medicine and does not give each question a large independent panel.
Participants were recruited at a professional congress rather than sampled across clinical roles, specialties and health systems. The exercise captures a moment of judgement, not repeated use during ward pressure. It cannot show whether familiarity improves calibration or encourages automation bias. A stronger crossed design would have multiple clinicians independently rate every question while blinded to system identity, then separate variation attributable to raters, cases and systems. A larger case set should include ambiguous presentations, missing context, rare conditions and changing guidance rather than only questions selected for this product comparison.[1]
The distinction that matters
What the study found—and what it did not test
The paired evaluation separated perceived qualities, but did not independently adjudicate clinical correctness or outcomes.
Participants
64 physicians and medical students
Each rated one of 15 paired responses
Higher ratings
Quality, trust and actionability
LiSA scored higher on perceived measures
No difference
Clarity
No significant difference reported
Uncertain gap
Safety concerns: 15.6% versus 26.6%
Difference was not statistically significant
Not measured
Objective accuracy and outcomes
No effectiveness conclusion
The safety result is more uncertain than the trust result
Safety concerns were recorded in 15.6% of LiSA evaluations and 26.6% of ChatGPT evaluations. Although the raw percentages look different, the reported p value was 0.143. The study therefore did not establish a statistically significant safety-concern difference under its analysis. It would be wrong to convert the percentages into a known reduction in unsafe answers. The paper also does not provide a clinical adjudication showing whether every concern represented a genuine hazard or whether unflagged answers contained hidden errors.
Trust is not automatically beneficial. Appropriate trust helps clinicians use a reliable tool; excessive trust can suppress checking. A grounded answer may expose recognisable concepts and appear more complete, which could improve scrutiny, but it can also sound authoritative. The safety question is whether users know when confidence should fall, whether sources support the exact claim, and whether a system signals uncertainty or missing context. Those properties require objective error analysis and prospective workflow testing. Perceived safety is a relevant human-factors outcome, but it is not a surrogate for misdiagnosis, medication harm or patient benefit.[1]
Grounding was not isolated as the cause
LiSA is described as knowledge-base-grounded, whereas ChatGPT is a general-purpose system. The comparison therefore bundles source material, retrieval, prompting, product interface, medical editing, response format, model choice and participant familiarity. Researchers did not take one model and randomly switch grounding on or off while holding everything else constant. They cannot establish that retrieval alone caused the higher ratings. Nor did they test whether citations faithfully supported claims, whether the knowledge base was current for every question, or how either system behaved when evidence was incomplete.
A post hoc analysis found larger quality and trust differences among current AMBOSS users than among non-current users. That may reflect useful familiarity with the product's structure, but it may also reflect recognition, expectations or preference. Because outputs were non-blinded, those explanations cannot be separated. Researchers could isolate the mechanism with blinded outputs, common formatting, an identical base model, controlled retrieval conditions and answer-level adjudication against a predefined reference. Until that is done, product teams should not present this paper as a clean causal test proving that grounding itself created superior clinical answers.[1]
The commercial role is disclosed and should shape interpretation
AMBOSS provided the clinical questions, contributed to study design and selection of the final question set, and supplied access to LiSA. The paper says AMBOSS did not fund the work, had no access to study data and was not involved in analysis, interpretation or manuscript preparation. The authors declare no other competing interests. Open-access publication was supported by Philipps-University Marburg. These disclosures do not invalidate the result, but they are central context because the company behind the evaluated system influenced which questions entered the comparison.
Question selection can shape a benchmark without touching the final statistics. A grounded knowledge product may perform especially well where its curated content is strong, while results may differ on rarer conditions, incomplete prompts or rapidly changing guidance. A robust follow-up should preregister selection criteria, include independently sourced cases, preserve difficult items and report results by question. External replication with other knowledge bases and general models would test whether the pattern travels beyond one product. Commercial clinical evidence becomes more persuasive when the evaluated company cannot choose the test set and scoring material can be independently inspected.[1]
What changes for clinicians—and what would change this assessment
For clinicians, the study supports a hypothesis rather than a procurement verdict: answers anchored in a clinical knowledge base may feel more usable than general-purpose output. Before consequential use, a clinician still needs to verify the evidence, patient-specific contraindications, local guidance and whether the question omitted decisive context. A high perceived-quality score does not transfer accountability. Health organisations should distinguish educational queries from medication, diagnosis and escalation decisions, and require stronger checks where an answer could directly affect a patient.
Confidence would rise with a preregistered, blinded, multi-centre study using an independently assembled question set. Several clinicians should adjudicate each answer for factual errors, missing qualifications, harmful recommendations and citation support. A prospective trial could then measure whether decisions improve, information is found faster and errors are caught without over-reliance. The assessment would weaken if higher trust persisted despite equal or worse accuracy. This is consequential research but not Breaking: it is a controlled perception study, not an urgent clinical warning, regulatory approval or active patient-safety incident.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring clinical ai: compare the finding and its evidence limits.
Neonatologists want AI as a second set of eyes—not the final word
Seventeen clinicians at one Dutch academic hospital welcomed AI for monitoring fragile newborns, but tied trust to local validation, human override and skills that must survive automation. The interviews reveal conditions for adoption, not evidence that an AI tool improved care.
Read the next reportExplore the full source and evidence record
What this means for people
- Clinicians may find grounded answers more usable, but must still verify evidence and patient-specific context before acting.
- Patients should not read perceived trust as proof of safer diagnosis, treatment or outcomes; those endpoints were not tested.
- Health organisations need objective error and workflow evidence before treating this study as procurement validation.
Global context
The study concerns German internal medicine and a particular commercial knowledge product. Health systems elsewhere use different guidance, languages, staffing and accountability rules, so the ratings cannot be assumed to transfer. The wider question is whether clinically curated retrieval improves objective decisions without creating misplaced confidence. That needs independent tests across specialties, regions and patient groups, including settings where reference knowledge and workflow differ.
What the evidence does not yet show
- The study included 64 congress participants, each evaluating one response pair from 15 questions.
- It was non-blinded and measured perceptions, not independently adjudicated factual accuracy or patient outcomes.
- AMBOSS supplied the questions, helped select the final set and provided LiSA, although it did not fund, analyse or write the study.
- The comparison bundles grounding with product, interface, content and familiarity differences.
What to watch next
- Blinded independent scoring of accuracy and citation fidelity.
- Multi-centre tests with independently selected cases.
- Prospective workflow and patient-safety outcomes.
- Whether trust stays calibrated when the knowledge base is incomplete.
Living evidence record
Impact record IAI-0VZCGBQ
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 November 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 5 November 2026
Research question, topics and regions
How clinicians and medical students perceived answers from a knowledge-base-grounded clinical AI compared with ChatGPT, and why perceived trust is not evidence of objective clinical accuracy
Topics: Clinical AI · Medical questions · Grounded systems · Trust · Safety · Human evaluation
Regions: Germany · Europe
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.