When an AI tutor was confidently wrong, learners often carried the error forward
In a small deception study, 32 adults absorbed planted falsehoods even when the assistant’s overall reliability fell. The experiment is vivid but deliberately extreme and not a deployment error rate.
The short version
- What happened
- Thirty-two mostly young, highly educated and frequent AI users learned five unfamiliar topics from assistants that confidently delivered controlled mixtures of true and false facts.
- Who it affects
- A learner may feel certain because the explanation was fluent, not because it was independently checked. Teachers need visibility into the evidence behind a tutor’s claim and a way to correct misconceptions before assessment.
- What remains unproven
- Only 32 adults participated; 21 were aged 18–24, 27 used AI daily and recruitment used university and personal networks.
Keep in mind
Only 32 adults participated; 21 were aged 18–24, 27 used AI daily and recruitment used university and personal networks.
The tutors were deliberately extreme: they never hedged, never departed from the planted briefing and made errors far more systematically than ordinary systems.
Participants were told not to use outside sources, removing the independent checking route that a real course should provide.
The surprise test was immediate and multiple-choice; delayed free recall and classroom learning may differ.
Source analysis · 1 linked source. Primary analysed source date: 22 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Confidence made the wrong lesson feel learned
Researchers gave each participant five versions of the same GPT-5-mini tutor. Depending on the session, zero to all of 12 briefing facts were replaced with plausible falsehoods, while the assistant was ordered to speak with complete confidence. Quiz scores fell from 7.4 to 2.9 out of 10 across the reliability range. When a false claim appeared in chat, participants adopted it about 65% of the time and rated absorbed errors 8.4 out of 10 for confidence.[1]
Asking the same source again did not provide a check
Participants could return to the tutor after seeing a quiz. Thirty per cent of turns pasted a quiz question back into the same assistant, but that source had been instructed never to concede its briefing. Pasted falsified items received the planted answer 91.5% of the time. Self-reported trust barely moved even as performance worsened, suggesting satisfaction questions alone can miss harmful reliance.[1]
The distinction that matters
The experiment’s trust loop
The assistant’s confidence stayed fixed while the proportion of planted facts changed.
Teach
Confidently present a controlled fact set
0%, 25%, 50%, 75% or 100% falsified
Test
Offer correct, planted and distractor answers
1,040 answers across 104 quizzes
Check
Let learners ask the same assistant again
Often repeated rather than corrected the planted claim
Classroom design needs an independent route to evidence
For teachers and product designers, the practical implication is not to make tutors timid; it is to make individual claims traceable. Curated references, prompts that ask learners to predict before receiving an answer, and a verification route outside the model being checked are testable safeguards. This study does not show that those designs work, and its immediate multiple-choice quiz did not measure durable learning.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Continue the story
Read next
Continue exploring ai tutoring: compare the finding and its evidence limits.
Do agent-style AI tutors improve learning? Two trials suggest gains—but attrition clouds the result
A Tsinghua preprint found a structured legal tutor beat plain chat and a coding workbench improved expert-rated mastery. The programming analysis included only 54 completers from 140 recruits, and the preregistrations are not linked.
Read the next reportExplore the full source and evidence record
What this means for people
- A learner may feel certain because the explanation was fluent, not because it was independently checked. Teachers need visibility into the evidence behind a tutor’s claim and a way to correct misconceptions before assessment.
Global context
The sample and English-language science topics do not support claims about children, other cultures or other languages. The design question—how learners verify unfamiliar claims—still matters wherever conversational tutors are introduced.
What the evidence does not yet show
- Only 32 adults participated; 21 were aged 18–24, 27 used AI daily and recruitment used university and personal networks.
- The tutors were deliberately extreme: they never hedged, never departed from the planted briefing and made errors far more systematically than ordinary systems.
- Participants were told not to use outside sources, removing the independent checking route that a real course should provide.
- The surprise test was immediate and multiple-choice; delayed free recall and classroom learning may differ.
What to watch next
- Larger preregistered replications
- Randomised tests of source links and prediction prompts
- Delayed free-recall outcomes
- Studies with school-age learners and diverse languages
Living evidence record
Impact record IAI-0SZW738
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
23 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 23 October 2026
Research question, topics and regions
Whether learners defer to a confident conversational tutor when controlled falsehoods are mixed into unfamiliar science topics
Topics: AI tutoring · Misinformation · Trust calibration · Learning · Human-AI interaction
Regions: India · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.