AI tutor access lowered grades in a university trial — but few students used it
A randomized Maryland study found lower grades and course-platform activity after instructors received a course-integrated assistant. Uptake was only about 15%, and the paper is not peer reviewed.
The short version
3 key points
- 1.The instructor-level randomized trial covered 30 instructors, 34 course sections and 2,379 undergraduates at one US university during autumn 2025.
- 2.In the researchers' closest matched sample, access to the assistant was associated with final grades 4.12 percentage points lower and substantially less course-platform activity.
- 3.Only about 15% of eligible students used the assistant, most instructors left it in direct-answer mode, and students in both groups could use other AI tools.
Access, rather than regular use, was randomized
Giving instructors a course-integrated AI assistant lowered student grades and online course activity in a randomized University of Maryland trial. The result is a warning against assuming that access alone improves learning, not evidence that every AI tutor harms students. Only about 15% of eligible students used the tool, and most participating instructors left it in a mode designed to provide direct explanations and answers rather than guided practice.
Researchers assigned instructors within 13 matched blocks to receive the assistant or continue teaching without it during autumn 2025. The final study covered 30 instructors, 34 course sections and 2,379 undergraduates across 11 of the university's 12 schools. The assistant used GPT-4o with retrieval from course materials. Students in both groups could still use public tools such as ChatGPT or Gemini, so the experiment measured the effect of offering this particular course tool, not AI use versus no AI use.[1][2]
Grades and course-platform activity both moved down
In the authors' exact-match analysis of 22 instructors and 1,353 students, access reduced final grades by an estimated 4.12 percentage points, equivalent to 0.37 standard deviations. It also corresponded to about 721 fewer course-page views, 7.19 fewer active days and 26.39 fewer participation actions. The full 2,379-student analysis estimated a smaller 2.88-point grade decline with weaker statistical precision, while the engagement measures remained negative. Homework scores did not show a clear improvement.
The activity pattern matters because it offers a plausible route from the intervention to the result: students may have substituted quick answers for course engagement. But the trial did not directly establish that mechanism. Roughly three-quarters of recorded requests sought information, explanations or solutions; only 11% concerned practice or test preparation. By the term's end, two instructors were using the more scaffolded tutor mode. Average use among users was about four sessions, leaving a large gap between being assigned access and relying on the tool.[1]
The distinction that matters
What the randomized trial actually compared
The trial changed whether an instructor's class received one integrated assistant. It did not prevent students in the comparison group from using other AI.
Assignment
Randomize instructors within 13 matched blocks
Treatment classes received the course assistant; comparison classes did not
Actual use
Students chose whether to open the assistant
About 15% used it, averaging roughly four sessions among users
Semester outcomes
Compare grades and learning-platform records
Final grades and several engagement measures were lower in the access group
Universities should test the teaching design, not just the chatbot
For lecturers, the practical lesson is to decide what kind of learning behaviour an assistant should encourage, then measure whether students actually use it that way. A tool that retrieves accurate course material can still change when students open the learning platform, attempt problems or seek feedback. The study supports staged deployment with usage logs, learning outcomes and an alternative for students who do not want an AI assistant; it does not justify a blanket conclusion that guided tutoring cannot work.
This is a working paper from one US public university, not a peer-reviewed multi-campus trial. Assignment occurred at instructor level, leaving only 34 course-section clusters, and the authors warn that conventional standard errors may overstate precision. Their larger estimate for first-generation students is concerning but comes from subgroup tests with additional uncertainty and some imbalance in the matched sample. The university both developed and evaluated the system. A preregistered replication with more instructors, genuinely scaffolded tutor use, independent evaluation and direct measures of study behaviour would materially change confidence in the result.[1][2]
Evidence check
The working paper is not peer reviewed, comes from one institution, randomizes only 30 instructors across 34 sections, records low student uptake and cannot separate the effect of direct-answer design from broader changes in AI use.
Reporting basis: Source analysis. Original source date: 16 October 2026.
Explore the full source and evidence record
What this means for people
- Students may receive faster course-specific answers while spending less time in the wider learning environment; universities need to check both convenience and learning outcomes.
- First-generation students showed a larger estimated grade decline, but the subgroup evidence is too uncertain to treat as a settled effect.
Global context
The trial took place at one large US university and does not establish how course assistants will perform in other countries, age groups, subjects or teaching systems. Its most transferable finding is the need to evaluate the instructional design and actual use, not the model label alone.
What the evidence does not yet show
- The working paper is not peer reviewed, comes from one institution, randomizes only 30 instructors across 34 sections, records low student uptake and cannot separate the effect of direct-answer design from broader changes in AI use.
What to watch next
- Replications with more instructor clusters and campuses, preregistered outcomes, sustained use of scaffolded tutor mode, independent evaluation and evidence on whether students substitute the assistant for productive practice.
Living evidence record
Impact record IAI-19EXVON
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
16 October 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Topics: AI tutoring · Higher education · Student learning · Randomized trials · Generative AI
Regions: United States
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
UNESCO
AI-generated illustration · not a video frame
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
Education & human agency · Background explainer
Published 2025-01-23 · 8 min 30 sec
Teacher to Teacher: Is AI reshaping education for the better?
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Education
Does open AI access still let teachers see student judgement?
In one 33-manuscript university course, structural writing clustered near the top while evidence use and theoretical interpretation still varied. The case supports assessment redesign, not a claim that AI improved learning.
7 min · 1 source
Education
What do students say they need from AI policy?
In a trilingual survey of 186 mostly Indonesian and Hungarian respondents, routine AI use coexisted with requests for funded access and clearer rules. The self-selected sample cannot measure national differences, learning gains or policy effectiveness.
9 min · 1 source
Education
Does generative AI improve self-regulated learning?
A meta-analysis of 17 comparative studies found a positive average effect, but results varied sharply: heterogeneity reached 94.85% and the prediction interval included harm as well as benefit. Almost all evidence came from higher education.
7 min · 1 source
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.