AI feedback matched one teacher in an essay trial—but motivated students less
In a randomized study of 312 Chinese undergraduates, Qwen feedback and one instructor's comments produced equivalent revised-essay scores within a prespecified margin. Teacher feedback led to slightly higher intrinsic motivation; the result covers one task and one revision cycle.
The short version
- What happened
- Researchers randomized 312 first-year undergraduates to locally generated Qwen feedback, feedback from one experienced instructor or a minimal-prompt control; 287 were analysed after 8% attrition.
- Who it affects
- Teachers could use AI to expand the supply of rubric-based comments, but this study supports a blended role rather than removing human contact from feedback.
- What remains unproven
- Single Chinese university, first-year cohort, argumentative prompt, instructor, model and one-week feedback cycle; no course-level learning or retention outcome.
Keep in mind
Single Chinese university, first-year cohort, argumentative prompt, instructor, model and one-week feedback cycle; no course-level learning or retention outcome.
The study was not preregistered. Twenty-five of 312 randomized students were omitted from complete-case analysis, and no intention-to-treat analysis was reported.
Students knew the feedback source, and the relatedness and motivation pathway was measured rather than experimentally manipulated; the paper treats mediation as associational.
The authors reported no funding or commercial conflicts. Data are available by inquiry rather than through a public repository.
Source analysis · 1 linked source. Primary analysed source date: 30 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Comparable revisions, different motivation
AI feedback helped students revise one argumentative essay about as well as feedback from a single instructor, but it supplied less motivational lift. A Chinese university randomized 312 first-year undergraduates to Qwen2.5-27B-Instruct feedback, teacher feedback or a control prompt without essay-specific comments. After 25 students were lost or excluded, 287 revisions were scored by two raters who did not know the assignment.
Controlling for the first draft, adjusted scores were 54.70 for AI, 54.58 for the teacher and 48.28 for control. Both substantive-feedback groups beat control by about 0.6 standard deviations, and the AI-teacher difference fell within a prespecified equivalence margin of ±0.35 standard deviations. On a seven-point motivation scale, however, teacher feedback averaged 5.16 versus 4.90 for AI and 4.61 for control.[1]
A useful classroom tool, not a teacher-replacement result
The locally hosted model used a standardized rubric covering thesis, argument, evidence and clarity; researchers reviewed its output but did not edit or regenerate it. One instructor with six years of university writing experience produced the comparison comments. Students knew which source they received, so the motivation gap may partly reflect expectations about people and machines rather than an unavoidable property of AI feedback.
This was one 400-word task, one institution and one revision a week later. The paper was not preregistered, used complete cases rather than all randomized students, and the revised drafts scored lower than first drafts in every group under different writing conditions—the valid claim is a relative advantage over control, not absolute improvement. Repeated, multi-teacher trials would test whether blended feedback saves time without weakening motivation or learning over a course.[1]
The distinction that matters
What the randomized comparison found
The performance result is narrow; motivation separated the two substantive feedback sources.
AI feedback
Qwen comments on one draft
Adjusted revision score 54.70; motivation 4.90/7
Teacher feedback
One instructor's comments
Adjusted revision score 54.58; motivation 5.16/7
Control
No essay-specific comments
Adjusted revision score 48.28; motivation 4.61/7
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Continue the story
Read next
Continue exploring randomized trials: compare the finding and its evidence limits.
AI tutor access lowered grades in a university trial — but few students used it
A randomized Maryland study found lower grades and course-platform activity after instructors received a course-integrated assistant. Uptake was only about 15%, and the paper is not peer reviewed.
Read the next reportExplore the full source and evidence record
What this means for people
- Teachers could use AI to expand the supply of rubric-based comments, but this study supports a blended role rather than removing human contact from feedback.
- Students received equivalent short-term revision outcomes in this controlled task, while teacher comments carried a modest motivational advantage that may matter across repeated assignments.
Global context
Universities worldwide are testing automated writing feedback to manage staff workload. This trial adds controlled evidence from China, but language, assessment culture, teacher practice and student expectations could change both performance and motivation elsewhere.
What the evidence does not yet show
- Single Chinese university, first-year cohort, argumentative prompt, instructor, model and one-week feedback cycle; no course-level learning or retention outcome.
- The study was not preregistered. Twenty-five of 312 randomized students were omitted from complete-case analysis, and no intention-to-treat analysis was reported.
- Students knew the feedback source, and the relatedness and motivation pathway was measured rather than experimentally manipulated; the paper treats mediation as associational.
- The authors reported no funding or commercial conflicts. Data are available by inquiry rather than through a public repository.
What to watch next
- Preregistered multi-institution trials with intention-to-treat analysis
- Repeated assignments measuring durable writing gains rather than one revision
- Several teachers, languages, disciplines and newer or smaller models
- Direct tests of teacher-endorsed or teacher-edited AI feedback
- Workload, cost, privacy and error outcomes in ordinary classrooms
Living evidence record
Impact record IAI-1E65BMC
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
31 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Psychology published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 31 October 2026
Research question, topics and regions
Whether AI-generated writing feedback can match teacher feedback on revision quality without losing motivational value
Topics: Education · Writing feedback · Student motivation · Generative AI · Randomized trials
Regions: Asia · China
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.