Did structured AI preparation help students feel they belonged?
A peer-reviewed trial across six undergraduate classes found a larger 12-week rise in classroom belonging under a guided AI-and-teacher protocol. With only six randomised sections, the exact test could not rule out chance at the conventional 5% threshold.
The short version
- What happened
- Researchers randomised six intact undergraduate class sections at one Chinese university, assigning three sections and 110 students to each condition after a baseline assessment.
- Who it affects
- Students may benefit when AI-supported preparation is designed to expose uncertainty and connect it to responsive teaching, but this study cannot establish that the same gains would occur in another course or university.
- What remains unproven
- Only six intact class sections were randomised, leaving four degrees of freedom for the main section-level contrasts
Keep in mind
Only six intact class sections were randomised, leaving four degrees of freedom for the main section-level contrasts
All exact two-sided randomisation p-values were 0.100, the smallest attainable in the balanced six-section design
One Chinese university limits transfer to other subjects, institutions and education systems
The intervention bundled AI exploration, confidence reflection, question generation and instructor response, so it cannot isolate an AI effect
The confidence–performance discrepancy index was preliminary, and individual-level pathway analyses were exploratory
No measured evidence yet on persistence, attendance, grades or longer-term outcomes
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 5 min The evidence, methods, implications and limitations
The intervention joined AI preparation to teacher follow-through
The study did not test whether giving students a chatbot improves belonging. It tested a combined instructional protocol: students used generative AI to explore material before class, reflected on their confidence, recorded questions they could not resolve and brought those uncertainties into a classroom where the instructor responded. The comparison condition followed the course without that complete structure. The distinction matters because any difference may come from reflection, question generation, teacher responsiveness or their interaction—not from AI alone.
After baseline measurement, the researchers randomised six existing undergraduate class sections at one Chinese university: three sections with 110 students received the structured protocol and three sections with 110 students served as the comparison. The intervention began in week 2, with measures collected in weeks 1, 6 and 12. Analyses gave each class section equal weight, recognising that classmates share an instructor and learning environment and therefore cannot be treated as 220 fully independent randomised units.[1]
The estimated differences favoured the structured protocol
By week 12, the change in classroom-belonging score was 0.361 points larger in the intervention condition. The reported 95% confidence interval ran from 0.145 to 0.576, and the Bonferroni-adjusted parametric p-value was 0.029. The corresponding between-condition changes were 0.282 points for emotional engagement, minus 0.497 points for resistance to teacher guidance, and minus 15.24 points on a preliminary index comparing confidence with performance. In this sample, the package was associated with students feeling more connected and less resistant while judging their own performance more closely.
Those estimates are useful, but the experiment had only four degrees of freedom at the section level. The authors therefore also calculated exact randomisation tests, asking how unusual the observed differences were among all assignments of three of the six sections to the intervention. Every primary two-sided exact p-value was 0.100. That was the smallest attainable value in this design: with so few possible assignments, an exact test cannot cross the conventional 0.05 threshold even when the observed arrangement is the most extreme.[1]
The distinction that matters
Why two statistical readings point to different confidence levels
The same observed difference can look conventionally significant under a parametric model yet remain unresolved under an exact test with only six randomised classes.
Parametric model
Uses section-level estimates and distributional assumptions
Belonging difference: adjusted p=0.029
Exact randomisation
Compares every balanced assignment of six sections
Two-sided p=0.100, the design minimum
Editorial reading
Treats the effect estimate as promising, not settled
Replication needs more independent classes
What a lecturer can use—and what still needs testing
The practical lesson is not that AI creates belonging. It is that an AI preparation task may fit better when students must surface uncertainty and a teacher visibly responds to it. A lecturer considering the model could keep the human chain intact: define the pre-class question, ask students to state confidence and unresolved issues, review the resulting questions, and use class time to address patterns rather than treating chatbot output as finished learning.
The trial was conducted at one university, with six clusters and no independent replication. Students and instructors knew their condition, the confidence–performance discrepancy measure was preliminary, and individual-level pathway analyses were explicitly exploratory and sensitive to measurement timing. The assessment would become stronger with more courses and institutions, preregistered cluster-level analyses, a comparison that separates AI use from reflection and teacher response, and outcomes beyond self-report—such as attendance, persistence, achievement and equitable participation.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Continue the story
Read next
Continue exploring generative ai: compare the finding and its evidence limits.
Do positive views of AI mean students use it reflectively? A 27-person study says not necessarily
In one Chinese programming course, students generally liked the AI tools even as logged activity became more restricted near the final task. The study was exploratory, did not measure learning gains and cannot show that the tool caused the change.
Read the next reportExplore the full source and evidence record
What this means for people
- Students may benefit when AI-supported preparation is designed to expose uncertainty and connect it to responsive teaching, but this study cannot establish that the same gains would occur in another course or university.
Global context
The trial took place at one Chinese university and addresses a wider international problem: how to connect private chatbot use to a shared classroom rather than letting it displace human exchange. The protocol is portable in principle, but the reported effect size is not yet portable evidence.
What the evidence does not yet show
- Only six intact class sections were randomised, leaving four degrees of freedom for the main section-level contrasts
- All exact two-sided randomisation p-values were 0.100, the smallest attainable in the balanced six-section design
- One Chinese university limits transfer to other subjects, institutions and education systems
- The intervention bundled AI exploration, confidence reflection, question generation and instructor response, so it cannot isolate an AI effect
- The confidence–performance discrepancy index was preliminary, and individual-level pathway analyses were exploratory
- No measured evidence yet on persistence, attendance, grades or longer-term outcomes
What to watch next
- A preregistered replication with substantially more independently randomised class sections
- Factorial comparisons that separate AI use from reflection, question generation and instructor responsiveness
- Objective outcomes such as attendance, achievement, persistence and participation across student groups
- The final Version of Record and any changes made during copy-editing
Living evidence record
Impact record IAI-0J9D3Z5
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
26 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 26 October 2026
Research question, topics and regions
Whether a structured protocol combining generative-AI preparation with reflection and instructor response changes classroom belonging and engagement
Topics: Higher education · Classroom belonging · Generative AI · Cluster randomisation
Regions: China · Asia
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.