AI-personalised courses looked promising—but four studies left the effect uncertain
A meta-analysis of 420 computing and engineering students found a large pooled difference, but a small-sample correction and prediction interval both included no benefit.
The short version
- What happened
- Only four controlled studies involving 420 students met the review criteria: one cluster-randomized study and three studies with weaker non-randomized or historical comparisons.
- Who it affects
- A student may receive more tailored feedback, but a university cannot yet promise better learning from AI personalisation. Teachers need results from their own course design, assessment and student population.
- What remains unproven
- Four studies
Keep in mind
Four studies
420 students
Three non-randomized comparisons
Two studies at serious risk of bias
I² 89%
Very-low-certainty evidence
No long-term retention
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
A large average with very wide uncertainty
AI-personalised instruction was associated with higher post-course performance across four controlled studies, but the evidence is too small and inconsistent for a confident causal claim, according to a peer-reviewed Frontiers in Education review published on 29 October. The standard random-effects model produced a large pooled standardized difference of Hedges' g 1.01 (95% CI 0.33–1.68). A small-sample Hartung–Knapp adjustment widened that interval to −0.10–2.11, which includes no benefit.
The 95% prediction interval—an estimate of where a comparable new study might land—also crossed zero, from −0.42 to 2.43. Results varied sharply between studies, with I² of 89%. For a lecturer or student, the useful conclusion is therefore not that personalised AI adds one dependable unit of learning. It is that a few small implementations sometimes produced higher scores, while the result expected in another course remains uncertain.[1]
Four studies from 84 search records
The reviewers searched Scopus and Web of Science in April 2026 and screened 84 unique records. Eligible studies needed an AI system that adapted content, feedback, pacing or recommendations, a concurrent or historical comparison group, and an objective academic-performance outcome. Four studies with 420 students qualified, all in computing or engineering courses in Egypt, China or Morocco; no intervention lasted longer than one semester.
Only one study randomized intact groups. One claimed random allocation without explaining the procedure, one used a non-randomized concurrent control and one compared with a previous year's students. Two studies were judged at serious risk of bias; the other two had moderate risk or some concerns. One paper's reported effect size did not match its own means, standard deviations and test statistic, while another depended on an unblinded historical control.[1]
The distinction that matters
Why the headline average is not a dependable forecast
Different uncertainty checks change how confidently the pooled effect can be read.
Ordinary model
Pool four studies
g 1.01; 95% CI 0.33–1.68
Small-sample adjustment
Use Hartung–Knapp
95% CI −0.10 to 2.11
New-study forecast
Calculate prediction interval
−0.42 to 2.43
What teachers still need before changing a course
The review rated the overall certainty very low because of bias, inconsistency and imprecision. It also could not assess publication bias with fewer than ten studies, and its protocol was not registered before the authors knew the data. Some interventions bundled AI with prompt training or other support, so the observed difference cannot always be assigned to personalisation itself. The findings do not address long-term retention, independent exams, accessibility or teacher workload.
The authors reported no financial support and no commercial conflicts. Stronger evidence would come from preregistered multi-course trials that randomize enough classrooms or students, keep grading independent, separate the AI component from co-interventions and report retention and subgroup outcomes. Until then, institutions can pilot carefully and measure local results, but should not market AI-personalised learning as proven to improve achievement.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Continue the story
Read next
Continue exploring higher education: compare the finding and its evidence limits.
Does generative AI improve self-regulated learning?
A meta-analysis of 17 comparative studies found a positive average effect, but results varied sharply: heterogeneity reached 94.85% and the prediction interval included harm as well as benefit. Almost all evidence came from higher education.
Read the next reportExplore the full source and evidence record
What this means for people
- A student may receive more tailored feedback, but a university cannot yet promise better learning from AI personalisation. Teachers need results from their own course design, assessment and student population.
Global context
The studies came from Egypt, China and Morocco, adding evidence outside North America and Western Europe. All were confined to computing and engineering, so the findings should not be generalized to other subjects or education systems.
What the evidence does not yet show
- Four studies
- 420 students
- Three non-randomized comparisons
- Two studies at serious risk of bias
- I² 89%
- Very-low-certainty evidence
- No long-term retention
What to watch next
- Preregistered randomized trials
- Independent grading
- Long-term retention
- Non-STEM subjects
- Teacher-workload effects
- Equity and accessibility outcomes
Living evidence record
Impact record IAI-0VL65L9
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
29 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Education published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 29 October 2026
Research question, topics and regions
Controlled evidence for AI-personalised learning in computing and engineering higher education
Topics: Higher education · Personalised learning · Computing · Engineering · Meta-analysis · Evidence quality
Regions: Egypt · China · Morocco · Middle East · Asia · Africa · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.