The same facts were wrong 15.9% of the time in Uzbek—and model choice narrowed the gap
Eight open models answered the same 100 questions in English, Russian and Uzbek. Language-adapted 7–8B models performed far better in Uzbek, but this was a model audit, not a study of students or learning.
The short version
- What happened
- Eight open-weight models answered 100 matched factual questions in English, Russian and Uzbek: 2,400 deterministic prompts in a fully crossed design.
- Who it affects
- A student working in Uzbek may receive a confident false fact from the same model that answers correctly in English. Institutions can test and choose tools in the language actually used, rather than treating English performance as a proxy.
- What remains unproven
- 100 short-answer questions
Keep in mind
100 short-answer questions
Three languages
Eight open-weight models
No proprietary frontier systems
One deterministic sample per prompt
Refusals excluded from error denominator
Machine-assisted translations
No student or learning outcome
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
The language of the question changed the answer risk
Eight open language models made factual errors on 15.9% of scored Uzbek answers, compared with 10.9% in Russian and 3.5% in English, in a peer-reviewed Frontiers in Education study published on 29 October. The 95% confidence intervals were 13.5%–18.6% for Uzbek, 8.8%–13.4% for Russian and 2.4%–5.0% for English. The overall language difference had a medium reported effect size, although the practical gap varied sharply by model and topic.
That matters to a student using a chatbot as an informal study aid: the answer may sound equally fluent while being more likely to name the wrong person, number or scientific fact. The widest pooled English-to-Uzbek gaps appeared in Central Asian regional knowledge, general science and attribution of works or discoveries. This experiment measured model answers, however; it did not observe students, assignments, trust or learning.[1]
Every fact was tested across three languages and eight models
Researchers at Kokand University built 100 short-answer questions with unambiguous answers: 16 geography, 17 science, 17 history, 16 elementary mathematics, 17 attribution and 17 Central Asian regional items. Each question was asked in English, Russian and Uzbek to eight open-weight models ranging from one to 12 billion parameters, producing 2,400 prompts. Russian and Uzbek translations were drafted with machine assistance and checked by native-speaking authors. Models produced one greedy-decoded answer at temperature zero.
Scoring used accepted answer forms rather than another language model. Misspellings and recognisable transliterations counted as correct; wrong or unresolvable referents did not. Authors manually reviewed every automatic incorrect or refused verdict. Refusals were excluded from the denominator—a material choice because the one-billion-parameter Uzbek-adapted model returned only a question mark on 83 of 100 Russian prompts. Its apparent error rate therefore does not capture that severe non-answer behaviour.[1]
The distinction that matters
One question bank, three language conditions
The fully crossed design kept the fact and model fixed while changing the language of the prompt.
English
800 model-question combinations
3.5% pooled factual error
Russian
800 model-question combinations
10.9% pooled factual error
Uzbek
800 model-question combinations
15.9% pooled factual error
Adaptation helped, but the evidence stays narrow
Two Uzbek-adapted mid-sized models recorded 2% and 8% error in Uzbek. By contrast, general Gemma 3 4B and Qwen3 4B models reached 24% and 25%; a larger general Gemma 3 12B reached 5%. The pattern suggests that targeted language exposure and general capability both matter. It does not prove that any named model is safe for coursework, because the bank tests short facts rather than reasoning, citation, dialogue or subject teaching.
The audit excluded proprietary frontier systems, covered only three languages and sampled each prompt once. Translation artefacts may remain, and the authors inferred educational consequences without a learner study. They reported no funding or financial conflicts and disclosed generative-AI help with drafting and language editing. A university can use the released question bank as a starting point, but a stronger assessment would add local curriculum questions, count refusals as a separate failure, involve independent scorers and test whether students detect and correct errors before those errors enter assessed work. Evidence that students reliably verify the replies—or that adaptation fails on a broader local syllabus—would materially change the practical assessment.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 8 min 30 sec · Background explainer
Teacher to Teacher: Is AI reshaping education for the better?
YouTube loads only when you choose; playback starts when you press play.
A conversation about how teachers can guide AI use while preserving pupils’ judgement. Watch for the difference between helping a learner think and supplying an answer.
Evidence check: Teachers’ perspectives provide context. They do not establish that a particular AI tool improves learning outcomes.
Published 2025-01-23. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the education evidencePhoto: Vitaly Gariev / Unsplash · Licence
Continue the story
Read next
Continue exploring higher education: compare the finding and its evidence limits.
Do better prompts make AI fairer for students? Outputs improved, but learning and equity were not tested
A structured prompt raised expert and model ratings across 800 outputs. A separate student survey linked prompt confidence with perceived gains, but neither stage measured learning or unequal outcomes.
Read the next reportExplore the full source and evidence record
What this means for people
- A student working in Uzbek may receive a confident false fact from the same model that answers correctly in English. Institutions can test and choose tools in the language actually used, rather than treating English performance as a proxy.
Global context
Uzbek is a comparatively lower-resource Turkic language and the study comes from an Uzbek university. The result is evidence for this specific question bank and model set, not a universal ranking of languages or a forecast for every multilingual classroom.
What the evidence does not yet show
- 100 short-answer questions
- Three languages
- Eight open-weight models
- No proprietary frontier systems
- One deterministic sample per prompt
- Refusals excluded from error denominator
- Machine-assisted translations
- No student or learning outcome
What to watch next
- Replication in other low-resource languages
- Audits on local curricula
- Separate reporting of wrong answers and refusals
- Independent scoring
- Learner studies on verification and belief
- Tests of current proprietary and open models
Living evidence record
Impact record IAI-0J0BOIX
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
29 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Education published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 29 October 2026
Research question, topics and regions
Cross-language factual reliability of open-weight language models
Topics: Higher education · Uzbek language · Factual reliability · Open models · Language equity
Regions: Uzbekistan · Central Asia · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.