Does bilingual AI dialogue improve learning?
In a Karnataka field study, AI dialogue increased interest and self-efficacy but did not produce reliably larger knowledge gains than written responses. Bilingual voice reduced some participation barriers, while dialogue had lower completion. The work is an unreviewed preprint.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The between-subjects field study assigned 305 students to six written, text-dialogue or voice-dialogue conditions in English or bilingual Kannada-English; 173 completed the programme.
- 2Knowledge rose in every condition, but the study found no reliable difference in gains between formats or languages; dialogue did increase reported interest and self-efficacy.
- 3Dialogue was preferred by completers but completed less often: 68.6% finished written responses, 52.3% text dialogue and 48.4% voice dialogue.
Research topic
Whether text or voice dialogue with an AI tutor, in English or bilingual Kannada-English, changes engagement, completion and learning compared with written-response activities
The answer is engagement, not proven extra learning
Bilingual AI dialogue did not reliably improve measured knowledge over written responses in this study. The clearest positive result was motivational: students who completed text or voice dialogue reported greater interest in the AI-literacy topic and stronger confidence that they could understand it. Bilingual voice also appeared to make participation easier for some Kannada speakers. Those gains matter because willingness to contribute can determine whether a learner gets useful practice, but they should not be translated into a claim that conversation taught more.
The trade-off is unusually important. Dialogic activities elicited more words, more time and richer interaction among completers, yet fewer students finished them. A learning tool can therefore look engaging in session data while excluding or losing more learners before the end. For colleges deciding whether to replace short written work with an AI tutor, the practical question is not simply whether conversation feels better. It is whether the design can sustain completion and produce durable learning for the whole assigned group.[1]
Six conditions separated format, modality and language
The Cornell-led team ran an ethics-approved field study with 305 native Kannada-speaking undergraduates aged 18 to 23 at English-medium institutions in Karnataka. Students were randomly assigned for the duration of the study to one of six conditions: written response, text dialogue or voice dialogue, each delivered in English-only or bilingual Kannada-English form. Everyone watched the same four short bilingual instructional videos and answered the same activity prompts. The dialogue groups interacted with an AI tutor; the writing groups submitted a response independently.
The technology was not one model alone. Gemini 2.5 Flash generated tutoring and feedback. Voice sessions used GPT-4o Transcribe for speech recognition, Gemini for the reply and Cartesia Sonic 3 for speech output. Written and text-dialogue groups used web interfaces. The same content, prompts and feedback rubric were held constant, but feedback timing was not identical: dialogic groups received it immediately, while written-response students waited 12 hours to approximate conventional homework. That difference may itself affect experience and confidence.[1]
The denominator fell from 305 enrolled to 173 completers
The first deployment enrolled 167 students through 25 village Model Digital Inclusion Centers; only 51 completed after the centres shut permanently because of an unexpected funding cut. A second college deployment enrolled 138 and retained 122. Overall, the outcome analyses generally used the 173 completers, while discourse analysis included anyone who generated conversation data. Condition-level completion ranged from 20 of 45 in English voice to 38 of 53 in English writing, so some cells used for outcome comparisons contained only about 20 to 30 students.
Random assignment strengthens causal comparison at enrolment, but differential attrition weakens it after assignment. If the students who persisted in voice or text dialogue were more motivated, better connected or more comfortable with the interface, adjusted post-study comparisons can reflect that selection. The authors checked the college deployment separately, where all writing students completed versus 83.0% of text-dialogue and 79.4% of voice-dialogue students. The format pattern remained, suggesting the first site closure was not the whole explanation.[1]
Learning rose across groups without a reliable winner
Knowledge was measured with 15 multiple-choice and open-ended questions administered before and after the programme. Human researchers scored a subset of 236 open-ended responses, then GPT-5.4-mini applied the rubric to the full set; the paper reports quadratic-weighted kappa and intraclass correlation of 0.85 against the human ratings, followed by manual verification of all model-assisted grades. Regression models adjusted post-scores for pre-scores, activity, language, their interaction and deployment site, using robust standard errors.
Knowledge increased in all six conditions. The bilingual voice group had the highest adjusted post-score at 9.91 out of 15, but differences among the six groups were not statistically reliable. Within voice, the bilingual estimate was 1.89 points above English voice with p=0.052—suggestive, not conventionally significant—and the authors emphasise that small cells limit power. The responsible conclusion is parity on this short assessment, not proof that format or language makes no difference under all conditions.[1]
Dialogue changed interest, confidence and participation
On five-point scales, the combined dialogue formats scored 0.47 points above writing for interest and 0.29 points above it for self-efficacy, both with p<0.001. Text and voice estimates pointed in the same direction. Interaction logs and interviews suggest immediate feedback, follow-up questions and hints helped students continue reasoning without fear of being judged. Bilingual voice reduced articulation difficulties, increased turns coded as showing understanding and reduced abandonment compared with English-only voice.
Language preference was not a simple access story. Students sometimes found Kannada easier for explaining concepts, especially by voice, but many valued English for academic and professional opportunity. Kannada typing could also be cumbersome. The paper therefore proposes learner control over modality, language, persistence and pace, with a local language used as translanguaging support rather than an automatic replacement for English. That is a useful design hypothesis, not yet a tested long-term policy.[1]
Completion is part of effectiveness
Across all enrollees, 68.6% completed written response, 52.3% text dialogue and 48.4% voice dialogue, a significant difference across formats. Among those who did complete and answer the post-survey, 68% in dialogue preferred the study activity over their usual work, compared with 14% in writing. These measures describe different populations: preference among finishers cannot cancel the fact that more assigned learners dropped out of dialogue.
Schools should consequently measure starts, completions, time, learning and accessibility together. Voice can reduce a language-production barrier while adding bandwidth, privacy, noise, latency and turn-taking burdens. Text can be reviewable and editable but harder in a local script. A flexible system may outperform a single mandated channel, yet flexibility also complicates evaluation. Any roll-out should retain a non-AI route and examine who stops using each option rather than reporting only average engagement from successful sessions.[1]
What would change the assessment
Confidence would rise with a preregistered multi-institution replication large enough to estimate format-by-language effects, using an intention-to-treat analysis that keeps all assigned students in the main comparison. Required or credit-bearing settings would test whether the completion gap persists when participation has consequences. Longer follow-up should measure retention and transfer to new problems, and a design that equalises feedback timing would isolate conversation from immediacy.
Researchers should also publish performance by language proficiency, device quality, disability and access conditions; compare a trained human discussion or peer activity; audit speech recognition separately for Kannada and code-switching; and report model failures. Until that evidence arrives, the study supports a narrower claim: AI dialogue can make a low-stakes activity more engaging for completers, and bilingual voice can change how some students participate, without demonstrating superior learning or universal accessibility.[1]
What this means for people
- Students may gain a less intimidating way to ask questions and express partial understanding.
- Learners with poor connectivity, little privacy or difficulty using voice or local-script keyboards may face new barriers.
- Educators should not infer better learning from longer conversations or higher satisfaction alone.
Global context
The study concerns Kannada-speaking students in one Indian state, a valuable addition to an evidence base dominated by English and wealthy-country campuses. English-medium institutions across Asia and Africa face related tensions between access through local languages and the perceived opportunity attached to English. Effects will depend on language technology quality, classroom norms, device access and whether local-language support expands learner agency rather than narrowing future options.
What the evidence does not yet show
- The paper is an unreviewed preprint and does not establish effects outside this specific Karnataka English-medium context.
- Only 173 of 305 enrolled students completed the study, with different completion rates by format and small outcome-analysis cells.
- The first deployment was disrupted by a funding-related centre closure, although the lower dialogue completion pattern remained in the second deployment.
- The activity was optional, low-stakes and short; durable learning, transfer and performance in required courses were not tested.
- Immediate feedback accompanied dialogue while writing feedback was delayed, so format and feedback timing were not fully isolated.
What to watch next
- Preregistered, larger replications with intention-to-treat reporting.
- Longer-term retention and transfer rather than immediate post-tests alone.
- Independent audits of Kannada speech recognition and code-switching performance.
- Completion and accessibility outcomes when learners can freely switch modality and language.
Living evidence record
Impact record IAI-0B06JIH
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
7 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 7 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
Education
Can expert checks make AI study materials useful?
A two-cohort economics-course study associated tutor-verified AI materials with fewer marks below a UK degree boundary, but the design cannot isolate verification, rule out cohort differences or prove that use caused the result. It is an unreviewed preprint.
8 min · 1 source
Education
Can AI avoid overreacting to one wrong answer?
A diffusion-based knowledge-tracing model led four educational benchmarks, including difficult response reversals. The evaluation reuses folds for early stopping and scoring and does not test classroom decisions.
8 min · 2 sources
Education
Did ChatGPT improve writing scores in a 16-week Taiwan course?
Twelve university students used ChatGPT throughout a five-stage inquiry-writing course. Their closed-book writing scores rose by 2.79 points, but the change was not statistically significant and the study had no comparison group.
8 min · 1 source
The Impact Brief
Keep the evidence trail, not the noise.
Get the most consequential AI developments with direct sources and clear limits.
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.