Did a global AI course prove clinicians learned more?
Not yet. In a peer-reviewed evaluation, 63 completers from 12 countries reported large gains in confidence after an eight-week course, but there was no objective knowledge test, control group, retention measure or patient outcome.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The analysis included 63 people from 12 countries who attended at least 70% of the course, attended the final session and completed the final survey; 168 attended at least one live session.
- 2Retrospectively rated confidence in explaining AI concepts rose from 2.68 to 4.22 on a five-point scale, while confidence in evaluating ethical implications rose from 2.68 to 4.27.
- 3The design cannot show objective learning, retention, changed clinical practice or better patient care because it lacked a contemporaneous pre-test, control group, knowledge assessment and outcome follow-up.
Research topic
A retrospective self-report evaluation of a free virtual AI and global-health course for a geographically diverse convenience sample
The answer: the course changed reported confidence, not demonstrated competence
A peer-reviewed evaluation of the free AI Catalyst for Global Health course found large increases in how confident completers said they were about explaining AI concepts and evaluating ethical implications. That is useful evidence that a short, accessible programme can feel relevant to a geographically dispersed health audience. It is not proof that participants acquired durable knowledge, changed clinical decisions or improved care.
The distinction matters for hospitals, universities and funders choosing AI training. Confidence can support participation and willingness to ask questions, but it can also rise without accurate knowledge. A responsible programme evaluation therefore needs to measure both how learners feel and what they can do. This study supplies the first part and clearly acknowledges that the second remains open.[1]
The Impact Brief · Free
Follow the evidence in education.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
Who took part and who reached the final analysis
The course ran virtually from July to September 2025 as an eight-week, 12-hour webinar series followed by a concluding session. It used lectures, case discussions and question-and-answer components covering AI foundations, data ethics, oncology, global health systems and community-centred innovation. Recruitment used the Global Oncology University network, professional associations, social media and targeted outreach to low- and middle-income regions.
A total of 168 people attended at least one live session. To enter the final analysis, a participant had to attend or verify asynchronous viewing of at least 70% of sessions, attend the final session and complete the post-course survey. Sixty-four met those criteria and responded; one incomplete response was excluded, leaving 63. That completion pathway selects a committed subset and makes the denominator important: the findings describe eligible completers, not everyone initially exposed to the course.[1]
The cohort was international but small and self-selected
The 63 respondents lived in 12 countries: Kenya, Nigeria, the United States, Kosovo, Tunisia, Spain, Algeria, Cameroon, Australia, Senegal, South Africa and India. Kenya contributed the largest group, followed by Nigeria and the United States. Twenty-six respondents, or 41%, were healthcare providers; 15 were students, 14 researchers or academics, three worked in technology or engineering, one in policy or government and four in other roles.
This spread gives the evaluation value beyond a single institution, especially because most participants were in low- and middle-income settings. It does not make the sample representative of clinicians, oncology workers or countries in those income groups. Participation was voluntary and recruitment was connected to an education network. People with better connectivity, more interest in AI or more time for a multi-week course were more likely to enrol and complete it.[1]
What the reported numbers show
After the course, respondents retrospectively rated how confident they had been before it and how confident they were after it. Mean confidence in explaining core AI concepts rose from 2.68 to 4.22 on a five-point scale, a 1.54-point difference with a 95% confidence interval from 1.23 to 1.85. The reported paired effect size was 1.24. Confidence in evaluating ethical implications rose from 2.68 to 4.27, a 1.59-point difference with a 95% confidence interval from 1.26 to 1.92 and an effect size of 1.21.
Participants also rated the feasibility of using AI approaches at 3.86 out of five and their likelihood of applying course insights within 12 months at 4.29. Funding was the most frequently selected implementation barrier, cited by 27 of 63 respondents. Thirteen cited limited technical capacity, ten insufficient stakeholder awareness, six ethical or regulatory concerns and six restricted access to data. These are perceptions in a convenience sample, not measured implementation rates or a national needs assessment.[1]
Why the retrospective design needs careful interpretation
The study did not administer the same objective knowledge test before and after the course. Instead, participants at the end were asked to reconstruct their earlier confidence. This retrospective pre-post approach can reduce one problem in self-assessment: before training, learners may not know enough to judge what they do not know. But it introduces recall and response-shift bias, and the positive course experience itself may influence both ratings.
There was no untrained comparison group, so time, selection and enthusiasm cannot be separated from course effects. There was no blinded assessment, no observed task, no test of error recognition, no follow-up showing whether knowledge persisted and no link to clinical workflow or patient outcomes. Statistical significance and large self-report effect sizes do not remove those design limits. The paper appropriately describes the work as an exploratory educational programme evaluation and reports that no formal advance sample-size calculation was performed.[1]
What training commissioners can use now
The study provides a workable delivery model: free online access, live and asynchronous participation, practical cases, ethics content and attendance verification across time zones. It also identifies the audience's stated constraints. A commissioner could use those features to design a more rigorous next evaluation, adding a preregistered objective test, scenario-based safety questions, a comparison group, completion and dropout analysis, and follow-up at three or six months.
For clinicians and public-service professionals, training should be tied to the decisions they actually face: interpreting an AI output, recognising uncertainty, protecting confidential information, escalating an error and deciding when not to use a system. For educators, satisfaction and confidence are early signals, not endpoints. An effective course should show calibrated judgment—confidence that rises when answers become more accurate and falls when uncertainty is warranted.[1]
What would change the assessment
Evidence would strengthen if a larger, locally diverse cohort completed validated knowledge and decision tasks before and after training, with results compared against an appropriate control and followed over time. It would be especially useful to know whether people recognised unsafe AI advice, whether they changed real procurement or clinical processes and whether improvements persisted without widening gaps for participants with weaker connectivity or language support.
The authors declare no competing interests. Publication costs were supported by ecancer, and selected participating students received support through an NIH R25 award. De-identified data, survey instruments and analytic code are available from the corresponding author on reasonable request rather than deposited openly. The present conclusion should therefore remain narrow: this international course reached a small committed group and was associated with much higher self-reported confidence. Objective learning and effects on work or care remain unproven.[1]
What this means for people
- Clinicians and trainees can treat confidence as a prompt to practise and verify skills, not as proof that they can safely evaluate or deploy a system.
- Educators can reuse the accessible virtual format while adding objective assessments, retention checks and local-language or connectivity support.
- Health organisations should evaluate whether training changes real decisions, error recognition and governance before claiming workforce readiness.
Global context
Most respondents were in low- and middle-income settings, with the largest groups in Kenya and Nigeria, but the 63-person convenience sample cannot represent national workforces. Time zones, connectivity, language, local data governance and access to implementation funding will shape whether a virtual model transfers.
What the evidence does not yet show
- The outcome was retrospective self-reported confidence rather than an objective test of knowledge or performance.
- There was no control group, formal advance sample-size calculation or correction for selection into course completion.
- Only 63 of the 168 people who attended at least one live session entered the final analysis, and the study does not estimate effects for non-completers.
- The evaluation did not measure retention, behaviour, clinical decisions, patient outcomes or national representativeness.
What to watch next
- Prospective pre-post knowledge and safety assessments
- Completion and dropout analysis across connectivity, language and professional groups
- Controlled comparisons and longer-term retention
- Evidence of changed practice, safer decisions or patient benefit
Living evidence record
Impact record IAI-0MJ1VZX
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
13 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what ecancer medicalscience published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 13 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
Education
Can AI write medical exam questions that are ready to use?
Almost never in this epilepsy benchmark. Seven language models produced 420 questions, but two blinded clinicians judged only two—0.48%—directly usable without revision. Answer accuracy ranged from 61.67% to 98.33%, and weak distractors were the recurring problem.
7 min · 1 source
Education
How are Iranian medical students using AI?
In a stratified survey of 384 students at one medical university, research, writing and self-directed learning attracted the highest AI-use preference scores. The study measured stated preferences, not competence, accuracy or patient safety.
7 min · 1 source
Education
How are Kuwaiti health students using AI to write in English?
In a peer-reviewed survey of 241 nursing and health-sciences undergraduates, 94.6% reported previous generative-AI use and 83.4% used it for academic work at least sometimes. Students also reported accuracy and dependency concerns, while the single-institution self-report design cannot show that AI improved writing or learning.
7 min · 2 sources
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.