A retinal AI changed with the camera—what should screening programmes test?
In 8,524 Brazilian patients, performance stayed similar for some retinal tasks but dropped sharply for others when the camera platform changed. The unreviewed study cannot separate hardware effects from image-quality differences or show patient benefit.
The short version
- What happened
- An unreviewed University of British Columbia study re-analysed 16,266 retinal photographs from 8,524 patients at three Sao Paulo clinics, comparing Canon CR-2 and Nikon NF5050 images.
- Who it affects
- A screening programme using a cheaper or locally available camera could widen access, but a model that misses camera-specific degradation may give patients inconsistent results. The study supports local validation before clinical reliance, not rejection of lower-cost equipment.
- What remains unproven
- The new analysis is an unreviewed preprint by one author; the underlying BRSET dataset paper was peer reviewed, but that does not peer-review this re-analysis.
Keep in mind
The new analysis is an unreviewed preprint by one author; the underlying BRSET dataset paper was peer reviewed, but that does not peer-review this re-analysis.
Only two camera platforms and one Brazilian dataset were studied, with camera, image quality, acquisition and patient characteristics confounded.
The model was used as a frozen feature extractor; results may not transfer to fine-tuned systems, other foundation models or deployed screening software.
AUROC and quadratic-weighted kappa are benchmark metrics, not evidence of better referrals, treatment, vision or patient safety.
The preprint does not state funding or competing-interest disclosures in its available HTML text.
Source analysis · 2 linked sources. Primary analysed source date: 21 October 2026. Earliest linked supporting source: 11 July 2024.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
One model, two cameras and uneven changes across diseases
The study used BRSET, a peer-reviewed Brazilian dataset containing 16,266 retinal photographs from 8,524 patients seen at three outpatient centres in Sao Paulo between 2010 and 2020. Researchers froze a pretrained retinal foundation model, trained downstream classifiers on Canon CR-2 images and compared performance on a patient-disjoint Canon test set with a Nikon NF5050 cohort the classifiers had not seen during training. They repeated the analysis across 12 tasks, four image resolutions and five classifiers—240 comparisons in all.[1][2]
Some headline tasks travelled well; others did not
For the validation-selected configurations, area under the receiver operating characteristic curve was similar for diabetic-retinopathy detection (0.940 on Canon and 0.946 on Nikon), scar (0.964 and 0.955) and macular oedema (0.952 and 0.952). Larger gaps appeared for haemorrhage (0.804 and 0.569) and hypertensive retinopathy (0.864 and 0.690). Five-stage diabetic-retinopathy grading fell from a quadratic-weighted kappa of 0.692 to 0.527. Higher resolution and a different classifier did not consistently remove the gaps.[1]
The distinction that matters
What changed when the camera changed?
The same frozen representation did not shift uniformly across retinal tasks.
Little change
Test diabetic retinopathy, scar and macular oedema
Selected-task performance remained similar
Larger drop
Test haemorrhage, hypertension and DR staging
Performance fell on the unseen Nikon cohort
Unresolved cause
Compare camera and image-quality labels
Hardware, acquisition and quality remain confounded
The camera comparison is also an image-quality comparison
Nikon images were more than twice as likely to be rated inadequate overall: 18.6% compared with 8.8% for Canon. Most of the difference came from image-field adequacy, at 15.84% versus 4.76%. Because camera platform, acquisition conditions, image quality and patient mix are intertwined in this retrospective dataset, the analysis cannot show that hardware alone caused the performance changes. The study also uses only two cameras and one dataset, reports no prospective workflow test and measures no missed referrals, treatment decisions, visual outcomes or clinician workload.[1][2]
What screening services can take from the study now
A service should not accept one aggregate accuracy figure as proof that a retinal model will transfer to every device. Before deployment, performance should be reported by camera and disease task using patient-level splits, with image-quality failures and uncertain outputs counted. The assessment would strengthen with prospective multi-site testing across more cameras, matched acquisition protocols and evidence that quality control or domain adaptation prevents clinically important errors. Until then, the work is a useful warning about hidden domain shift, not evidence that one camera or model should be chosen for patient care.[1][2]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring medical imaging: compare the finding and its evidence limits.
Why did a pneumonia AI fall from 98% accuracy to 60% on outside data?
A peer-reviewed chest X-ray model scored 98.12% on its internal test set, then 60.42% and 74.30% on two external datasets. The gap is a warning about domain shift—not evidence of a deployable diagnostic tool.
Read the next reportExplore the full source and evidence record
What this means for people
- A screening programme using a cheaper or locally available camera could widen access, but a model that misses camera-specific degradation may give patients inconsistent results. The study supports local validation before clinical reliance, not rejection of lower-cost equipment.
Global context
Retinal screening is especially valuable where specialist access is limited, and lower-cost cameras may expand reach. Equity therefore depends on testing the exact equipment and workflow a community will use, rather than assuming that a model validated on one imaging system will transfer unchanged.
What the evidence does not yet show
- The new analysis is an unreviewed preprint by one author; the underlying BRSET dataset paper was peer reviewed, but that does not peer-review this re-analysis.
- Only two camera platforms and one Brazilian dataset were studied, with camera, image quality, acquisition and patient characteristics confounded.
- The model was used as a frozen feature extractor; results may not transfer to fine-tuned systems, other foundation models or deployed screening software.
- AUROC and quadratic-weighted kappa are benchmark metrics, not evidence of better referrals, treatment, vision or patient safety.
- The preprint does not state funding or competing-interest disclosures in its available HTML text.
What to watch next
- Peer review and independent replication
- Prospective multi-site testing across more camera platforms
- Matched analyses that separate hardware from image quality and patient mix
- Sensitivity and referral outcomes at clinically chosen thresholds
- Evidence that quality control or domain adaptation prevents task-specific degradation
Living evidence record
Impact record IAI-0HK8MPF
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
22 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 22 October 2026
Research question, topics and regions
Retinal AI robustness across camera platforms
Topics: Retinal screening · Medical imaging · Domain shift · Health equity · Clinical validation
Regions: Brazil · Latin America · Canada
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.