A cancer-image model scored 98.5%—then fell to 82.0% on a second dataset
A peer-reviewed lung-and-colon pathology study found a hierarchical model beat a flat classifier on two public datasets. The lower second-dataset result is the more useful warning: this was still an image-level experiment, not a clinical diagnostic test.
The short version
3 key points
- What changed
- The model scored 98.50% accuracy on the 25,000-image LC25000 benchmark, only 0.24 percentage points above the strongest flat classifier tested on the same split.
- Why it matters
- A pathology aid could help prioritise slides, but a false negative could delay investigation and a false positive could intensify anxiety or testing. This study did not evaluate either outcome in care.
- What to watch
- Locked multicentre patient-level and whole-slide validation, expert-annotated explanation checks, comparisons with pathologists, prespecified thresholds and prospective effects on diagnostic accuracy and turnaround time.
The second dataset mattered more than the near-perfect benchmark
Researchers built a coarse-to-fine system that first separates normal from cancerous tissue and then routes images to organ- and subtype-specific classifiers. On LC25000—5,000 images in each of five lung and colon classes—it reached 98.50% accuracy, compared with 98.26% for the best flat ResNet101 baseline. That is a 0.24-point gain on an unusually balanced public benchmark, not a clinical breakthrough.[1]
Why this cannot yet guide a patient's diagnosis
The paper calls LungHist700 an independent dataset, but it did not take one model trained on LC25000 and deploy it unchanged elsewhere. It applied the framework independently, using a shuffled image-level split for training, validation and testing. Because multiple images came from 45 patients and patient-level separation was not reported, the test may not measure transfer to wholly unseen patients. That is a material limit for pathology evidence.[1]
Who it affects
A pathology aid could help prioritise slides, but a false negative could delay investigation and a false positive could intensify anxiety or testing. This study did not evaluate either outcome in care.[1]
Keep in mind
Public image datasets; shuffled image-level splits; patient-separated testing not reported; no unchanged cross-dataset deployment, pathologist comparison, prospective workflow or patient outcome.
Source analysis · 1 linked source. Original source date: 17 October 2026.
Continue the story
Read next
Health & Life Sciences
Can AI predict five-year colorectal cancer survival from pathology slides?
A retrospective preprint reports moderate discrimination across a US trial cohort and an external NHS dataset. The model is not ready to guide treatment, and its selected ensemble did not clearly beat the strongest comparator.
2 min · 1 source
Health & Life Sciences
Does 99% MRI accuracy mean an Alzheimer’s diagnostic is ready?
No. A peer-reviewed model classified images from one 6,400-image Kaggle collection with reported 99% test accuracy, but it has no external clinical validation and the published test counts do not reconcile cleanly.
3 min · 2 sources
Read more The full report, methods and context
The second dataset mattered more than the near-perfect benchmark
Researchers built a coarse-to-fine system that first separates normal from cancerous tissue and then routes images to organ- and subtype-specific classifiers. On LC25000—5,000 images in each of five lung and colon classes—it reached 98.50% accuracy, compared with 98.26% for the best flat ResNet101 baseline. That is a 0.24-point gain on an unusually balanced public benchmark, not a clinical breakthrough.
The same procedure was evaluated separately on LungHist700, which contains 691 lung histology images from 45 patients at two magnifications. On its 139-image test portion, the hierarchy was correct on 114 images and the flat model on 100. Accuracy was 82.01% versus 71.94%; the paired bootstrap interval for the 10.07-point difference ran from 2.88 to 17.99 points. The fall from 98.50% also shows how strongly results depend on the dataset.[1]
Why this cannot yet guide a patient's diagnosis
The paper calls LungHist700 an independent dataset, but it did not take one model trained on LC25000 and deploy it unchanged elsewhere. It applied the framework independently, using a shuffled image-level split for training, validation and testing. Because multiple images came from 45 patients and patient-level separation was not reported, the test may not measure transfer to wholly unseen patients. That is a material limit for pathology evidence.
Four explanation tools highlighted pixels associated with predictions, but the authors did not compare those maps with expert-annotated disease regions. There was no blinded pathologist benchmark, whole-slide workflow, pathology-confirmed prospective cohort or measurement of changed treatment decisions. Severe synthetic image noise also cut accuracy as low as 48.20%, showing that visual robustness is not assured.[1]
The distinction that matters
Why the two accuracy figures answer different questions
The headline benchmark was large and balanced; the second test was smaller, imbalanced and closer to real dataset variation.
Benchmark
25,000 LC25000 images across five equal classes
98.50% accuracy; 0.24 points above the flat baseline
Second dataset
691 LungHist700 images; 139 in the test split
82.01% accuracy; 114 of 139 correct
Clinical gap
No reported patient-separated or prospective test
Performance on new patients and hospitals remains unknown
The useful finding is a research direction, not a diagnostic claim
Breaking a difficult classification task into staged decisions can outperform one flat five-class model, and the second dataset supplies more credible evidence than LC25000 alone. For pathology teams, however, the relevant question is whether the method holds across laboratories, staining protocols, scanners and unseen patients—and whether it adds value beside a pathologist without delaying care.
The assessment would strengthen with locked models tested at patient and whole-slide level across multiple hospitals, prespecified thresholds, confidence intervals for every clinically important class, expert review of explanations and a prospective comparison with routine practice. Until then, the model is an engineering result on public image collections, not an automated cancer diagnosis.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Evidence check
Public image datasets; shuffled image-level splits; patient-separated testing not reported; no unchanged cross-dataset deployment, pathologist comparison, prospective workflow or patient outcome.
Reporting basis: Source analysis. Original source date: 17 October 2026.
Explore the full source and evidence record
What this means for people
- A pathology aid could help prioritise slides, but a false negative could delay investigation and a false positive could intensify anxiety or testing. This study did not evaluate either outcome in care.
Global context
The research team is based in Bangladesh and Australia, while the public image collections do not represent the full variation of staining, scanners, laboratories or patient populations worldwide.
What the evidence does not yet show
- Public image datasets; shuffled image-level splits; patient-separated testing not reported; no unchanged cross-dataset deployment, pathologist comparison, prospective workflow or patient outcome.
What to watch next
- Locked multicentre patient-level and whole-slide validation, expert-annotated explanation checks, comparisons with pathologists, prespecified thresholds and prospective effects on diagnostic accuracy and turnaround time.
Living evidence record
Impact record IAI-0RGX19I
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
18 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 18 October 2026
Research question, topics and regions
Whether a coarse-to-fine hierarchy improves public-dataset lung and colon histology classification over a flat model
Topics: Cancer · Digital pathology · External datasets · Explainable AI · Clinical validation
Regions: Bangladesh · Australia · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.