Can AI count kidney-transplant inflammation consistently? Lymphocytes were easier than monocytes
A peer-reviewed international challenge tested algorithms on 118 transplant biopsies and more than 131,000 labelled cells. The best reported F1 score reached 0.78 for lymphocytes but only 0.47 for monocytes, and no study yet shows that the system improves diagnosis, treatment or graft outcomes.
The short version
- What happened
- The peer-reviewed paper reports a multicentre dataset of 118 biopsies with more than 131,000 immunohistochemistry-guided cell annotations and held-out validation and test evaluation.
- Who it affects
- Patients could benefit if reviewed cell maps make biopsy assessment more consistent, but the system has not been shown to improve rejection diagnosis, treatment or graft survival.
- What remains unproven
- The accessible peer-reviewed abstract reports 118 biopsies but does not expose the final training, validation and test denominators; the preregistered counts should not be treated as the final analysis.
Keep in mind
The accessible peer-reviewed abstract reports 118 biopsies but does not expose the final training, validation and test denominators; the preregistered counts should not be treated as the final analysis.
Monocyte detection remained weak, with a best reported F1 score of 0.47, compared with up to 0.78 for lymphocytes.
The external 180-biopsy analysis reports strong correlations for some lesions without coefficients, confidence intervals or grade-boundary error tables in the accessible abstract.
Correlation with consensus lesion scores does not establish agreement, diagnostic accuracy, faster reporting, better treatment or improved graft outcomes.
The organisers created the challenge and reference data; independent locked-model reproduction and prospective reader studies are still needed.
Multi-source analysis · 3 linked sources. Primary analysed source date: 6 November 2026.
Read the full report · 9 min The evidence, methods, implications and limitations
The answer: a useful benchmark, not an automated transplant diagnosis
The MONKEY Challenge provides unusually grounded evidence for one narrow part of kidney-transplant pathology: finding inflammatory cells on the routine Periodic Acid-Schiff, or PAS, slides that pathologists already inspect. The journal paper reports 118 biopsies from several centres, co-registered with immunohistochemically restained slides and more than 131,000 annotated cells. Algorithms were evaluated on held-out validation and test data rather than only on the images used to train them. That makes the benchmark more informative than a single-centre random split.
Performance was not uniform. The strongest reported F1 score was up to 0.75 for inflammatory-cell detection overall and up to 0.78 for lymphocytes. For monocytes, it reached only 0.47. F1 balances precision and recall: a system loses score when it misses target cells or labels the wrong locations. It is not the percentage of biopsies diagnosed correctly, and it does not say whether a transplant is being rejected. The paper therefore supports a claim about cell detection, not autonomous Banff classification or treatment selection.[1]
Why counting inflammation matters to transplant care
Pathologists use the Banff framework to describe lesions in kidney-transplant biopsies. Several lesion scores depend on how much inflammation appears in particular tissue compartments. Those scores help clinical teams distinguish possible rejection from infection, chronic damage and other explanations, alongside laboratory results and the wider clinical picture. Manual assessment is skilled work, but semi-quantitative grades can vary between readers and take time, especially when inflammatory cells are sparse or distributed unevenly across a whole slide.
An accurate counting aid could give a pathologist a reproducible map of candidate cells, make it easier to revisit borderline areas and document how a measurement was produced. It could also fail in clinically important ways. Missing monocytes may distort subtype-sensitive measures; false detections could overstate inflammation; and a system trained on selected scans may behave differently after changes in staining, tissue preparation or scanner. The paper's uneven results are therefore more useful than a single headline accuracy: they identify which part of the proposed workflow remains least dependable.[1][3]
The distinction that matters
What the benchmark measured—and what clinical evidence is still missing
The study moves from labelled cells to pathologist-score correlations, but it stops before testing assisted diagnosis or patient outcomes.
Reference
Co-register PAS and immunohistochemistry
More than 131,000 annotated cells across 118 biopsies
Detect
Evaluate algorithms on held-out images
F1 up to 0.78 for lymphocytes and 0.47 for monocytes
Contextualise
Combine cells with tissue segmentation
Measurements correlated with some consensus Banff scores in 180 biopsies
Next
Test pathologists with and without a locked aid
Measure agreement, time, treatment effects and harm
How the ground truth and centre separation were designed
The organisers did not ask annotators to identify cells from PAS appearance alone. Their structured report says each biopsy was also restained with markers intended to distinguish lymphocytes and monocytes, with the images co-registered so the immunohistochemical signal could guide point annotations on the PAS slide. An existing model accelerated initial marking, student annotators reviewed the material, and an advanced renal-pathology resident checked the cases. This creates a more defensible reference than unverified clicks, while still depending on registration quality, staining and human review.
The preregistered design collected cases from six pathology departments in the Netherlands, Austria, Switzerland and the United States. It planned centre-separated training, live validation and final testing, including a test centre using a different scanner. That is a meaningful defence against leakage from near-identical images. However, the preregistration described 121 planned cases, while the final abstract reports 118 biopsies. The accessible journal record does not expose the final counts in each partition, so this report does not silently substitute the planned 83 training, 10 validation and 28 test cases for the final analysis.[1][2]
The external 180-biopsy analysis is encouraging but incomplete
The authors combined the winning detector with models that segment renal tissue compartments, then calculated measurements related to Banff lesions. In an independent multicentre cohort of 180 transplant biopsies, those measurements showed what the abstract calls strong correlations with consensus pathologist scores for several inflammation-related lesions, particularly total inflammation and interstitial inflammation. This is an important step beyond counting isolated cells because the location of inflammation—not merely its total amount—matters to pathology assessment.
Correlation is not agreement. A measurement can rise with a pathologist's score while being systematically too high, too low or unreliable around the thresholds that change a grade. The accessible abstract does not give the coefficients, confidence intervals, lesion-by-lesion denominators or threshold error tables, so readers cannot judge the size and precision of those relationships. Nor does it say whether the 180-biopsy cohort was used only once after model locking, how many centres and scanners it represented, or how performance varied by diagnosis and tissue quality.[1]
What the study did not test
This was a technical benchmark and retrospective association study, not a clinical impact trial. Pathologists were not randomised to work with or without the system. The study did not measure diagnostic agreement, reporting time, treatment changes, unnecessary immunosuppression, missed rejection, graft function, graft survival or patient experience. It also did not compare an AI-assisted pathologist with the consensus panel that supplied the lesion scores. The challenge organisers had planned a reader study, but those results are not part of the evidence reported in the accessible journal abstract.
The preregistration says the challenge was funded by the Dutch Kidney Foundation under grant 21OK+012, with prize money from a legacy connected to a former IBEX employee. It also states that members of the Radboud computational-pathology team could access test labels, while core organisers were excluded from participation. Those disclosures do not invalidate the benchmark, but independent reproduction matters because the organisers designed the task, assembled the reference data and authored the resulting paper. The accessible abstract does not provide the paper's complete competing-interest statement.[1][2]
What this means for patients, pathologists and transplant teams
For a person living with a transplanted kidney, the promising use is a second set of eyes that makes inflammation assessment more traceable—not a machine verdict about rejection. A pathologist could review highlighted cells and compartment-level counts, correct errors and explain uncertainty in the report. Clinicians would still need to interpret pathology alongside creatinine trends, donor-specific antibodies, infection testing, medication history and the risks of changing immunosuppression. A cell detector cannot resolve that full decision.
For laboratories, implementation would require locked software, versioned scanner and staining checks, local validation, audit trails and a clear fallback when image quality is poor. Error review should be separated by cell type because the 0.47 monocyte F1 score signals a different risk from the stronger lymphocyte result. Teams should also measure whether the tool shifts attention toward highlighted regions and away from unmarked but important findings. Automation bias can arise even when the system is described as assistive.[1][3]
What evidence would change the assessment
Confidence would rise with the full final partition counts, site-by-site and scanner-by-scanner performance, confidence intervals, false-positive counts per tissue area, and a public locked model evaluated by independent centres. The 180-biopsy analysis should report exact correlations and agreement statistics, confusion matrices at Banff grade boundaries, missing or excluded cases and performance across rejection categories, infection, chronic injury and low-quality tissue. Monocyte detection needs either a substantial improvement or a workflow that makes its uncertainty unmistakable to users.
The decisive evidence would be a prospective reader study in which renal pathologists assess the same cases with and without the locked aid in counterbalanced order. It should measure accuracy against an independent adjudicated reference, inter-reader agreement, time, corrections, automation bias and downstream treatment recommendations. A later impact study should examine patient outcomes and harms. Until then, MONKEY is a valuable peer-reviewed benchmark and shared research resource, but it is not Breaking news, not a clinical device endorsement and not evidence that AI improves kidney-transplant outcomes.[1][2][3]
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Continue the story
Read next
Continue exploring medical imaging: compare the finding and its evidence limits.
Can AI find heart fat on routine ultrasound? Five cohorts linked the score to metabolic risk—not patient outcomes
A peer-reviewed study trained PanAdipo on 1.1 million echocardiography videos and compared its score with CT-measured fat, biomarkers and later diagnoses. The model found a reproducible signal, but retrospective associations cannot show that screening or treatment would help patients.
Read the next reportExplore the full source and evidence record
What this means for people
- Patients could benefit if reviewed cell maps make biopsy assessment more consistent, but the system has not been shown to improve rejection diagnosis, treatment or graft survival.
- Pathologists would need editable overlays, cell-type-specific uncertainty and a record of corrections rather than an unexplained automated Banff grade.
- Transplant teams must keep pathology in context with laboratory, immunological and clinical evidence; an algorithmic count is one input, not a treatment instruction.
Global context
The challenge deliberately drew on pathology departments in four countries and several scanner types, making it more internationally grounded than many single-hospital imaging studies. Even so, transplant populations, biopsy practices, staining, scanners and Banff interpretation vary. Before adoption, laboratories need local technical validation and independent evidence that assistance improves reproducibility without hiding rare patterns or shifting treatment in harmful ways.
What the evidence does not yet show
- The accessible peer-reviewed abstract reports 118 biopsies but does not expose the final training, validation and test denominators; the preregistered counts should not be treated as the final analysis.
- Monocyte detection remained weak, with a best reported F1 score of 0.47, compared with up to 0.78 for lymphocytes.
- The external 180-biopsy analysis reports strong correlations for some lesions without coefficients, confidence intervals or grade-boundary error tables in the accessible abstract.
- Correlation with consensus lesion scores does not establish agreement, diagnostic accuracy, faster reporting, better treatment or improved graft outcomes.
- The organisers created the challenge and reference data; independent locked-model reproduction and prospective reader studies are still needed.
What to watch next
- The full paper's final partition counts, exact correlation and agreement statistics, and complete funding and competing-interest disclosures
- Independent validation of the locked winning model across additional countries, scanners and staining workflows
- Prospective reader-study results comparing renal pathologists with and without the aid
- Substantial improvement or clearly bounded use of monocyte detection
- Clinical impact studies measuring treatment decisions, missed rejection, unnecessary immunosuppression and graft outcomes
Living evidence record
Impact record IAI-1OZR404
Evidence stage
Studied
Confidence
Corroborated
Reporting basis
Multi-source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
8 November 2026
Source trail
3 direct sources across 3 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Evidence trail
Sources used for this report
Links checked 8 November 2026
Research question, topics and regions
Whether deep-learning systems can detect lymphocytes and monocytes on routine PAS-stained kidney-transplant biopsy slides and produce measurements that align with pathologists' Banff lesion scores
Topics: Kidney transplantation · Digital pathology · Inflammation · Clinical validation · Medical imaging
Regions: Europe · North America · Global
This report is labelled multi-source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.