Can a foundation model track tumours during MR-guided radiotherapy?
On 38 held-out cine-MR sequences, fine-tuned SAM2 variants reduced average boundary and centre errors and ran fast enough for the tested image stream. The work remains a retrospective technical test: it did not steer radiation or measure patient safety.
The short version
3 key points
- 1.TrackRAD2025 supplied 50 labelled training sequences with 5,027 frames and 38 held-out test sequences with 2,905 frames from six international institutions.
- 2.Across model configurations, fine-tuning raised mean Dice from 0.874 to 0.891 and reduced mean centroid distance from 5.299 mm to 1.769 mm; no significant accuracy difference emerged between manual and semi-automated training labels.
- 3.Novices labelled the first frame of 200 additional sequences and SAM2 propagated masks across 33,149 frames, but those propagated labels were not manually reviewed and no model controlled a treatment beam.
The answer: promising tracking accuracy, without evidence of clinical readiness
A family of open foundation vision models tracked tumours and surrogate anatomy across short magnetic-resonance image sequences with high overlap and low average centre error in a retrospective benchmark. In the paper's combined analysis, fine-tuning increased mean Dice overlap from 0.874 to 0.891, reduced the 95th-percentile Hausdorff boundary distance from 9.069 to 4.378 millimetres and reduced centroid distance from 5.299 to 1.769 millimetres. All three changes met the Bonferroni-adjusted significance threshold. Inference took roughly 0.04 to 0.08 seconds per frame on one Nvidia A100 GPU, which is fast enough to keep pace with the image streams examined by the researchers.
Those results support technical feasibility, not a claim that SAM2 can safely guide a patient's radiation today. The system was tested after the fact against curated contours. It did not decide whether to interrupt or redirect a beam, and investigators did not measure missed targets, irradiated healthy tissue, treatment time or patient outcomes. The authors explicitly call for prospective testing and evaluation of worst-case failures. A good mean can hide a short tracking loss at precisely the moment accuracy matters most, so clinical validation needs safety rules and case-level failure analysis as well as average segmentation scores.[1]
What data the models saw
The study used TrackRAD2025, an open collection of two-dimensional sagittal cine-MR sequences from 0.35-tesla and 1.5-tesla MR-linac systems at six international institutions. The labelled training set contained 50 sequences and 5,027 frames; the held-out test set contained 38 sequences and 2,905 frames. One institution appeared only in the test set, giving the evaluation some protection against simply memorising every site's image style. Each frame contained one contour, usually the gross tumour volume. When a lesion could not be seen reliably, a more visible surrogate such as the whole liver was used instead.
Medical and dental students, physicists or radiation oncologists created the original frame-by-frame contours, depending on the contributing institution. A medical-physics doctoral researcher with four years of tumour-tracking experience reviewed and corrected them. That creates a stronger reference than unreviewed crowd labels, but it is not an infallible biological truth. The reported inter-observer range itself shows that trained people disagree. Surrogate structures also change the task: tracking a high-contrast organ boundary is not equivalent to tracking a low-contrast tumour that the radiation team must keep inside a treatment margin.[1]
How less than 1% manual labelling was tested
The unlabelled part of TrackRAD2025 contained 2,000 sequences from 477 patients and more than 2.8 million frames. The researchers built a browser tool in which medical students without specialist MRI training drew a box around an object on the first frame. SAM2 produced the first mask and then propagated it through the sequence. Using that process, the team labelled 200 sequences and 33,149 frames while people directly marked less than 1% of frames. The final propagated masks were not manually reviewed, an important limitation when interpreting the apparent saving in expert labour.
Five starting models—four sizes of SAM2.1 and the medically adapted MedSAM2—were evaluated zero-shot and after fine-tuning on manual labels, semi-automated labels or both, producing 20 model-and-training combinations. The researchers found no statistically significant accuracy difference between models fine-tuned on the 5,027 manually labelled frames and those fine-tuned on the much larger semi-automated set. That finding suggests a practical route to cheaper pilot datasets. It does not prove that automatic labels are interchangeable with expert contours for every anatomy, scanner or disease, because the comparison concerned one benchmark and one family of related models.[1]
What the statistics establish—and what they do not
Accuracy distributions were strongly non-normal, so the authors used Kruskal–Wallis tests and generalised linear mixed models rather than relying on ordinary mean comparisons. Each fine-tuning model used 760 values: 20 model-configuration pairs evaluated on 38 test sequences. A Bonferroni-adjusted alpha of 0.016 covered the three similarity metrics. There was no significant zero-shot accuracy difference among the five model variants, and no significant difference across all 20 model-and-training combinations in the omnibus tests. The clearest signal was therefore the broad contrast between zero-shot and fine-tuned use, not a victory for one branded model size.
Mean fine-tuned performance approached the benchmark's reported inter-observer variability, but 'approached human variability' should not be translated into 'as safe as a clinician'. Dice overlap gives larger targets more tolerance than small ones, while boundary and centroid distances condense an entire motion sequence into averages. Runtime was measured on one high-end GPU configuration, and the study did not report an end-to-end latency budget including image acquisition, data transfer, verification and beam-control safeguards. Hardware, software and clinical workflow can each introduce delay or failure beyond the model's inference time.[1]
Why this matters to patients and radiotherapy teams
MR-guided radiotherapy can show a moving target while radiation is delivered. Better tracking could reduce interruptions and help a team keep a tumour inside the planned field as breathing changes its position or shape. For patients, the potential benefit is more confident targeting with less unnecessary exposure of nearby tissue. The risk is equally concrete: if a tracker silently loses a low-contrast lesion, a fast answer can be worse than a slower system that stops and asks a specialist to check. The paper noted one zero-shot sequence where a model lost a low-contrast object early; fine-tuning prevented that particular error, but the benchmark cannot enumerate every rare failure.
For medical physicists and oncology services, the immediate result is about development workflow. A small team may be able to create a research dataset by asking novices to mark starting frames and using a model to propagate masks, reserving expert time for targeted review. Any clinical programme would still need documented acceptance tests, uncertainty or failure detection, escalation rules and validation on its own machines and patient mix. Cloud-hosted labelling also needs a privacy and governance review before protected images are uploaded; this study used a fully de-identified public dataset, which is not the same as routine clinical data.[1]
What would change the assessment
The next persuasive evidence would be a locked evaluation on new institutions, including small and low-contrast targets, irregular breathing, large deformation and sequences deliberately chosen for difficult failures. Results should be reported per patient and per anatomy alongside worst-case deviations, not just pooled means. Lower-cost GPUs and complete clinical pipelines should be timed. Independent teams should also compare semi-automated labels with expert-reviewed contours to learn when propagated mistakes survive fine-tuning and whether targeted quality control can detect them efficiently.
Prospective testing must begin in shadow mode, where the tracker runs alongside treatment without controlling it, before any interventional trial. Investigators should record alarms, human overrides, target loss, beam interruptions, dose to tumour and nearby organs, and any safety incident. The work was conducted by Mayo Clinic and University of Iowa researchers and included US National Institutes of Health support for one author; the authors declared no competing interests. That transparent funding record does not substitute for independent replication. Until those steps are completed, the study supports further evaluation and cheaper data preparation, not clinical deployment.[1]
What this means for people
- Patients could benefit if reliable tracking keeps radiation on a moving tumour, but this study did not measure delivered dose, harm or outcomes.
- Radiotherapy teams may reduce frame-by-frame labelling work, while retaining expert review and local safety validation for consequential use.
- Hospitals need to test whole-system latency and failure handling on their own scanners rather than infer readiness from GPU inference speed alone.
Global context
The benchmark drew sequences from six institutions and two MR-linac field strengths, with one institution held out from training. That is broader than a single-centre study, but the published experiment does not describe performance for every cancer site, health system, scanner configuration or population represented in MR-guided radiotherapy worldwide.
What the evidence does not yet show
- Retrospective segmentation and tracking accuracy do not establish safe radiation delivery or patient benefit.
- The 38-sequence test set is small for characterising rare but serious tracking failures.
- The semi-automatically propagated labels were not manually reviewed, and the training comparison covered one benchmark and model family.
- Runtime was measured on a single A100 GPU and excludes much of the clinical pipeline.
- Some reference contours represented a visible surrogate structure rather than the tumour itself.
What to watch next
- External evaluation on difficult low-contrast targets and new MR-linac sites
- Shadow-mode prospective studies with alarms, overrides and worst-case errors
- Dose and patient-safety outcomes before any beam-control use
- Expert quality-control studies for semi-automated training labels
Living evidence record
Impact record IAI-0YHHUUY
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
13 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Journal of Imaging Informatics in Medicine published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 13 October 2026
Research question, topics and regions
Whether SAM2 foundation-model variants can track targets in cine-MR images and whether mostly automated labels can support fine-tuning as well as frame-by-frame manual labels
Topics: Radiotherapy · Medical imaging · Foundation models · Cancer care · Tumour tracking
Regions: United States · International
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Health & Life Sciences
How far has AI reached in low-resource radiotherapy?
A peer-reviewed systematic review found 18 eligible AI deployment studies in low- and middle-income countries. Only one joined multiple stages of the radiotherapy workflow, none showed fully operational use, and no study reported continuous performance monitoring.
9 min · 1 source
Health & Life Sciences
Can mouse MRI predict glioblastoma treatment response?
A deep-learning model separated cure from relapse across 298 MRI examinations, but the effective test cohort was only 10 treated mice from one laboratory. It is preclinical evidence, not a patient-ready predictor.
7 min · 1 source
Health & Life Sciences
Can brain MRI reveal more than BMI?
A peer-reviewed study trained a deep-learning model on 45,702 MRI scans from six cohorts. Brain-derived features tracked BMI and separated several disease groups better than BMI alone—but the disease analysis stayed inside UK Biobank and cannot establish cause, diagnosis or clinical benefit.
9 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.