Spine-surgery AI looked accurate—but extra imaging features added little

Discrimination was high; calibration was not

The prospective cohort was recruited consecutively between April 2024 and February 2025. Twelve of 120 recruits lacked complete follow-up, leaving 108 for analysis. A poor outcome meant residual pain above two on the visual analogue scale or disability above 20% on the Oswestry index at 12 months. The model reached 85.3% sensitivity, 95.9% specificity and 92.6% overall accuracy; ROC-AUC was 0.976.

Those numbers describe ranking and classification, not trustworthy individual probabilities. Before recalibration, the prospective Brier score was 0.101, the calibration intercept 2.629 and slope 3.180—well away from the ideal intercept of zero and slope of one. Training-derived recalibration improved the Brier score to 0.084 but left material miscalibration. A patient should not be told a numerical risk as though it were already reliable.

The expensive-looking feature stack also added limited discrimination beyond ordinary clinical and conventional-imaging information. That matters for hospitals weighing extraction, annotation and compute against a simpler reference model. More variables do not automatically create more useful decisions.[1]

The image pipeline cannot yet be fully audited

A further qualification sits upstream. The archived records described a ViT-Base image encoder, but the implementation, pretrained-weight source, output layer, input processing and any fine-tuning were not retained. The authors therefore could not fully reconstruct or independently audit how the deep features were created. That is a serious reproducibility gap for a model intended to inform clinical risk.

For surgeons and patients, the evidence check is encouraging but bounded: one centre, 108 prospective patients and only 34 poor outcomes leave wide uncertainty, particularly around calibration and subgroups. Twelve of the 120 prospectively recruited patients were excluded because follow-up was incomplete, which could matter if their recovery differed systematically. The model was not compared in a trial where clinicians acted on predictions, and no treatment, complication or recovery benefit was measured. Chinese public grants funded the work and no commercial conflict was declared. Confidence would rise with multicentre locked-model validation, a reproducible image pipeline, comparison against a simple clinical score and a prospective impact trial showing safer decisions or better recovery.[1]