Can an ICU forecast model see kidney injury earlier?
A model trained on 13,277 ICU admissions forecast eight physiological targets and improved retrospective discrimination for 45 moderate-to-severe kidney-injury events in an external cohort. That small event count and untested workflow effects keep it far from clinical deployment.
The short version
3 key points
- 1.OmniTFT trained on 13,277 MIMIC-IV admissions and was evaluated without retraining on 4,188 MIMIC-III CareVue and 23,238 eICU admissions.
- 2.For the primary kidney-injury analysis, only 45 of 4,685 eligible stays developed stage 2 or worse AKI in the next 24 hours.
- 3.Adding forecast summaries raised AUROC from 0.693 to 0.763 and captured 73.3% rather than 57.8% of events when 30% of stays were flagged, but no clinician acted on the forecasts.
The answer: a stronger retrospective signal, not a proven warning system
OmniTFT improved a retrospective model's ability to separate ICU stays that would and would not develop moderate-to-severe acute kidney injury over the next day. In an external eICU analysis, adding summaries of forecast creatinine values to the observed record increased AUROC from 0.693 to 0.763. At a review capacity of 30% of eligible stays, the forecast-augmented pipeline identified 73.3% of later events compared with 57.8% using the observed record alone.
The result is promising but fragile. Only 45 of 4,685 eligible stays developed the primary stage 2-or-worse outcome. That makes calibration, precision and subgroup estimates uncertain even when an AUROC difference reaches statistical significance. The study was retrospective; it did not send an alert, change treatment, reduce kidney injury or measure false-alarm burden for clinicians.
The peer-reviewed paper was published in Frontiers in Digital Health on 16 October by researchers at the University of Tokyo and Kazusa DNA Research Institute. It used public de-identified US critical-care databases. JST SPRING and a JSPS fellowship funded the work. The authors declared no commercial conflict; one author was a Frontiers editorial-board member, which the journal says did not affect review.[1]
The three cohorts and the prediction task
Only MIMIC-IV was used for development. Its 13,277 admissions were split by patient 70:20:10 for training, validation and internal testing. MIMIC-III CareVue contributed 4,188 older, non-overlapping admissions for temporal external validation. The multicentre eICU database contributed 23,238 admissions from a different hospital population. Neither external cohort was used for fine-tuning, hyperparameter selection or model choice.
All cohorts were restricted to adults meeting at least one definition for sepsis, acute respiratory distress syndrome or acute kidney injury. That disease enrichment is important: this is not evidence about an unselected ICU population. The model forecast blood pressure, heart rate, oxygen saturation, respiratory rate, temperature, creatinine, lactate and an oxygenation ratio derived from SpO2 and inspired oxygen.
Vital signs used 12 hours of history to forecast the next two hours on ten-minute grids. Laboratory values used 24 hours to forecast the next 24 hours on two-hour grids. Inputs were forward-filled only from observations available at or before each grid time. Missing values and scaling used parameters learned from the MIMIC-IV training set rather than statistics from external cohorts, reducing one common source of leakage.[1]
What the model added
OmniTFT extends the Temporal Fusion Transformer with one shared backbone and separate output heads for each target. Training deliberately sampled similar numbers of stable and volatile time windows. It also regularised rare categorical embeddings, encouraged concentration among groups of historical inputs and aligned short-range attention with changes in the learned patient representation. Each final target checkpoint contained about 1.83 to 1.85 million parameters.
On the MIMIC-IV internal test set, the model had the lowest mean absolute error among tested systems for five of eight targets, ranked second for creatinine and respiratory rate, and third for temperature. External errors generally worsened for several targets: for example heart-rate MAE rose from 4.12 beats per minute internally to 6.29 in MIMIC-III and 6.65 in eICU; oxygen-saturation MAE rose from 1.04 percentage points to 1.90 and 2.15. External validation therefore demonstrates transfer, not invariant performance.
Ablations across ten matched training seeds found that removing the balanced sampler increased mean absolute error by about 37% to 60% for blood pressure, heart rate, oxygen saturation and respiratory rate. Removing the grouped regulariser caused the largest deterioration for the other targets. These comparisons support the design choices within this dataset, but paired seed tests are not a substitute for prospective hospital validation.[1]
How the kidney-injury analysis was constructed
The researchers fixed a decision point 24 hours after ICU admission and excluded stays that already met stage 1 kidney-injury criteria. The primary outcome was incident stage 2 or worse AKI in the following 24 hours. Three pipelines used the same stays and labels: creatinine from hours 12 to 24, all observed creatinine up to hour 24, or the observed record plus summaries of OmniTFT's forecast for hours 24 to 48.
Within each setting, the downstream risk model was selected independently from logistic-regression and gradient-boosting candidates using hospital-grouped folds. This means the comparison tests complete pipelines rather than adding forecasts to one frozen risk model. The authors reported AUROC, average precision, Brier score, calibration and decision curves, with 2,000 clustered bootstrap resamples. Some secondary differences, including average precision and decision-curve estimates at several thresholds, remained uncertain.
Forecast uncertainty was not consistently calibrated after transfer. The reported 10th-to-90th percentile bands departed from their intended coverage for some external targets, including temperature and the oxygenation ratio. Before deployment, each hospital would need target-specific recalibration, monitoring for missingness and drift, and a workflow study measuring whether a forecast changes decisions without causing harmful distraction.[1]
What clinicians and health systems should take from it
The useful idea is a shared forecasting layer that can reuse one representation while respecting different clinical time scales. That could reduce the engineering burden of maintaining separate models for every measurement. It might also support a dashboard that displays expected trajectories instead of another binary alarm. The open code and public datasets make independent replication possible.
It is not ready for bedside care. None of the data is contemporaneous: MIMIC-IV covers one US academic centre from 2008 to 2019, MIMIC-III CareVue is older, and eICU covers 208 US hospitals from 2014 to 2015. Treatment, documentation and patient populations have changed. The model omitted continuous waveforms such as ECG and arterial-line traces, and no study tested clinician comprehension, alert fatigue or treatment effects.
The assessment would change with a preregistered silent deployment across several current hospitals, followed by a controlled trial if calibration and fairness remain acceptable. Investigators should publish event counts, alert workload, calibration by site and patient group, treatment changes, missed events and clinical outcomes. A higher AUROC alone cannot show that patients benefit.[1]
What this means for people
- ICU clinicians could gain trajectory forecasts across several measurements, but only if local validation shows that added information is timely and does not worsen alarm burden.
- Patients would need evidence that forecasts improve decisions or outcomes, not just retrospective discrimination.
- Hospital teams must validate calibration, missing-data handling and subgroup performance on current local data before any clinical use.
Global context
The research team is based in Japan, but all patient data came from historical US ICU databases. Hospitals in other health systems, and modern US units, should not assume transportability without fresh local validation and governance.
What the evidence does not yet show
- The study was retrospective and no clinician acted on the forecasts.
- Only 45 of 4,685 stays had the primary stage 2-or-worse kidney-injury outcome.
- Cohorts were enriched for sepsis, ARDS or AKI and do not represent all ICU patients.
- External uncertainty bands were miscalibrated for some targets and performance varied by cohort.
- All patient data were historical US records; clinical practice and documentation have changed.
What to watch next
- Preregistered silent validation on current multi-country ICU data
- Local recalibration and subgroup results with event denominators
- A workflow study measuring alert burden and clinician response
- Controlled evidence of treatment or patient-outcome benefit
Living evidence record
Impact record IAI-0UG4DN8
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
16 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Digital Health published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Whether one shared time-series model can forecast several ICU measurements across hospitals and add information to retrospective acute-kidney-injury risk stratification
Topics: Clinical AI · Intensive care · Acute kidney injury · Electronic health records · External validation
Regions: Japan · United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
The Royal Society
AI-generated illustration · not a video frame
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
Healthcare, culture & human impact · Background explainer
Published 2026-07-01 · 44 min 28 sec
Is this the AI future we want? | The Royal Society
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Health & Life Sciences
An Alzheimer’s MRI model scored 94.5%. On new data, it fell to 61.6%
For a patient, the important test is whether a model works beyond familiar images. This classifier’s accuracy dropped sharply on an external dataset, and it remains a research system rather than a clinical diagnostic tool.
7 min · 1 source
Health & Life Sciences
Can AI flag a longer ICU stay?
A peer-reviewed model separated longer-stay risk across 5,281 retrospective sepsis admissions in one US development cohort and two Korean external cohorts. Its AUROC fell from 0.848 internally to 0.781 at the smaller Korean site, and no prospective study tested whether alerts improve care or capacity.
7 min · 2 sources
Health & Life Sciences
Can EHR-note AI help distinguish epilepsy from PNES?
EpiScreen separated epilepsy from psychogenic non-epileptic seizures across 13,633 records from two US datasets and improved accuracy in an eight-clinician simulation. It remains retrospective decision support, not a substitute for neurological assessment or video-EEG.
8 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.