When should utilities commit crews using AI weather forecasts?

The data and decision being modelled

The study linked EAGLE-I utility outage reports with operational GraphCast and FourCastNet forecasts, ERA5-Land variables, tree canopy, NOAA storm reports and US Census housing-unit estimates. It covered the contiguous United States from 1 January 2022 through 31 December 2024 at county and six-hour resolution. After excluding counties that reported on fewer than half of study days, 2,491 counties remained.

Before missing intervals were removed, the grid contained 10.9 million county-window cells; 46.3% were unobserved and were masked rather than filled with zeros. An outage event meant at least 5% of housing units equivalent without power for 24 hours, allowing gaps up to 12 hours. Housing units from the 2018–2022 American Community Survey supplied the denominator because EAGLE-I customer counts were inconsistent and a denominator derived from the observed maximum could leak test-year information.

The researchers trained LightGBM occurrence and conditional-severity models on 2022, used 2023 for early stopping, model selection and conformal calibration, and reserved 2024 for testing. Cross-lead comparisons used 4,721,771 county-windows available at all four leads. The occurrence base rate in 2024 was 2.27%, making precision-recall area more informative than a receiver-operating curve dominated by non-events.

For the decision analysis, counties were ranked by predicted outage occurrence and the top 10% were treated as staging sites. The outcome was the share of observed outage severity inside that staged set. Main comparisons focused on 71 high-burden six-hour windows common to all leads. Those windows fell in only 18 storm-week blocks, which the authors resampled 5,000 times. That block count—not millions of county records—is the effective denominator for the main uncertainty intervals.[1][2]

What forecast lead time changed

Occurrence skill fell steadily with lead time: precision-recall area was 0.1207 at 24 hours and 0.0646 at 120 hours with all predictors, compared with a county-climatology reference of 0.047. Absolute skill remained modest even at one day. The AI forecast archive also underestimated 10-metre wind, and it lacked an instantaneous gust field. In a wind-only comparison, substituting analysis wind improved skill by about 15%, while adding observed gusts improved it by roughly 33%; those analysis fields are upper bounds, not operational forecasts.

For captured burden, most of the decline occurred between three and five days. The 24-versus-48-hour difference was +0.029 with a 95% interval of +0.004 to +0.065; 24 versus 72 hours was not distinguishable from zero on the pooled measure; and the 72-versus-120-hour gap was supported. When the authors restricted burden to outages near their onset, the decline inside three days became clearer: within 24 hours of onset, captured burden fell from 0.506 at a 24-hour lead to 0.431 at 72 hours and 0.347 at 120 hours.

The onset analysis highlights a practical problem. Some high-burden windows occurred after an outage had already started, when pre-staging could no longer prevent travel delay. Removing older ongoing burden made shorter lead times look more valuable. Yet the study cannot tell how much earlier staging would shorten any restoration, because it lacks feeder damage, crew productivity, travel and actual dispatch records.[1]

A data-leakage warning extends beyond this application

The paper deliberately measured a common timing error. Land-surface features such as antecedent precipitation can be joined at the forecast's valid time even though some of that information did not exist when the forecast was issued. In the intentionally leaky specification, skill was inflated by 42%–53%, enough to create an apparent long-lead advantage over forecast meteorology. Moving the variables back to issue time reversed that conclusion.

Correctly timed land-surface variables still improved occurrence skill by 17.5% at 24 hours, from 0.1027 to 0.1207. But captured burden changed by only +0.028, with a 95% interval spanning −0.003 to +0.069; no lead-time interval excluded zero. Better average ranking therefore did not demonstrably improve the small set of high-stakes staging decisions. Utilities and vendors should validate a model on the operational choice it supports, not stop at a global accuracy gain.

Uncertainty estimates also failed unevenly. Nominal 90% conformal intervals covered 86.1% of all test observations but only 47.6% of tropical-cyclone events. The 2023 calibration year contained no major mainland-US hurricane landfall, while the 2024 test year contained 1,783 tropical event-windows including Beryl, Helene and Milton. Retrospective calibration by hazard class improved coverage, but the class was assigned after the event and therefore was not an operational fix.[1]

What utilities, regulators and communities should take from it

Utilities can compare the study's break-even framing with their own mutual-assistance records: how much crew availability is actually lost by waiting from 72 to 24 hours, and how often does an earlier placement need revision? That local evidence is essential because the study supplies no empirical availability curve. Dispatch centres should also test whether ranking counties improves their real service territories, contracts and staging constraints rather than assuming national county results transfer directly.

Regulators and emergency planners should ask for time-correct feature construction, out-of-year testing, hazard-specific calibration and decision metrics tied to restoration operations. A model can improve precision-recall performance while leaving the actionable top of the ranking unchanged. It can also appear well calibrated overall while failing for the event class that produces the largest outages.

For households, earlier crew positioning can matter most where power loss threatens medical equipment, heating or cooling, but this study does not analyse equity in the recommended locations. County resolution can conceal neighbourhood and feeder differences, and reported outage coverage varies by utility. Future work should test whether improved aggregate capture also reaches medically vulnerable and historically underserved communities.[1][2]

Evidence limits and what would change the assessment

Missing outage reports were not random, the severity model was weak, and only three years were available, with one training year and an unusually active hurricane test year. The primary decision intervals rely on 18 storm-week blocks and may still be too narrow. Only one pair of archived AI weather models was evaluated, and neither supplied the gust field the analysis suggests could matter more than modest improvements to existing wind variables.

Confidence would rise with prospective tests inside utilities, multiple years and regions, additional weather models, observed mutual-aid availability, feeder-level damage and actual restoration times. Analysts should predefine staging constraints and equity outcomes, preserve every forecast vintage and compare recommendations with both operational practice and simpler baselines.

The assessment would strengthen if preregistered deployments showed that lead-time-aware rankings improved crew arrival or restoration without shifting risk to poorly observed communities. It would weaken if the three-to-five-day gap did not replicate outside the hurricane-heavy 2024 test year. For now, the paper offers a useful decision framework and a strong leakage diagnostic, but not a universal dispatch rule.[1][2]