Can epidemic models be accurate overall yet still miss who bears the harm?

Historical simulations show why subgroup validation changes the answer

The researchers applied the framework to stochastic epidemic simulations of early COVID-19 spread in London, New York City and Santiago de Chile, and to the 2016 Zika outbreak in Colombia. The models included different structures and equity variables, allowing the team to test ordered groups, categorical groups and geography. These are retrospective examples calibrated to previously reported outcomes, not forecasts run before those outbreaks unfolded.

In London, four COVID-19 models combined either homogeneous or age-stratified contacts with either a flat or age-varying infection-fatality ratio. All were calibrated to total weekly deaths from March to June 2020. Yet their age-fairness scores differed sharply. Homogeneous mixing with a flat fatality ratio scored 0.58, while age-stratified contacts with the same flat fatality ratio scored 0.53. Adding age-varying fatality risk lifted the score to 0.92 with age-stratified contacts and 0.98 with homogeneous mixing. More detailed contact structure therefore did not automatically make the age distribution more faithful.

The New York analysis examined reported COVID-19 deaths by racial and ethnic group. Across 1,000 simulations per model, a baseline with evenly distributed risks scored 0.714. Adding group-varying susceptibility increased the score to 0.901, while assortative contact patterns reached 0.906. Models using age-adjusted heterogeneous fatality ratios sometimes scored worse than counterparts with a homogeneous ratio. The authors interpret this as a warning that a plausible extra layer can reproduce the wrong inequality if its assumptions do not capture the pathways that generated observed disparities.

The Santiago and Colombia cases broaden the demonstration to socioeconomic and spatial inequality. Across the paper, no single metric replaces conventional calibration. A model still needs epidemiological validity, uncertainty analysis and sensitivity testing. The fairness score adds a question that an aggregate error can miss: does the model reproduce who was harmed and where? The result is a validation tool, not a moral verdict on an algorithm or an automated decision system.[1][2]

The vaccine exercise is informative precisely because it is simplified

To show how the metric could affect a decision, the paper simulates a second New York wave with a hypothetical variant that is 50% more transmissible. It assumes that only susceptible people are vaccinated and that vaccination provides sterilising immunity. The researchers compare allocating doses in proportion to group population with strategies that also weight each racial or ethnic group's contribution to the Theil index. A parameter controls how strongly inequality influences allocation.

In the simulation, a fully Theil-based strategy produced a median 73.9% reduction in deaths and a combined population-and-Theil strategy produced 71.6%, with larger reductions across all groups than the population-only baseline. The pattern held across simulated vaccine supplies from 10% to 30% of the population. This suggests that explicitly representing unequal burden can, under some assumptions, improve both equity and total simulated outcomes rather than forcing a simple trade-off.

Those percentages must not be read as a clinical effectiveness estimate or a recommended real allocation formula. The model does not include adverse effects, incomplete protection, behavioural response, delivery barriers, hesitancy, eligibility rules or the ethics of allocating scarce doses. It uses historical group outcomes to guide a stylised intervention under fixed assumptions. A real programme would need current infection, vulnerability and access data; legal and ethical review; operational feasibility; and prospective monitoring for unintended effects.[1][2]

The framework creates a better audit question, not an automatic policy

For public-health agencies, the immediate implication is procedural. A model report should not stop at overall fit. It should identify which inequality dimensions matter for the decision, show data quality for each group, report fairness scores beside conventional error, and explain which assumptions drive differences. Where data are incomplete or categories conceal diversity, the uncertainty should be visible rather than converted into a confident group ranking.

The authors acknowledge that they do not solve the hardest upstream problem: deciding which variables represent actionable health inequities and how social conditions cause differences in exposure, vulnerability and care. Their models do not mechanistically connect structural causes with biological outcomes. A fairness score can show that a simulation fails to reproduce a disparity, but it cannot by itself identify whether the cause is housing, occupation, healthcare access, discrimination, age structure, reporting bias or another factor.

There is also a risk of treating observed inequality as a target to reproduce rather than a harm to change. The framework avoids that mistake only if users distinguish model validation from policy goals. Reproducing a historical disparity can make a model more descriptively faithful; a policy may still aim to reduce that disparity. Analysts should therefore show both the observed baseline and the intervention objective, and should never encode racial or socioeconomic categories as if they were fixed biological properties.

The current publication is peer reviewed, while the detailed record originated as a November 2025 medRxiv preprint. The authors provide data and code, report no competing interests, and disclose support from the Lagrange Project, Fondazione Cassa di Risparmio di Torino, MATRIX-Simons and the University of Trento's open-access initiative. Stronger evidence would come from independent replication, prospective outbreak use and studies showing that adding these checks changes decisions without worsening outcomes or reinforcing unreliable categories.[1][2]