Can AI evidence support health-spending decisions?
A 2,476-record scoping review found rapid growth but little validation—and its own audit judged only 51.3% of a 300-record sample genuinely relevant. The study's LLM reference standard makes that estimate informative, not definitive.
The short version
3 key points
- 1.The PRISMA-ScR review searched PubMed for 2010–2025, screened 2,628 records and included 2,476, then audited a random sample of 300 against a blinded LLM classifier.
- 2.Only 154 of 300 sampled records were judged genuinely relevant, implying a precision-adjusted estimate of 1,271 studies; however, the reference was an LLM rather than dual human expert review.
- 3.Only 38 of 2,476 records were randomised trials and 17 were validation studies. The review found stronger evidence for observable clerical tasks than for decision-level or counterfactual claims.
The answer: useful automation, weak evidence for the final decision
AI can accelerate identifiable parts of health-economics work, including screening references, extracting structured information, writing code and processing real-world data. A new systematic scoping review found documented gains where there is an observable answer against which output can be checked. It found much less evidence that a strong predictive score makes a reimbursement estimate, causal effect or cost-effectiveness conclusion more trustworthy.
That distinction matters to public services, insurers and patients. A model may find records quickly and still exclude the wrong study; predict an outcome accurately and still estimate a treatment effect poorly; or generate a plausible economic model whose assumptions drive the result. The review's central rule is therefore practical: predictive performance is not evidentiary performance. Decision makers need to test how the AI changes the quantity used for funding, access or reimbursement—not merely whether it produces a high AUC or fluent report.[1]
How 2,642 search results became 2,476 records
The author searched PubMed on 9 September 2026 for publications from 2010 through 2025 that combined AI terms with health-economics and outcomes-research terms. Eligible records were English-language journal articles, reviews or conference proceedings with an abstract. Fourteen duplicates and 152 ineligible records were removed, leaving 2,476. A published term dictionary assigned eight overlapping application domains and one of six AI method categories from titles, abstracts and indexing terms.
The raw count rose from 29 records in 2010 to 852 in 2025. Economic evaluation was assigned to 1,559 records, while real-world data and causal inference appeared in 496 and governance, reporting and validity in 494. Generative AI appeared in 103 records in 2025. Yet those are classification outputs, not verified practice counts. The author deliberately tested the search and dictionary because bibliometric totals can become marketing claims when their false-positive rate is unknown.[1]
The review audited its own headline number
A random sample of 300 included records was reclassified by an independent large language model that was blinded to the dictionary labels and received only the title and abstract. It judged 154 records, or 51.3%, genuinely relevant to health economics and outcomes research. Applying that proportion to the full corpus produced a precision-adjusted estimate of 1,271 relevant studies, with a 95% confidence interval from 1,131 to 1,410.
Agreement varied sharply by task. It was substantial for real-world data and causal inference, with Cohen's kappa of 0.729, but poor for economic evaluation, with kappa of 0.144. The dictionary labelled 186 of the 300 sampled records as economic evaluation; the blinded classifier labelled 69, and dictionary precision was 29.6%. In other words, words about cost, efficiency or resource use frequently appeared without the paper being an economic evaluation in the sense a reimbursement reviewer would use.[1]
The audit has an important circularity
The independent classifier was another LLM, not a panel of health economists. The paper openly acknowledges this circularity and treats the model as a reference standard rather than truth. It may be too strict about what counts as relevant or too lenient in unfamiliar subfields. A dual human review of the same 300 records is the necessary next step. Until then, 51.3% should be read as a measured warning about noisy counting, not a definitive correction factor.
There was also an unresolved difference of 273 records between a range query and the sum of per-year PubMed queries, probably related to ambiguous Medline dates. PubMed omits much conference and grey literature, the search was English-only, and title-and-abstract classification cannot recover every method or deployment detail. The review is stronger for exposing these errors than a study that hides them, but transparent uncertainty remains uncertainty.[1]
Validation was rare relative to publication volume
Of the 2,476 records, 674 were indexed as reviews, 121 as clinical studies, 38 as randomised trials and only 17 as validation studies. Those categories are based on PubMed indexing rather than an individual risk-of-bias assessment of every paper, but the imbalance is striking. The literature expanded far faster than evidence that tools travel between settings, preserve calibration or improve a real funding decision.
The review's narrative evidence map found the clearest measured value in tasks with checkable ground truth. Some literature-screening systems reached high sensitivity and throughput, although performance varied widely and false positives could be substantial. In causal estimation, synthetic data and deployed monitoring, apparently strong predictive summaries did not reliably guarantee accurate treatment effects, privacy protection or stable calibration. The outcome a health-spending body needs is often counterfactual: what would happen, and at what cost, under another decision. A prediction metric alone does not answer it.[1]
What commissioners, analysts and patients should require
For clerical use, organisations should measure recall, extraction accuracy, time saved and an auditable trail back to the source. Human reviewers need a way to inspect exclusions and correct structured fields. For inferential use, they should compare the final treatment effect, budget impact or cost-effectiveness conclusion with a credible non-AI analysis and vary model assumptions to show how much the decision depends on the tool. A faster pipeline is valuable only if it preserves the evidence needed for accountability.
Patients and clinicians are affected when evidence pipelines shape coverage, guidance or access. An AI-derived endpoint should not become more persuasive merely because its internal model performs well. Decision documents should disclose where AI was used, the human checks, the data population, external validation, error rates and whether the funding recommendation changes under reasonable alternatives. Procurement contracts should preserve access to prompts, code, versions and audit logs rather than accepting a vendor's performance summary.[1]
What would change the assessment
A human dual review of the archived 300-record sample would show whether the estimated false-positive problem survives expert adjudication. Wider searches across Embase, economics databases, regulatory submissions and non-English sources would test coverage. The most useful prospective studies would embed a locked AI workflow in real health-technology assessments and compare completeness, errors, time, cost, analyst workload and the final recommendation against ordinary practice.
The author reported no funding and no conflict related to the work, while disclosing an academic affiliation and a health-economics consultancy serving pharmaceutical and life-sciences clients; no client participated or reviewed the manuscript. Search strings, records, prompts, outputs and code are publicly archived. The defensible conclusion is not that AI has failed health economics. It is that automation can be faster without making a spending decision truer, and the field must measure that difference explicitly.[1]
What this means for people
- Patients can gain or lose access when uncertain AI-derived evidence influences reimbursement, so the final decision needs auditability and sensitivity analysis.
- Health economists should separate clerical automation from causal or cost-effectiveness inference and validate the decision-level quantity, not only the model.
- Public and private commissioners should require disclosure, reproducible inputs and a comparison with ordinary analysis before relying on AI-supported submissions.
Global context
The review covered English-language PubMed records and therefore underrepresents non-English work, economics databases, conference activity and operational grey literature. Health-technology assessment rules and reimbursement evidence differ across countries, so findings about publication volume do not establish adoption by any one payer or regulator.
What the evidence does not yet show
- Single-database, English-language search with limited conference and grey-literature coverage
- An LLM rather than dual human experts served as the reference for the 300-record audit
- Large measured classification errors, especially for the economic-evaluation label
- An unresolved 273-record discrepancy between PubMed date-query approaches
- Record-level indexing counts do not establish risk of bias, real-world adoption or decision benefit
What to watch next
- Dual human adjudication of the archived 300-record sample
- Prospective use in real health-technology assessments
- Decision-level sensitivity analysis for causal and cost-effectiveness outputs
- Disclosure of AI use in payer and regulatory submissions
- Replicated gains in screening and extraction with full audit trails
Living evidence record
Impact record IAI-1D7NLY5
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
15 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Journal of Health Economics and Outcomes Research published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 15 October 2026
Research question, topics and regions
How much AI work in health economics and outcomes research is genuinely relevant, validated and suitable for reimbursement or public-spending evidence
Topics: Health economics · Health technology assessment · Evidence synthesis · Reimbursement · Validation · Large language models
Regions: Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
UNESCO
AI-generated illustration · not a video frame
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
AI, jobs & human choice · Background explainer
Published 2024-10-14 · 11 min 27 sec
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Finance & Business
Can AI explain why kidney-disease hospital stays cost more?
A two-hospital Chinese study found that machine-learning models could retrospectively account for part of the variation in chronic-kidney-disease admission costs. The models used information accumulated during the stay, so they are not forecasts and should not set individual budgets.
8 min · 1 source
Finance & Business
Can an LLM explain a fraud alert without seeing transaction data?
An Indian research team tested a graph detector, fixed decision rules and a constrained language model on a 1,000-transaction Nigerian sample. The design limits what the LLM can invent, but simpler tabular models detected fraud better and no auditors judged the explanations.
9 min · 1 source
Finance & Business
Can AI health-insurance fraud tools be trusted?
A review of 33 studies finds broad technical promise but little comparable evidence on external validation, cost, fairness or real-world deployment. A suspicious-claim score is not proof of fraud and still requires accountable human investigation.
6 min · 1 source
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.