A stock model scored 63.84%—a calendar rule did better
A peer-reviewed audit found that forward-filled non-trading days created an easy target in S&P 500 data. SHAP then explained the artefact faithfully, while performance on genuine next-day moves fell below a majority baseline.
The short version
3 key points
- What changed
- The public file contained 1,825 calendar-day rows from November 2015 to November 2020; 567 rows, or 31.1%, were exact forward-filled copies of the previous trading session.
- Why it matters
- Workers and savers can be affected when an apparently accurate model informs trading, risk limits or product marketing. Independent data audits and simple baselines are safeguards against allocating money on a shortcut that disappears in live markets.
- What to watch
- Independent reproduction on other indices and assets, post-2020 temporal validation, preregistered pipelines, stress tests for duplicated timestamps and leakage, and live or paper-trading results after realistic costs.
A copied-price artefact created a predictable label
Researchers audited a public S&P 500 daily-price file used to train an XGBoost model for next-day market direction. The raw file covered 1,825 calendar days from 23 November 2015 to 20 November 2020. It included weekends and market holidays, with 567 rows—31.1%—created by copying the previous trading session's prices and volume. Because the target was defined as whether the next close was strictly higher, those repeated rows automatically became 'Down'. A calendar-only rule therefore reached 71.19% test accuracy, beating the model's reported 63.84% and the 59.32% majority baseline.[1]
Who it affects
Workers and savers can be affected when an apparently accurate model informs trading, risk limits or product marketing. Independent data audits and simple baselines are safeguards against allocating money on a shortcut that disappears in live markets.[1]
Keep in mind
One public S&P 500 file; 2015–2020 only; six walk-forward folds; no other asset, geography, learner or post-2020 market; historical backtest rather than prospective deployment.
Source analysis · 1 linked source. Original source date: 17 October 2026.
Continue the story
Read next
Finance & Business
How much do investment advisers disclose about using AI?
A peer-reviewed study found affirmative AI-use language in 23.7% of 388 US adviser brochures and risk language in 27.6%. It measures formal disclosure—not underlying adoption, safe performance or AI-washing.
2 min · 2 sources
Finance & Business
Is the AI boom propping up global growth—or concentrating it?
The IMF says AI investment and its hardware supply chain are helping activity withstand other shocks, but it does not assume an extra AI productivity gain in its baseline. Its 13 October outlook projects 3.1% global growth in 2026 while warning that the benefit is concentrated among economies connected to the technology cycle.
3 min · 3 sources
Read more The full report, methods and context
A copied-price artefact created a predictable label
Researchers audited a public S&P 500 daily-price file used to train an XGBoost model for next-day market direction. The raw file covered 1,825 calendar days from 23 November 2015 to 20 November 2020. It included weekends and market holidays, with 567 rows—31.1%—created by copying the previous trading session's prices and volume. Because the target was defined as whether the next close was strictly higher, those repeated rows automatically became 'Down'. A calendar-only rule therefore reached 71.19% test accuracy, beating the model's reported 63.84% and the 59.32% majority baseline.[1]
The score disappeared when the question became a real trading-day forecast
After feature construction, the original experiment used 1,766 rows and 32 engineered features, split sequentially into 1,412 training and 354 test observations. Among the test observations, 108 had a zero prior return created by the copied rows. On the remaining 246 genuine next-day moves, accuracy was 49.19%, below the 58.54% majority baseline. Removing non-trading rows left 1,199 genuine sessions. A six-fold walk-forward audit then produced pooled AUC of 0.476 with expanding training windows and 0.460 with rolling windows—near or below chance—and accuracy below the corresponding majority baselines.
The paper also tested a simple long-or-cash strategy with ten-basis-point trading costs. Its model strategy had an annualised return of minus 11.59%, a Sharpe ratio of minus 0.241 and maximum drawdown of minus 34.7%, while buy-and-hold returned 15.50% annualised with a 0.588 Sharpe ratio. That backtest is historical rather than investable evidence, but it reinforces the central finding: apparent label accuracy did not translate into a useful trading signal.[1]
The distinction that matters
How a calendar artefact became an explainable market signal
The model learned a real pattern in the file, but the pattern was created by data preparation rather than price discovery.
Input
Forward-fill weekends and holidays
567 copied rows; 31.1% of the raw file
Target
Label only strictly higher next closes as Up
Copied prices become automatically Down
Audit
Remove copied rows and walk forward
AUC 0.476 expanding; 0.460 rolling
An explanation can be faithful to the wrong problem
SHAP highlighted the zero-return feature that encoded copied calendar rows, so the explanation was faithful to what XGBoost had learned. The audit also found strongly correlated volatility features, with correlations up to 0.947, and showed that adding individual SHAP values across that cluster overstated the group's contribution. Dropping 14 of 32 features did not reduce predictive performance. Explainability therefore helped reveal the shortcut, but a plausible attribution chart could not repair a flawed target or duplicated observations.
The study is a forensic re-analysis of one US index file from 2015–2020, not evidence that every financial model leaks or that markets are inherently unpredictable. It did not test other assets, regions, learners or the 2021–2025 market. The practical lesson for analysts, vendors and investment committees is narrower: inspect timestamps and duplicate rows before modelling; compare against calendar and majority rules; use chronological holdouts; and report economic performance after costs. The assessment would change if a locked pipeline retained value across later periods, different markets and realistic execution assumptions.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Evidence check
One public S&P 500 file; 2015–2020 only; six walk-forward folds; no other asset, geography, learner or post-2020 market; historical backtest rather than prospective deployment.
Reporting basis: Source analysis. Original source date: 17 October 2026.
Explore the full source and evidence record
What this means for people
- Workers and savers can be affected when an apparently accurate model informs trading, risk limits or product marketing. Independent data audits and simple baselines are safeguards against allocating money on a shortcut that disappears in live markets.
Global context
The authors are based in China and audited US market data. Calendar handling, market holidays, liquidity and costs vary across exchanges, but the validation lesson travels: time-series rows must represent real observations, and every model should be compared with rules that exploit the same calendar structure.
What the evidence does not yet show
- One public S&P 500 file; 2015–2020 only; six walk-forward folds; no other asset, geography, learner or post-2020 market; historical backtest rather than prospective deployment.
What to watch next
- Independent reproduction on other indices and assets, post-2020 temporal validation, preregistered pipelines, stress tests for duplicated timestamps and leakage, and live or paper-trading results after realistic costs.
Living evidence record
Impact record IAI-01DTOGP
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
17 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 17 October 2026
Research question, topics and regions
How calendar artefacts and target leakage can create and explain spurious stock-direction accuracy
Topics: Financial markets · Target leakage · Explainable AI · XGBoost · Model validation
Regions: China · United States · Asia · North America · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.