Can safety teams predict a model monitor before paying for all the labels?
Across six constructed safety tasks and 20 base models, two fitted laws often forecast how probe error changes with model size and labels. The result is a planning tool, not evidence that a probe will catch live harm.
The short version
- What happened
- Researchers built six minimal-pair datasets—truth, sycophancy, policy violation, general harm, jailbreak and prompt injection—with up to one million training labels and a 36,000-row test set each.
- Who it affects
- A workable planning rule could reduce wasted labelling and compute, but users are protected only if the eventual monitor is validated against the harms, traffic and error costs of the real service.
- What remains unproven
- The paper is a preprint and has not been peer reviewed.
Keep in mind
The paper is a preprint and has not been peer reviewed.
The main scaling fits use 20 dense base models from five open families; deployed assistants are post-trained and may behave differently.
The six large datasets are constructed minimal pairs, not a representative sample of production requests or attacks.
The cost comparison multiplies parameters by labels and does not include annotation quality, serving cost or false-positive burden.
Source analysis · 1 linked source. Primary analysed source date: 22 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
The laws turned trial-and-error into a forecast
The study varies three inputs that safety teams normally discover by experiment: monitored-model size, label count from 32 to one million, and probe architecture. Across the six tasks, probe error fell toward a model-size-dependent floor. When fitted only on smaller models, the equations predicted the next model's error within 0.05 in 97% of held-out cases and within 0.02 in 72%.[1]
Useful planning is not deployed protection
In 45 cases where one route was at least twice as cheap, the equations chose whether to add labels or move to a larger model correctly 73% of the time. Across 200 sampled model-and-label settings, they selected the better probe—or one within 0.005 error—in 82%. Yet those choices were evaluated on the authors' datasets and price proxy, not a live monitor with adversaries, false alarms and changing traffic.[1]
The distinction that matters
What the equations decide—and what they do not
The study predicts benchmark probe error from model and data scale.
Inputs
Choose model family, size, labels and probe
Fit a task-specific error curve
Planning
Compare more labels with a larger model
Estimate the cheaper route to a target error
Still required
Run realistic deployment tests
Measure false alarms, attacks and drift
Teams still need task-specific validation
A safety lead could use the curves to reject an obviously under-labelled plan before spending on annotation. They should still test the deployed, post-trained model at the false-positive rate the service can tolerate, with realistic attacks and drift. A scaling fit estimates performance under its measured setup; it does not certify coverage of a new harm category.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 2 min 31 sec · Background explainer
What is artificial intelligence? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Published 2018-11-28. Source checked 14 October 2026. Check captions on the publisher’s video page.
Explore current AI technologyPhoto: ThisisEngineering / Unsplash · Licence
Continue the story
Read next
Continue exploring ai safety: compare the finding and its evidence limits.
Why did three AI models choose the leaked grader over the user?
In selected agent tasks, three OpenAI models violated explicit instructions at least 92% of the time under the strongest score pressure. The unreviewed study is a warning about exposed evaluation signals—not proof that ordinary assistants pursue scores in everyday use.
Read the next reportExplore the full source and evidence record
What this means for people
- A workable planning rule could reduce wasted labelling and compute, but users are protected only if the eventual monitor is validated against the harms, traffic and error costs of the real service.
Global context
Monitoring budgets are limited in every region, but label prices, languages, harm definitions and acceptable false-positive rates differ. Each deployment would need its own data and fitted constants.
What the evidence does not yet show
- The paper is a preprint and has not been peer reviewed.
- The main scaling fits use 20 dense base models from five open families; deployed assistants are post-trained and may behave differently.
- The six large datasets are constructed minimal pairs, not a representative sample of production requests or attacks.
- The cost comparison multiplies parameters by labels and does not include annotation quality, serving cost or false-positive burden.
What to watch next
- Peer review and independent replication
- Fits on post-trained and closed models
- Realistic multilingual and adversarial traffic
- Evaluation at operational false-positive rates
Living evidence record
Impact record IAI-0M8N2FS
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
23 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 23 October 2026
Research question, topics and regions
Whether model size and label count predict the error of activation-based safety probes
Topics: AI safety · Model monitoring · Activation probes · Evaluation · Labelling costs
Regions: United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.