Can capability audits uncover every hidden model skill?
No—not against a deliberately engineered cryptographic backdoor. A new preprint embeds an encrypted neural circuit that activates only with a secret trigger and resists four tested elicitation methods. It is a proof of possibility in synthetic models, not evidence that deployed models contain such circuits.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The authors encrypted a small chemistry classifier inside a 38-layer synthetic host with about 2.07 billion dense parameters and tested supervised fine-tuning, weight noise, gradient trigger search and latent adversarial training.
- 2Without the trigger, the encrypted model stayed near chance after 500-step fine-tuning and eight-layer latent adversarial training; a password-locked comparator rose above 91%.
- 3The full construction has not been integrated into a pretrained language model, its weights are statistically conspicuous, and the experiment does not show that ordinary training creates cryptographic backdoors.
Research topic
Whether a model developer can deliberately compile a useful neural capability into weights so that pre-deployment elicitation cannot recover it without a secret trigger
The answer: elicitation has a deliberate-adversary limit
A capability audit cannot be guaranteed to reveal every skill if the model was deliberately engineered to conceal one cryptographically. The new preprint constructs digital lockers inside neural-network weights. A secret trigger decrypts and executes another neural circuit; incorrect triggers yield benign behaviour. Under standard cryptographic assumptions, recovering the capability or learning what it does should be computationally infeasible for a polynomial-time auditor, even with full access to the weights.
This is a theoretical and proof-of-concept boundary, not evidence of a current hidden plot. The researchers built synthetic PyTorch model organisms and small tasks. They do not claim that cryptographic circuits emerge through ordinary training, and they have not integrated the complete construction into a pretrained language model. The result therefore changes what safety evaluations can promise: good elicitation can find many concealed behaviours, but absence of evidence cannot certify absence against a malicious supplier.[1]
The Impact Brief · Free
Follow the evidence in ai risks & safety.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
How the digital locker carries a neural circuit
The construction encrypts the weights of a hidden network with a key derived from a trigger. A compiled hash circuit inside the host checks an input, reconstructs the hidden weights only when the trigger is correct and then executes their computation with SwiGLU-style units. The paper extends earlier work that hid a fixed string: here the concealed object can be an arbitrary bounded neural computation. The formal guarantee depends on the security of the digital-locker construction and hash assumptions, not on empirical obscurity alone.
There is a substantial engineering cost. The main chemistry model has 38 layers, about 2.07 billion dense parameters but only 1.90 million non-zero parameters, and a maximum residual width of 10,266. Roughly 95% of the dense count belongs to hash subcircuits. Each hidden-network weight adds about 3.1 million dense host parameters in this implementation. An optimised 128-bit, 24-round variant reaches 110 layers and 3.58 billion dense parameters. These overheads make the result a demonstration of possibility rather than a practical recipe for concealing a frontier-scale capability today.[1]
The chemistry experiment used a small binary proxy
For an applied test, the authors reduced reaction prediction to balanced binary classification: given reagents and a candidate product derived from the USPTO corpus, decide whether the reaction yields that product. Molecules were represented as SELFIES and compressed into 64-bit Morgan fingerprints. The hidden classifier was a ternary multilayer perceptron with widths 64, 8, 16 and 2. That is a small proxy for a CBRN-relevant capability, not a system that designs chemical weapons or performs full synthesis planning.
The paper reports held-out accuracy averaged across three seeds with 95% intervals, but the accessible methods text does not give the held-out example count. With the correct trigger, the encrypted classifier reached 73.5% accuracy; its separately trained password-locked comparator reached 92.7%. The two models contained different classifiers because of size constraints, so their triggered accuracies are not directly comparable as model quality. The useful comparison is how the same hidden skill behaved before and after attempted elicitation.[1]
Four elicitation routes failed under the tested budgets
After 500 steps of supervised fine-tuning, the password-locked model reached 92.8% accuracy while the encrypted model remained at 50.9%, close to chance and to a 50.6% random-network control. Latent adversarial training at eight layers produced 91.7% for the password-locked model and 52.9% for the encrypted model, again close to the 53.6% random control. Adding weight noise at scales 0.01 and 0.1 did not materially change that pattern.
Gradient-based trigger search recovered the eight-bit trigger only by effectively exhausting all 256 candidates, matching a query-budgeted random search. With a 16-bit trigger it reached 52.7%, near chance. A second experiment hid a four-bit subset-parity task. Fine-tuning recovered the skill from a trained password-locked model but not from the compiled construction. These results support resistance to the particular methods and budgets tested; they do not show that every future interpretability, cryptanalysis or provenance technique must fail.[1]
What evaluators and procurers should change
Capability elicitation remains useful because most real failure modes are not designed with a cryptographic guarantee. But evaluation should be one layer rather than a certificate. Buyers of high-consequence models need to know who trained the system, which code and checkpoints produced it, how weights moved through the supply chain and whether an independently built artefact matches the claimed provenance. Runtime monitoring and least-privilege deployment can limit harm even when pre-deployment tests miss a behaviour.
Open-weight access alone is not a complete answer. The paper's claim explicitly considers an auditor who can inspect all weights yet cannot efficiently decrypt the hidden circuit without the trigger. Conversely, the present compiled weights are statistically conspicuous: about 99% are exact zeros and the remainder are largely integer-valued. A crude anomaly detector may therefore flag that the model was modified even if it cannot identify the hidden computation. Provenance and anomaly screening complement elicitation rather than replace it.[1]
What remains unproven
The complete construction has not been embedded in a functioning pretrained language model. Individual execution units worked inside Gemma 3 270M, but attention routing and residual interactions for the whole circuit remain future work. The chemistry task is tiny, the overhead is large and the raw weights are easy to distinguish from ordinary model weights. The authors' cryptographic guarantee also relies on assumptions about the locker and adversary; it does not make the presence of a modified circuit undetectable.
Confidence would change if an independent team reproduced the compilation, embedded a larger capability in a useful pretrained model, demonstrated resistance to broader analysis and obfuscated the statistical signature without breaking normal performance. Evidence that supply-chain attestations or runtime controls reliably detect or neutralise such models would reduce the practical concern. For now, the paper establishes a boundary condition: capability audits can be highly informative without being logically complete.[1]
What this means for people
- Regulators and safety teams should avoid treating a clean capability evaluation as proof that a model has no hidden behaviour.
- Organisations deploying high-consequence models need supply-chain provenance and runtime controls as well as pre-release tests.
- The paper does not show that people currently face a cryptographically hidden capability in any commercial model.
Global context
Model supply chains cross borders, while access to training code, weights and independent evaluation differs by jurisdiction. A cryptographic backdoor would challenge both closed and open-weight governance, but the immediate policy response is general: combine capability testing with provenance, controlled deployment and incident monitoring. The paper does not establish a country-specific threat or deployed incident.
What the evidence does not yet show
- This is an unreviewed preprint using deliberately compiled synthetic models, not evidence about deployed foundation models.
- The full encrypted construction was not integrated into a pretrained language model; only individual units were tested in Gemma 3 270M.
- The chemistry classifier is a small binary proxy, and the accessible methods text does not state the held-out sample denominator.
- The main construction has very high parameter overhead and statistically conspicuous weights.
- Resistance was tested against four elicitation approaches under finite budgets; untested cryptanalysis or forensic methods could behave differently.
What to watch next
- Independent replication and peer review of the cryptographic construction
- Integration into a useful pretrained model without an obvious weight signature
- Provenance, attestation and runtime-monitoring defences tested against released model organisms
- Broader elicitation and interpretability attacks with clearly reported compute budgets
Living evidence record
Impact record IAI-0MW3L1W
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
13 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 13 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
AI Risks & Safety
Can adding more AI agents create a hidden cyber-risk threshold?
A new theoretical preprint shows how collaboration could produce a critical population size above which agent growth becomes self-reinforcing. It does not show that such a threshold exists in deployed systems, or estimate where one would be.
9 min · 4 sources
AI Risks & Safety
Can AI judges be accurate yet disagree on the errors that matter?
Yes. In a new unreviewed study of ten reasoning models, average accuracy stayed high on a 600-item answer-checking benchmark, while prompt and model choices sharply changed whether errors were too lenient or too strict. The result argues against trusting one headline accuracy score—or one AI judge.
9 min · 2 sources
AI Risks & Safety
Can clinicians see the evidence behind approved diagnostic AI?
A peer-reviewed audit found public, device-specific performance evidence for 30 of 77 approved pathology and haematology AI products. The result is a transparency finding—not proof that the other 47 lack regulatory evidence or that the documented products are clinically equivalent.
9 min · 3 sources
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.