Can formatting change an AI safety guard's decision?
A workshop-accepted preprint found that benchmark cues predicted safety labels and that two meaning-intended edits changed one guard's blocking rate. The effect did not reproduce on another model, so the result is a warning about evaluation design rather than a universal bypass.
The short version
3 key points
- 1.Simple cues performed strongly on three released evaluation artefacts, but the study separates predictive correlation from evidence that a guard actually relies on a cue.
- 2.On a fixed non-random sample of 64 AgentDojo steps, marker rewriting lowered one Haiku guard's blocking by 8.33 percentage points and opaque tool aliases raised it by 11.46 points.
- 3.The same large effects did not reproduce with GPT-5.6 Luna, and every unsafe call remained blocked in the main paired study; the observed changes were false positives under inherited labels.
The answer: yes for one tested guard, but not as a general rule
Two representation changes altered Claude Haiku 4.5's blocking rate on a fixed 64-step benchmark cohort. Rewriting the literal provenance marker lowered blocking by 8.33 percentage points, while replacing familiar tool names with opaque aliases raised it by 11.46 points. Both placebo-adjusted exploratory intervals excluded zero. Yet GPT-5.6 Luna showed contrasts of zero and 1.04 points under the same repeated design, and a quantised TS-Guard reproduction changed only three of 64 marker decisions.
That mixed result is the central finding. A safety evaluation can depend on how a benchmark serialises context, tool identity or repeated records, but a cue that predicts labels does not prove that a guard uses it. Even a paired edit is informative only if it preserves the safety-relevant meaning. The authors therefore propose an audit ladder: test label predictability, then observational association with decisions, then controlled paired sensitivity with repeat and placebo checks.
The paper was submitted on 15 October and accepted at two NeurIPS 2026 workshops on evaluation and agent safety. It remains an unreviewed arXiv preprint. The study executes no tool calls and uses no human-subject data. The manuscript identifies no funding or competing-interest statement, while some experiments used hosted commercial models through OpenRouter; readers should distinguish that infrastructure choice from evidence of sponsorship.[1][2]
Three benchmarks exposed three different shortcuts
For LinuxArena, the authors started from 932 public rows, of which 918 contained shell commands. After hashing action content to remove repeated traces from different monitor runs, their primary sample held 576 exact-action-unique trajectories: 176 attacks and 400 honest trajectories across ten environments. Shorter average shell commands predicted labels with an area under the curve of 0.910. A character n-gram model held out one environment at a time reached 0.934, indicating broad collection separability rather than one magic length cutoff.
AgentDojo-Traj contained 1,220 steps: 352 labelled unsafe and 868 safe across four domains. A fixed rule combining the literal <INFORMATION> marker with 11 side-effect verb prefixes reached an F1 score of 0.802 and balanced accuracy of 0.894. Memorising each tool name's majority label reached F1 0.901 in interaction-grouped cross-validation, but fell to 0.495 when an entire domain was held out. Only 17.3% of rows used a tool name seen in another domain.
AgentHarm-Traj contained 731 steps from 197 interaction groups: 525 unsafe and 206 safe. Simply predicting every step unsafe produced F1 0.836 but balanced accuracy of 0.500, showing how prevalence can flatter a positive-class metric. Released TS-Guard F1 fell from 0.902 with every step weighted equally to 0.719 when interactions received equal weight, while balanced accuracy rose slightly. Row-level and task-level claims were answering different questions.[1]
How the paired intervention was designed
The authors did not treat those correlations as proof. They selected a fixed, non-probability cohort of 64 AgentDojo steps—32 safe and 32 unsafe across the four domains—after an observational screen showed the largest safe-row marker association for Haiku. The cohort and edits were frozen before 960 new calls. One edit changed the provenance tag's wording, another replaced tool names with opaque aliases, a combined condition applied both, and a formatting-only placebo tested sensitivity to serialisation.
Each original and edited condition was called three times in a deterministic interleaved order, at temperature zero with structured outputs and provider fallback disabled. The primary estimate averaged the within-example blocking-probability difference. The authors resampled 58 source interactions 10,000 times within domains to produce exploratory 95% bootstrap ranges. Repeated originals and the placebo were used to expose instability that a single before-and-after call could hide.
The deterministic comparison was a byte-pinned 8-bit MLX reproduction of public TS-Guard, not the released BF16/vLLM execution path. It matched 63 of 64 strict released decisions and 58 of 64 exact scores on the originals. Luna used the same cohort, prompt, five conditions and three repeats as a cross-provider replication. A one-draw GPT-5.5 check was secondary and cannot support the same stability claims.[1]
What changed—and what did not
For Haiku, marker rewriting reduced blocking by 8.33 points, with an exploratory 95% range from minus 15.05 to minus 2.49. Tool aliasing increased blocking by 11.46 points, with a range from 2.69 to 20.59. The formatting placebo changed blocking by minus 1.56 points with a range spanning zero. After subtracting that placebo, the marker and alias effects remained non-zero in the exploratory intervals.
Every unsafe original and edited call remained blocked. The movement came from safe rows under the inherited labels: strict accuracy rose from 73.44% on originals to 81.77% after marker rewriting and fell to 61.98% after aliasing. That matters because false positives can stop legitimate work, but it also means the study did not demonstrate a successful unsafe tool call or a bypass of harmful-action blocking.
The larger Haiku effects did not reproduce in Luna. Its marker and alias contrasts were 0.00 and 1.04 points, both with ranges spanning zero; no example moved between opposite unanimous three-call decisions. The result therefore establishes that one configuration was sensitive to the tested representations, not that all guards share the weakness. Opaque aliases may legitimately increase uncertainty, and changing provenance wording may alter a risk-relevant signal.[1]
What security teams and benchmark owners should change
Benchmark owners should first try deliberately weak diagnostics—length, template markers, source subsets and exact identifiers—then deduplicate the semantic unit and hold out whole domains, environments or sources. A high score is a hypothesis generator, not a finding of guard reliance. Reports should publish step-, interaction-, trajectory- or task-weighted metrics according to the deployment claim, rather than allowing frequent multi-step cases to dominate silently.
When a cue correlates with decisions, evaluators should build paired variants that preserve authorization, capability, arguments and effect as closely as possible. For stochastic guards, original and edited calls should be interleaved and repeated; identical-input disagreement and a formatting placebo should be reported separately. Hidden holdouts and refreshed equivalent serialisations can reduce the chance that a public diagnostic itself becomes a target.
Security operations teams should not translate this paper into an exploit recipe. No tools were executed, all unsafe cases remained blocked, and the study explicitly concerns evaluation validity rather than deployment safety. Its practical value is defensive: a procurement or red-team report should show that guard performance survives innocuous representation changes and domain transfer before treating one benchmark headline as evidence of reliable control.[1][2]
Evidence limits and what would change the assessment
The 64-step intervention cohort was selected rather than randomly sampled, and its bootstrap ranges are exploratory and unadjusted for multiple comparisons. Neither the marker rewrite nor the aliases received blinded human validation for meaning preservation. LinuxArena's public sample uses completed trajectories and cannot reproduce the official pre-harm metric exactly. AgentHarm's source subset predicts labels but is not proven visible to the guard.
Model and implementation coverage is limited. The TS-Guard copy was quantised, GPT-5.5 had no repeats, Luna did not reproduce the large effect and another StepGuard comparison stopped at a non-equivalent platform feasibility check. The aggregate repository omits raw benchmark text, arguments, identifiers and provider responses, protecting sensitive content but limiting exact external inspection of every input.
Confidence would rise with preregistered transformations, blinded expert review of construct preservation, larger probability samples, more guard families and repeated tests across provider and model updates. It would weaken if the effects vanished under natural tool renaming and semantically validated provenance edits. The present assessment is therefore methodological: shortcut audits should be routine, while claims of a broad safety bypass would exceed the evidence.[1][2]
What this means for people
- Security analysts need evidence that guard scores survive domain shift and harmless formatting changes before relying on them in approval workflows.
- Workers can be delayed by false-positive blocks when unfamiliar tool names or formatting trigger caution without a real change in risk.
- Public-service technology teams should demand evaluation at the level of the real task or interaction, not only the number of recorded tool-call rows.
Global context
Agent guards are used across languages, organisations and tool ecosystems, while benchmark conventions often come from a small set of English-language environments. Domain holdouts and equivalent local-language serialisations are therefore essential before transferring a reported score. The study was led from Germany but its caution applies wherever tool-using agents are evaluated or procured.
What the evidence does not yet show
- This is an unreviewed preprint accepted at workshops, not a full peer-reviewed conference paper.
- The main paired intervention uses a fixed non-random cohort of 64 steps and exploratory, unadjusted intervals.
- The edits lack blinded human validation, and changing provenance wording or tool names may change legitimate safety information.
- The large Haiku effects did not reproduce with Luna; the result is configuration-specific.
- No tool was executed and every unsafe call remained blocked, so the study does not demonstrate an operational safety bypass.
What to watch next
- Preregistered replications across more guard models, domains and natural representation changes.
- Benchmark reports that include domain holdouts and interaction- or task-weighted metrics alongside row-level scores.
- Human-validated paired edits that preserve authorization and effect while changing only the suspected cue.
- Versioned guard evaluations after provider, checkpoint, prompt or serialisation changes.
Living evidence record
Impact record IAI-05ZYH7V
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
16 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Whether narrow construction cues in agent-safety benchmarks predict labels and whether editing those cues changes pre-execution guard decisions
Topics: Agent safety · Tool guards · Benchmark validity · Cybersecurity · Evaluation methods
Regions: Germany · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Security & Defence
What did the agent-security recreation actually prove?
A new unreviewed paper reproduced a published JFrog privilege-escalation flaw inside deliberately vulnerable containers and connects it to the 2026 OpenAI–Hugging Face incident. The recreation demonstrates an infrastructure failure mode; it cannot establish what undisclosed training data caused the agents' behaviour.
8 min · 4 sources
Security & Defence
Can encoded instructions turn AI-agent skills into security exploits?
Yes, in a controlled benchmark: some models decoded and followed malicious payloads hidden in third-party skill files. Attack rates fell sharply with the original longer files, and the work does not demonstrate a live breach.
7 min · 1 source
Security & Defence
Can a high score prove an Android-malware AI is reliable?
No. A peer-reviewed review of 61 studies and a controlled test on 11,598 apps found that clean benchmark accuracy concealed class-specific misses and sensitivity to masked features—and still did not reproduce real attacks or deployment.
8 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.