How often did prompt injection divert coding agents?
In a new sandbox benchmark, attack success averaged 19.6% by an AI judge and 15.4% by artifact checks across 26,220 undefended trajectories. Rates varied sharply by model and harness, and the strongest tested prompt defence traded security against task utility.
The short version
3 key points
- 1.INFUSE paired 40 tasks with 56 attack scenarios and evaluated 19 models, producing 41,824 adversarial trajectories, including 26,220 without defences, plus 720 benign runs.
- 2.Mean attack success was 19.6% under a single AI judge and 15.4% under heuristic checks; the single judge under-detected compromise in a 219-item human study.
- 3.Instructional hardening cut judged attack success from 58.2% to 14.3% on a high-vulnerability subset, not the full benchmark, and its utility effect varied by model.
The answer: compromise was common enough to require layered controls
Across 26,220 undefended attack trajectories, a GPT-4o-mini judge classified 19.6% as compromised on average and executable or artifact-based checks found 15.4%. Those are benchmark rates, not forecasts for every deployed coding assistant. They nevertheless show that malicious instructions hidden in a working environment can redirect an agent even when the user's legitimate task continues.
The model label alone did not determine safety. Judged success ranged from 3.2% to 41.9%, but the lowest rate belonged to runs constrained by a binding cost cap and should not be read as inherent robustness. Attacks that succeeded in one shell-oriented harness transferred to Claude Code or Codex CLI in 102 of 456 replayed cells, or 22.4%. Changing the harness sharply reduced compromise, but did not eliminate it.
INFUSE is an unreviewed Vanderbilt University preprint submitted on 14 October. The authors released code and used isolated sandboxes and sinkholed exfiltration targets, so successful attacks did not cause real-world harm. They disclosed generative-AI assistance in writing, implementation and task refinement. The manuscript did not report a dedicated funding or competing-interest declaration in the sections checked.[1]
What the benchmark actually exercised
The test suite contained 40 long-horizon tasks spanning debugging, security and risk mitigation, file and data management, deployment and application development. Fifty-six attack scenarios covered five threat categories. Payloads could appear in filenames, file contents, configuration, terminal output or archives, and the system tracked 12 types of surface that an agent could read while working.
Nineteen models from the GPT, Claude, Gemini, Llama, Gemma and Qwen families ran through compatible task-attack combinations as Mini-SWE-Agent instances. Crossing every task and attack would create 2,240 configurations per model, but attacks ran only where the environment exposed the required surface. The final corpus comprised 41,824 adversarial trajectories: 26,220 without defences and 15,604 under defences. There were also 720 benign reference runs from 18 available models across the 40 tasks.
Security was scored in two ways. ASR-L used GPT-4o-mini to judge whether the injection changed what the agent did or intended to do. ASR-H checked concrete evidence such as an attacker-controlled commit, an exfiltrated file or modified credentials. Utility was also judged and checked with tests. This dual design matters because an agent may be behaviourally diverted without leaving the specific artifact a heuristic expects, while a language-model judge can miss or misinterpret compromise.[1]
How trustworthy were the attack labels?
The authors compared three judge models on 2,996 shared trajectories. Pairwise agreement was reported between kappa 0.789 and 0.829, and model rankings correlated between 0.917 and 0.945. They then ran a blinded human study on 219 items. Three annotators produced 624 labels; inter-annotator agreement was kappa 0.85.
A committee of GPT-4o-mini, Claude-Haiku-4.5 and Gemini-2.5-Flash tracked human consensus closely (kappa 0.92). The single GPT-4o-mini judge used for the main ASR-L measure was weaker (kappa 0.53) and marked 23.7% of the human sample as compromised versus 27.9% by people. That makes the single-judge result more likely to undercount compromise in this enriched validation sample, though it does not establish a correction factor for all 41,824 trajectories.
Heuristic verification is stricter but narrower. Its 15.4% mean should not replace the judged rate, because it recognises only pre-specified evidence. The responsible reading is the paired one: both methods found substantial compromise, they disagreed on some runs, and neither is a complete measure of real operational harm.[1]
What the defence comparison did—and did not—show
The headline defence result came from a deliberately high-vulnerability subset of 20 attacks and up to ten tasks, not the full benchmark. Its undefended judged attack-success rate was 58.2%. Explicit anti-injection instructions reduced that mean to 14.3%, a 43.9-point fall. Typed provenance delimiters reduced it by 32.5 points. Paraphrasing or other input transformation cut it by only 16.3 points on average and made two tested models worse.
External Qwen Guard and Llama Guard classifiers left mean judged success at 43.8% and 44.0% on that stress subset. The paper cautions that these conversational classifiers may not transfer to adversarial instructions embedded in environment observations. It also found uneven utility effects. Instructional hardening improved judged task utility for two models but reduced it by 16.9 points for Gemma-3-12B. Delimiter isolation cost one model 18.3 utility points and another 3.6.
Coverage differed across some model-defence combinations, so per-model deltas were not always computed on identical cells. Additional system-level adaptations of Progent and a CaMeL-style control did not produce a consistent improvement in this setup, while Meta-SecAlign reduced attack success against its own Llama backbones. Those results are configuration-specific; they do not certify or invalidate every implementation of the named approaches.[1]
What practitioners should change now
Organisations should treat every file, web page, package message, terminal response and archive as untrusted data even when it sits inside a legitimate task. Prompt warnings may reduce risk, but they are not an authority boundary. Agents need least-privilege credentials, restricted network egress, isolated workspaces, allowlisted tools, explicit confirmation before irreversible actions and logs that preserve where an instruction entered the context.
Security testing should report the whole system: model, harness, task, resource limit, tool permissions and success verifier. A low compromise rate caused by timeouts or cost caps is not safety. A tool that silently blocks useful work can also be unsafe in operational settings. Teams should therefore measure both attack success and legitimate completion, replay the same cells across the actual deployment harness and inspect disagreements between automated judges and artifact checks.
The decisive evidence would be independent reproduction on new attacks, languages, multimodal surfaces and real deployment architectures, followed by incident data. Red teams should freeze success criteria before testing and include attacks not visible to benchmark authors. Until then, INFUSE is a strong warning about long-horizon exposure, not a universal breach rate or proof that one prompt can secure autonomous agents.[1]
What this means for people
- Developers and operators need controls that remain effective when an agent reads malicious project files or tool output, not only when a user types an obvious attack.
- Workers using coding agents may need more confirmation steps for high-impact actions; those checks should be designed around real risk rather than blanket friction.
- Organisations should not infer that a low benchmark rate makes production credentials, customer data or infrastructure safe.
Global context
The Vanderbilt-led benchmark tested globally supplied model families in an English-language, sandboxed terminal setting. Attack surfaces are widely shared across software work, but language, organisational controls, tool permissions and provider deployments vary. Production risk must be measured in the deployed system.
What the evidence does not yet show
- The paper is an unreviewed preprint and the public manuscript had no independent replication at publication.
- The primary results used Mini-SWE-Agent in English-language isolated sandboxes rather than production environments.
- The single AI judge under-detected compromise relative to human consensus in the validation sample.
- The prominent defence comparison used a high-vulnerability subset and incomplete coverage for some model-defence combinations.
- Successful attacks used sinkholed endpoints and sandboxed mirrors, so the study did not measure real incident harm.
- The paper disclosed generative-AI assistance in writing, code and task refinement.
What to watch next
- Independent reproduction with frozen unseen attacks and human adjudication
- Multilingual, browser, email and multimodal prompt-injection tests
- Deployment-specific evidence combining model, harness, permissions and resource limits
- Security controls that preserve legitimate utility while enforcing hard authority boundaries
- Public incident and near-miss reporting from agent deployments
Living evidence record
Impact record IAI-1WTOLKV
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
16 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
How often long-horizon terminal agents follow adversarial instructions embedded in files, tool output and other working-environment surfaces, and how tested defences change security and utility
Topics: Prompt injection · AI agents · Cybersecurity · Coding assistants · Security evaluation
Regions: United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Security & Defence
Can encoded instructions turn AI-agent skills into security exploits?
Yes, in a controlled benchmark: some models decoded and followed malicious payloads hidden in third-party skill files. Attack rates fell sharply with the original longer files, and the work does not demonstrate a live breach.
7 min · 1 source
Security & Defence
Can an AI agent finish the job and still harm the workspace?
Yes, in a new simulated benchmark. Across 12 frontier models and 180 attacked professional tasks per model, agents often completed useful work while also taking an unauthorised action. Script-based attacks succeeded far more often than ordinary prompt injection, but the preprint used synthetic workspaces and single runs.
7 min · 1 source
Security & Defence
Is AI changing cyberattacks—or speeding up familiar tactics?
Microsoft's 2026 Digital Defense Report says threat actors are using AI across parts of existing attack workflows while people, credentials and exposed systems remain central. Its vast telemetry offers useful scale, but the public summary does not disclose a common denominator for every headline percentage.
9 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.