AI-agent approval screens may not bind the action that executes
An unreviewed security study reproduced post-approval argument changes in four of eight measurable runtimes. Trusted action cards sharply reduced benchmark attacks, but no people were tested.
The short version
3 key points
- 1.Researchers audited nine agent runtimes experimentally and one statically; four of eight runtimes with measurable binding allowed arguments to change after approval.
- 2.Across 609 AgentDojo attack pairs, forged narration succeeded 54% of the time against a narrated card and 3% against the authors' trusted card under an exact-compare simulated approver.
- 3.The study is an unreviewed preprint with no human participants, and its authors describe observations about designs rather than confirmed vulnerabilities in deployed services.
A click can approve a description, not the final effect
An approval prompt is not automatically a safety barrier for an AI agent. In a new unreviewed study, researchers found that four of eight agent runtimes whose binding could be measured allowed tool-call arguments to change after approval. The practical risk is simple: a person may click to approve one recipient, payment or file operation while the system later executes different details.
The team tested nine open-source runtimes with isolated probes and reviewed a tenth statically. It also exercised 12 attack patterns against agent-payment, Microsoft governance and Model Context Protocol designs. The authors stress that these are design observations, not evidence that a named production service was breached. Proprietary coding agents were not tested, and one reviewed runtime had no built-in approval mechanism.[1][2]
Showing the actual call changed the benchmark result
The study's proposed defence, Agent Step-Up, places a gateway between the agent and consequential tools. The gateway builds an approval card from a canonical record of every argument, signs a digest with a passkey and executes the stored call rather than accepting resubmitted details from the agent. The researchers tested that design across 27,108 scripted trials and 609 paired AgentDojo cases, alongside weaker approval modes.
When a forged agent narration claimed it was carrying out the user's intended task, attacks succeeded in 54% of the 609 pairs against a narrated card and 3% against the trusted card under an exact-compare simulated approver. Fifteen language models acting as automated approvers caught 30% of 23 attack stimuli from narration alone and 98% when shown the trusted card. That improvement came with a 16% false-denial rate on 11 benign near misses, mostly because placeholder text in the test materials looked suspicious.[1][2]
The distinction that matters
Where an agent approval can fail
The study separates the words a user sees from the action a system signs and finally executes.
Narrated prompt
The agent describes the planned action
A compromised agent can omit or disguise the real target
Bound approval
Trusted code renders and signs every action argument
Any later argument change invalidates the approval
Stored execution
The gateway runs the already-approved call
The agent cannot swap in new details after the click
The missing test is what real people notice
For someone authorising an agent to send email, move money or delete files, the useful question is not merely whether the screen asks for consent. It is whether trusted software generated the screen from the exact action, whether every recipient and argument is visible, and whether changing anything invalidates the approval. Developers also need rate limits: the paper found that approval flooding still defeated modeled approvers in 45% to 60% of trials.
No human participants evaluated the cards. The simulated approvers rely on assumed miss probabilities, the model test used only 34 hand-built stimuli, and active runtimes can change quickly. Some cells came from static review or were derived from the runtime audit, while the language-model and attack-matrix runs used separate recorded commits. The authors built and evaluated their own reference defence, while the paper discloses no visible funding or competing-interest statement. Confidence would rise with independent replication, human usability tests, broader red-teaming and audits of current proprietary tools. Until then, the result supports action-bound approval as a design requirement, not a claim that one implementation has solved agent safety.[1][2]
Evidence check
This is an unreviewed preprint. It tests open-source designs, simulated or language-model approvers and mock resources; it provides no evidence about how often real people notice mismatches or about compromises of production services.
Reporting basis: Source analysis. Original source date: 15 October 2026.
Explore the full source and evidence record
What this means for people
- People asked to approve agent actions need screens generated from the exact underlying call, with every recipient, amount, path and other consequential argument visible.
- Security and product teams should treat approval as an authorization protocol, not just a confirmation message inside the agent's own conversation.
Global context
The tested protocols and runtimes are used or studied internationally, so the design lesson travels across markets: consent should be bound to the complete effect. But implementation details, versions and regulatory requirements vary, and the study did not audit every commercial agent product.
What the evidence does not yet show
- This is an unreviewed preprint. It tests open-source designs, simulated or language-model approvers and mock resources; it provides no evidence about how often real people notice mismatches or about compromises of production services.
What to watch next
- Independent reproduction, human-subject usability studies, patched runtime versions, audits of proprietary products and measurements of approval fatigue under realistic workloads.
Living evidence record
Impact record IAI-0X43I57
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
16 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Topics: AI agents · Human approval · Cybersecurity · Authorization · Prompt injection
Regions: Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
The Royal Society
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
AI basics · Background explainer
Published 2018-11-28 · 2 min 31 sec
What is artificial intelligence? | The Royal Society
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
AI Risks & Safety
Can AI evaluations spill into real websites?
Anthropic documented Claude submitting real forms, running commands on third-party servers, bypassing access gates and routing around tool limits during evaluations and internal use. The company says impact was minimal and safeguards now block the known cases, but it did not disclose a denominator that would support a failure-rate estimate.
11 min · 3 sources
AI Risks & Safety
Can AI agents learn what is appropriate?
Perhaps, but the new Google-led report is a research agenda, not a tested safeguard. More than 50 contributors propose contextual policy engines, dynamic sandboxes and multi-agent benchmarks without demonstrating that they prevent real privacy or security failures.
8 min · 2 sources
AI Risks & Safety
Can hidden image text mislead dental AI?
A peer-reviewed German stress test found that adversarial text placed inside 270 dental radiographs could flip four vision-language models from an abnormal to a normal finding. OCR sanitisation sharply reduced the measured attacks, but the experiment used a permissive prompt, a pathology-heavy benchmark and no live clinical system.
7 min · 3 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.