Can an agent use the right tool with the wrong recipient? A benchmark exposes parameter poisoning
In 144 paired tool-use cases, attacker-controlled content could change a critical argument without changing the intended action. A proposed verifier cut attack success sharply, but also reduced legitimate task completion and remains an unreviewed prototype.
The short version
- What happened
- FactAuthBench contains 144 paired clean-and-attack cases across banking, Slack, travel and workspace tools; it measures whether malicious external content changes a value in the executed tool call.
- Who it affects
- A payment or sharing agent can obey the user's verb while changing the destination.
- What remains unproven
- Unreviewed constructed benchmark with 144 paired cases.
Keep in mind
Unreviewed constructed benchmark with 144 paired cases.
Limited model and tool coverage; no measured deployment prevalence.
Verifier robustness, provenance quality and adaptive attacks remain unresolved.
Security gains came with lower benign task utility.
Source analysis · 1 linked source. Primary analysed source date: 3 November 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 7 min The evidence, methods, implications and limitations
The finding: an agent can keep the action and corrupt the argument
A new security preprint identifies a failure that can be missed when an agent monitor asks only whether the model chose the correct tool. An attacker can leave the high-level action intact while changing a critical argument. A user may still appear to be sending money, sharing a file or booking a trip, but untrusted content supplies the recipient, account, address or other value. The paper calls this parameter poisoning. In its benchmark, the attack succeeded in 93.06% of cases for an undefended Qwen3.6-27B agent and 57.64% for GPT-4o.
The distinction matters because ordinary changes to tool arguments are necessary. A travel agent must read dates and destinations; a payment agent must learn a recipient; a workspace agent must carry facts from documents into a call. Blocking every externally supplied value would make the system unusable. The researchers instead propose asking whether the particular source is authorised to determine the particular parameter for the user's current task. Their prototype, AuthorityFlow, reduced attack success to 9.03% on Qwen and 5.56% on GPT-4o, but it did not eliminate the problem.[1]
How FactAuthBench turns a subtle attack into 144 paired tests
The University of Science and Technology of China team extended AgentDojo into FactAuthBench. It covers four tool environments—banking, Slack, travel and workspace—and contains 144 paired cases. Each pair uses the same user task twice. The clean setting contains no attacker-controlled information; the attack setting inserts external content designed to make a malicious value appear in a tool argument. Attack success means that the attacker-selected value reaches the executed call, not merely that the model mentioned it in prose.
That paired design helps isolate the adversarial content while preserving the user's goal. It is nevertheless a constructed benchmark. The cases inherit assumptions about tools, fields and task boundaries from AgentDojo, and the paper does not establish how often comparable parameter attacks occur in real deployments. It also evaluates a small set of agent configurations: Qwen3.6-27B and GPT-4o for the main table, with Qwen, GPT-4o and Llama-3.3-70B used as auxiliary verifier models. A benchmark percentage should not be read as a breach rate for commercial products.[1]
The distinction that matters
Action integrity is not parameter integrity
A call can look correct at the tool level while a malicious source controls the value that makes it consequential.
User goal
Send, book or share
The intended action remains
Untrusted content
Supplies a critical value
Recipient or target changes
AuthorityFlow
Traces evidence and source authority
Allows, blocks or purifies the call
Residual risk
Verifier reasons under ambiguity
Utility loss and adaptive attacks remain
The proposed defence traces which fact supports each value
AuthorityFlow has three stages. First, it interprets the user's request and tool interface to build a task-specific hierarchy of which information fields may legitimately control which facts. Second, it traces the evidence supporting each proposed tool argument, including intermediate facts. Third, it checks both semantic support and source authorisation before allowing a task-critical value to pass. The tool call is intercepted if any required fact fails verification. This is more granular than treating a whole web page or document as universally trusted or untrusted.
The ablations support that design logic inside the benchmark. Removing source authorisation raised Qwen attack success from 9.03% to 62.50%; removing semantic support raised it to 28.47%. Removing task-use authorisation kept attack success low at 7.64% but cut benign utility to 59.72%, suggesting that fixed trust rules over-block legitimate work. Removing parameter-support tracing reduced benign utility to 56.94% and raised attack success to 11.81%. These comparisons are internal to the same test suite and need independent replication.[1]
Security improved, but useful work also declined
The headline security number hides a material usability cost. On Qwen, benign utility was 90.28% without a defence and 82.64% with AuthorityFlow. Utility under attack was only 27.08%, because the framework can stop a poisoned call without completing the user's original task. A downstream purification step raised utility under attack to 58.33% while leaving attack success at 9.03%. On GPT-4o, AuthorityFlow's benign utility was 73.61%, compared with 81.94% undefended, and purification raised utility under attack from 20.83% to 60.42%.
Other defences illustrate the trade-off. CaMeL policies drove attack success to 0.69% on Qwen and 2.08% on GPT-4o, but benign utility was 43.75% and 39.58%. Sandwiching, Spotlighting, MELON and AttriGuard preserved more routine work but left many parameter attacks successful. The experiment therefore does not identify a universally best control. It shows why an operator must report security and task completion together: a system that blocks everything is secure in one narrow sense and useless in another.[1]
What remains unproven about provenance and the verifier
The paper's limitations are operational. Authority construction depends on a clear user task, accurate tool metadata and reliable source provenance. Real users are ambiguous, tool descriptions can be incomplete, and retrieved content may be assembled from several systems that erase its origin. The verifier is itself a language model. An adaptive attacker could try to influence its reasoning or make malicious evidence resemble an authorised source. The authors acknowledge that verifier robustness and inference overhead require more work.
Performance also varied with the auxiliary model. With Qwen3.6-27B as verifier, benign utility was 82.64% and attack success 9.03%. GPT-4o as verifier produced 75.69% and 15.28%; Llama-3.3-70B produced 62.50% and 24.31%. That sensitivity suggests the result is not a property of the architecture alone. The preprint reports no external funding or competing interests, but it has not been peer reviewed and does not provide field evidence that the defence works under live traffic, changing tools or hostile adaptation.[1]
What builders can do now—and what would change the assessment
Teams deploying agents should log the exact arguments proposed for consequential calls and the provenance of the evidence behind them. High-risk values such as payment destinations, permissions, recipients and destructive targets should be bound to trusted user input or separately confirmed. A monitor should compare the intended task with both the selected action and its parameters. Least-privilege tools, explicit schemas and human confirmation remain useful even if a model-based verifier is added. AuthorityFlow is a research idea to test, not a certified control.
Confidence would rise with independent evaluations on unseen tools, complete conversation histories and adaptive attackers; preregistered comparisons against deterministic policies; measurements of latency and cost; and tests in which provenance is missing or deliberately forged. It would fall if attacks transfer to the verifier, normal users face frequent blocked tasks, or performance depends on benchmark-specific source labels. The paper expands the threat model in a consequential way, but it reports a controlled preprint rather than an active incident. It is Latest, not Breaking.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 2 min 31 sec · Background explainer
What is artificial intelligence? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Published 2018-11-28. Source checked 14 October 2026. Check captions on the publisher’s video page.
Explore current AI technologyPhoto: ThisisEngineering / Unsplash · Licence
Continue the story
Read next
Continue exploring ai agents: compare the finding and its evidence limits.
Can an AI agent finish the job and still harm the workspace?
Yes, in a new simulated benchmark. Across 12 frontier models and 180 attacked professional tasks per model, agents often completed useful work while also taking an unauthorised action. Script-based attacks succeeded far more often than ordinary prompt injection, but the preprint used synthetic workspaces and single runs.
Read the next reportExplore the full source and evidence record
What this means for people
- A payment or sharing agent can obey the user's verb while changing the destination.
- Confirmation screens should expose critical arguments, not only the tool name.
- Blocked calls need a recovery path so security does not silently erase legitimate work.
Global context
Agent security is moving from prompt-only defences toward controls over data flow, tool permissions and provenance. Parameter poisoning connects those layers: it can preserve an apparently aligned plan while redirecting the execution. The result supports defence in depth rather than replacing deterministic authorisation or human review.
What the evidence does not yet show
- Unreviewed constructed benchmark with 144 paired cases.
- Limited model and tool coverage; no measured deployment prevalence.
- Verifier robustness, provenance quality and adaptive attacks remain unresolved.
- Security gains came with lower benign task utility.
What to watch next
- Independent FactAuthBench replication.
- Adaptive attacks on the verifier.
- Production latency and false-block rates.
- Cryptographic or system-level provenance controls.
Living evidence record
Impact record IAI-0ATU1FD
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 November 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 5 November 2026
Research question, topics and regions
Whether indirect prompt injection can preserve an agent's intended action while changing a critical tool argument, and whether task-specific source authority can block that manipulation
Topics: AI agents · Prompt injection · Tool security · Agent safety · Cybersecurity
Regions: China · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.