How much of an AI agent’s output is actually waste—and can it be removed safely?
An unreviewed study marked about 18% of audited output as removable and pruned 41–46% of reasoning in two online tests. One benchmark lost 2.7 percentage points of task success, and an LLM judge defines what counts as waste.
The short version
- What happened
- Across collected trajectories from six models and four benchmarks, the authors' LLM judge marked roughly 18% of output tokens as waste; the definition is content removable without eliminating useful work.
- Who it affects
- Teams paying for agent workflows may save more by fixing retry loops, environment mismatches and repeated retrieval than by choosing a model on aggregate token price alone.
- What remains unproven
- The paper is an unreviewed preprint and uses an LLM judge to define and locate removable spans.
Keep in mind
The paper is an unreviewed preprint and uses an LLM judge to define and locate removable spans.
Online pruning was validated on two benchmarks with Kimi-K3; MCP-Atlas success fell from 65.0% to 62.3%.
Complete reasoning analysis does not generalise directly to providers that expose summaries or no reasoning trace.
Benchmarks do not capture every production agent, tool, language, permission model or cost component.
Removing 41–46% of reasoning does not imply the same reduction in total tokens, money, latency or energy.
Source analysis · 1 linked source. Primary analysed source date: 4 November 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The direct answer: some generated text was removable, but the headline percentage is not a universal cost saving
An unreviewed preprint reports that a measurable share of AI-agent output can be removed without destroying the work needed to complete benchmark tasks. Across collected trajectories, a language-model judge marked roughly 18% of output tokens as waste. In two online tests using Kimi-K3, pruning deleted 41% of reasoning on SWE-bench Verified and 46% on MCP-Atlas while task success changed from 89.0% to 89.5% and from 65.0% to 62.3%, respectively.
Those results support a practical idea: teams should inspect what an agent does, not only how many tokens it consumes. They do not establish that almost half of every production bill is avoidable. The percentages apply to generated reasoning in two benchmark settings, not cached input, tool execution, latency, energy or total monetary spend. The MCP-Atlas result also shows a measurable performance loss, and the framework depends on judgments about which spans were unnecessary.[1]
How the study defined waste
The authors define waste as content that can be removed without eliminating useful work. They separate four patterns. ‘Restate’ repeats information already present. ‘Duplicate-fetch’ calls a tool again for unchanged information. ‘Padding’ adds unnecessary preambles or narration. ‘Detour’ describes exploration that was plausible at the time but unnecessary in hindsight. The first three are treated as strict waste; detours are separate because exploration may be rational before the answer is known.
A language-model judge receives the task and full trajectory, marks candidate spans and assigns a category. The framework maps those spans back to provider token counts and estimates cost. This is more specific than labelling every long response inefficient, but it still encodes the judge's interpretation. A repeated statement might support error checking, and a failed route may be valuable in an unfamiliar environment. The authors therefore validate candidate waste by intervening on execution rather than relying on labels alone.[1]
The distinction that matters
From token count to tested waste
The framework only calls text waste after marking it and checking what happens when it is removed.
Trace
Record reasoning, responses, tools and tokens
Common trajectory format
Mark
LLM judge labels removable spans
Restate, duplicate-fetch, padding or detour
Intervene
Delete spans and resume execution
Compare actions and task success
Deploy
Validate in the real workflow
Benchmark savings do not transfer automatically
The intervention tests are the study's strongest evidence
For offline interventions, the team removed marked strict-waste spans from earlier Kimi-K3 reasoning and resumed the run. On SWE-bench Verified, median removals of 350, 649 and 813 tokens at three intervention points left aggregate solve rates unchanged, although individual tasks sometimes moved in both directions. On MCP-Atlas, removing 111 tokens after the first turn reduced success from 75.3% to 65.6%; later interventions left aggregate performance similar or slightly higher. That exception matters because early reasoning can influence every later action.
The next-action experiment sampled 16 continuations from original and pruned contexts and used 2,000 permutations per comparison. Removing marked waste changed actions in 3.0% of SWE-bench turns and 10.5% of MCP-Atlas turns, compared with 25.0% and 37.9% when all previous reasoning was removed. The effect was limited, not zero. This makes the evidence stronger than retrospective annotation while leaving uncertainty about tasks beyond the two validation benchmarks.[1]
The study compared six models, four benchmarks and five harnesses
The wider analysis included Kimi-K3, Gemini 3.1 Pro, GPT-5.6 Terra, GPT-6 Astra, Claude Sonnet 5 and Claude Fable 5.1. Benchmarks covered airline and retail customer service in tau-squared bench, software engineering in SWE-bench Verified, general tool use in MCP-Atlas and Claw-Eval. Because providers expose different information, visible responses and tool calls could be compared across models, while complete reasoning analysis was limited to Kimi-K3.
The authors also held the model constant and compared OpenCode, OpenClaw, OpenHands, Hermes and Kimi Code CLI. Waste varied more across tasks than models, and detailed action patterns changed with the harness. This matters operationally. A team switching models without inspecting prompts, tools, retry loops and environment configuration may leave the largest source of avoidable work untouched.[1]
Some waste came from ordinary software integration failures
Several harnesses launched SWE-bench tasks in the wrong Python environment, producing import failures and repeated retries. Kimi Code CLI exposed a timeout value rejected by its own runtime, forcing repeated calls. When internet access was removed, some agents kept attempting installations or alternative network routes even though the environment could not satisfy them. These are not mysterious capability limits; they are interface and control-loop problems.
For people running agents at work, this changes the optimisation order. First confirm that tools, paths, permissions and timeouts match the environment. Then instrument repeated reads, retries and unavailable capabilities. A smaller model or strict token cap may lower spend, but it can also remove useful work while preserving a bad loop. The most transferable lesson is to connect cost data to accepted task outcomes and trace-level causes.[1]
More reasoning did not map neatly to more waste
In a controlled Kimi-K3 comparison, maximum reasoning generated 6.6 times as many reasoning tokens per trajectory as low effort, yet the judge still marked an average 66% of that reasoning as non-wasteful. Moving from low to high added 0.99 million tokens and increased success from 69.7% to 78.4%. Moving from high to maximum added another 0.80 million tokens without a measurable overall gain. Among tasks solved by both, maximum effort cost 1.62 times as much.
That pattern warns against two simple stories. Longer reasoning is not automatically useful, but extra reasoning is not automatically removable. It can alter tools, verification and error recovery. Budget controls should therefore be evaluated task by task, using the same acceptance criteria and repeated trials. A single average token number cannot show whether a run was thorough, stuck or merely verbose.[1]
What would change the assessment
Confidence would rise with independent replication, preregistered definitions, human comparison labels and production trials reporting total cost, latency, energy, tool charges and accepted outcomes. Tests should include multilingual tasks, long-lived agents, safety-critical workflows and environments where exploration has value. Teams should compare pruning with simpler fixes such as correcting tool configuration, caching unchanged results and stopping impossible runs.
The assessment would weaken if another judge marks different spans, if pruning fails on new tasks or if cheaper trajectories create more human review and correction. For now, the paper offers a useful diagnostic framework and unusually direct intervention evidence. It does not prove that hidden reasoning is understood, that marked text has no value in every rerun or that a benchmark token reduction becomes an equal reduction in a real organisation's bill.[1]
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Continue the story
Read next
Continue exploring ai agents: compare the finding and its evidence limits.
Can real tool traces train better AI agents? ToolSpace raises benchmark scores, not proof of safe work
The unreviewed preprint turns observed tool-call dependencies into replayable training tasks. A 35-billion-parameter model rose from 23.0 to 41.0 across seven benchmarks, but the evidence remains synthetic and benchmark-bound.
Read the next reportExplore the full source and evidence record
What this means for people
- Teams paying for agent workflows may save more by fixing retry loops, environment mismatches and repeated retrieval than by choosing a model on aggregate token price alone.
- Workers reviewing agent output need accepted-task and correction measures alongside token counts, because a cheaper trajectory can shift cost into human rework.
- Developers should treat the 41–46% pruning result as a hypothesis to test on their own workflows, not permission to delete nearly half of every reasoning trace.
Global context
Agent costs matter wherever organisations pay for repeated model calls, tool execution and human review. The evaluated models and harnesses come from several providers, but the tasks are benchmarks and full-reasoning validation depends on one exposed-trace model. Organisations also face different data-residency, audit and labour requirements. A trace may contain confidential material that should not be sent to a separate judge, so waste auditing needs privacy controls and local evaluation as well as efficiency metrics.
What the evidence does not yet show
- The paper is an unreviewed preprint and uses an LLM judge to define and locate removable spans.
- Online pruning was validated on two benchmarks with Kimi-K3; MCP-Atlas success fell from 65.0% to 62.3%.
- Complete reasoning analysis does not generalise directly to providers that expose summaries or no reasoning trace.
- Benchmarks do not capture every production agent, tool, language, permission model or cost component.
- Removing 41–46% of reasoning does not imply the same reduction in total tokens, money, latency or energy.
What to watch next
- Independent replication with human annotations and alternative judge models.
- Production measurements of total cost, latency, energy and human correction time.
- Whether pruning remains safe on multilingual, regulated and long-running tasks.
- Harness fixes for invalid timeouts, wrong environments, repeated reads and unavailable capabilities.
- Comparisons between trace pruning, caching, early stopping and simpler tool-interface improvements.
Living evidence record
Impact record IAI-1SE0BDA
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
6 November 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 6 November 2026
Research question, topics and regions
Whether removable spans in AI-agent trajectories can be identified and deleted without changing task outcomes, and how waste differs across models, tasks, reasoning settings and harnesses
Topics: AI agents · Token cost · Agent harnesses · Reasoning · Benchmarks · Preprint
Regions: United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.