Can an AI agent reliably improve the software that controls it?
Across 27 autonomous harness lineages and 2,025 game evaluations, two rounds of self-revision produced only a small, uncertain average gain on held-out tasks. The unreviewed result separates impressive software construction from dependable self-improvement.
The short version
- What happened
- Nine model-and-effort conditions each produced three independent harness lineages. Every initial harness and two revisions were frozen and tested on five development games and twenty held-out games.
- Who it affects
- Workers and public-service teams using autonomous agents need version-by-version acceptance tests. An agent that appears to repair its own workflow can become cheaper by doing less, or improve familiar tasks while regressing elsewhere.
- What remains unproven
- The paper is an unreviewed preprint and has not been independently replicated.
Keep in mind
The paper is an unreviewed preprint and has not been independently replicated.
ARC-AGI-3 is an interactive game benchmark, not a workplace, safety-critical service or open-ended deployment.
Only two revision rounds and a fixed set of model-and-effort conditions were tested; longer adaptation chains may differ.
One author lists Amazon and university affiliations while stating that the work was done outside Amazon; no separate funding or competing-interest declaration is visible in the accessible paper.
Source analysis · 1 linked source. Primary analysed source date: 21 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
The agents changed their own control software
Researchers gave frontier language models a minimal repository and asked each to build an executable harness for ARC-AGI-3, a set of unfamiliar interactive grid games. A harness decides what the model sees, remembers and calls, how it recovers from errors and how much inference it spends. After playing five development games, the same model could revise the entire codebase twice without human edits between stages.
The protocol covered nine model-and-reasoning-effort conditions, with three independent constructions per condition: 27 lineages, 81 frozen snapshots and 2,025 game evaluations. Every snapshot was scored on twenty games that never informed construction or revision. Performance used Relative Human Action Efficiency, which combines completed levels with action efficiency against hidden human baselines; inference cost was recorded separately.[1]
Improvement on familiar feedback did not transfer reliably
After two revisions, mean performance rose 8.42 points on the five development games but only 2.80 points on the held-out set. The held-out 95% bootstrap interval ran from -1.44 to 7.35 points, so the pooled result is compatible with no average benefit. Fifteen lineages improved and twelve declined. Across games, mean performance increased on thirteen and fell on seven.
Cost also exposed a trade-off. Nine final harnesses improved while becoming cheaper, six improved at higher cost, nine became cheaper while losing performance and three worsened on both measures. The study estimates at least $18,129 of model inference under its frozen price schedule, but it does not show that the same revisions would behave consistently under changing models or private benchmark tasks.[1]
The distinction that matters
The study separates building from improving
Each lineage followed the same controlled sequence, with held-out games kept away from the revision process.
Build
Create a complete harness from a minimal repository
Shows autonomous software construction
Revise
Use results from five development games twice
Produces two frozen descendants
Verify
Evaluate every version on twenty unseen games
Reveals small, uncertain average transfer
Construction skill is not yet dependable self-improvement
The practical lesson for organisations deploying agents is to freeze each version and test it on tasks that did not guide the revision. Passing unit tests or improving a development set is not enough when a rewritten memory system, planner or cost control can quietly remove capabilities elsewhere. Rollback criteria and independent acceptance tests remain necessary even when the agent wrote the change itself.
This is an unreviewed benchmark study, not evidence that a deployed system is escaping control or recursively increasing its intelligence. Its game split, two-revision limit and model pool bound the conclusion. Prospective replication on workplace tasks, longer revision chains and independently maintained tests would materially change the assessment.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 2 min 31 sec · Background explainer
What is artificial intelligence? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Published 2018-11-28. Source checked 14 October 2026. Check captions on the publisher’s video page.
Explore current AI technologyPhoto: ThisisEngineering / Unsplash · Licence
Continue the story
Read next
Continue exploring ai agents: compare the finding and its evidence limits.
Should an AI agent turn one failure into a rule? A registered experiment found chance can stick
Across 90,084 synthetic answers, a lesson written after a random failure sometimes pushed a fresh model session towards the less reliable tool. A companion preprint found that rewritten instructions also need to match the model expected to use them. Neither study establishes production harm or safe autonomous self-improvement.
Read the next reportExplore the full source and evidence record
What this means for people
- Workers and public-service teams using autonomous agents need version-by-version acceptance tests. An agent that appears to repair its own workflow can become cheaper by doing less, or improve familiar tasks while regressing elsewhere.
Global context
The tested models come from more than one provider, but the benchmark and orchestration stack are controlled research artefacts. The reliability requirement—evaluate changes on evidence the updater never saw—travels across vendors and jurisdictions.
What the evidence does not yet show
- The paper is an unreviewed preprint and has not been independently replicated.
- ARC-AGI-3 is an interactive game benchmark, not a workplace, safety-critical service or open-ended deployment.
- Only two revision rounds and a fixed set of model-and-effort conditions were tested; longer adaptation chains may differ.
- One author lists Amazon and university affiliations while stating that the work was done outside Amazon; no separate funding or competing-interest declaration is visible in the accessible paper.
What to watch next
- Independent replication on private held-out tasks
- Longer revision chains with preserved rollback points
- Prospective comparison with human-reviewed software changes
- Whether cost savings preserve task coverage and safety checks
Living evidence record
Impact record IAI-0WBB3TB
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
24 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 24 October 2026
Research question, topics and regions
Whether frontier language models can construct and then reliably improve the executable software harness through which they act
Topics: AI agents · Self-improvement · Agent harnesses · Held-out evaluation · AI safety
Regions: Germany · United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.