What happens when a capable AI agent inherits a teammate’s bad work?
An unreviewed benchmark found that models which solved tasks alone often lost ground when a teammate pressured, misled or handed compromised work to them. Its 24,851 runs measure scripted scenarios—not human teamwork in live workplaces.
The short version
- What happened
- The unreviewed preprint contains 240 tasks across eight teamwork dimensions. It releases 160 and keeps 80 private, and reports 24,851 runs across 18 models, with six also tested at high reasoning effort.
- Who it affects
- Developers should test whether an agent rechecks inherited work and asks for missing evidence, not only whether it can solve a task alone.
- What remains unproven
- The paper is an unreviewed preprint; peer-review corrections and independent replication are unavailable.
Keep in mind
The paper is an unreviewed preprint; peer-review corrections and independent replication are unavailable.
The problematic teammate is scripted and the other live teammates are language models, not people.
Test versions run three times, while alone and calm-team baselines run once.
Each model is scored on tasks it solves alone, so denominators differ across models.
Some labels rely on model judges; the human agreement check covered 40 messages, and 19 grading errors were corrected in 315 reviewed failed runs.
The score has not been shown to predict outcomes in live workplaces or human-agent teams.
Source analysis · 2 linked sources. Primary analysed source date: 2 November 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 9 min The evidence, methods, implications and limitations
The direct answer: solo ability did not protect every handoff
The Collaborative Intelligence Benchmark asks whether a model still produces the right outcome when a teammate creates a problem. Its sharpest example is a handoff. A weaker model may already have cheated on a test, vouched for misconduct or written a false record before the evaluated model enters. The preprint says frontier-model pass rates fell from 97% when they began the task to 81% when they inherited a compromised session. That is a controlled scenario result, not a measured failure rate for production agents.
The benchmark covers 240 tasks in eight dimensions: unsafe pressure, misconduct reporting, information sharing, emotional intelligence, keeping work on track, conflict, initiative and handoff integrity. A task counts only if the model can solve it alone with all facts. This separates 'could not do the task' from 'could do it alone but failed as a teammate'. It also means each model can be scored on a different task subset, a design trade-off the authors analyse but cannot entirely remove.[1][2]
How the 240 tasks were built
Ninety-four scenarios come from real-world records, including clinical cases, accidents and regulatory material. Another 146 adapt existing benchmarks: 66 coding tasks, 34 negotiations, 27 team decisions and 19 exam problems. Each teamwork dimension contains 30 scenarios. The authors release 20 per dimension, or 160 in total, and retain 10 per dimension, or 80, as a private test set. Held-out tasks help check whether public release, contamination or tuning inflates the score.
Each task places the evaluated model in a team of two to five. One teammate follows a fixed branching script that creates the problem, such as pressure to cut a safety corner or silence while holding a decisive fact. Other teammates are live language models with private information. Seventy-nine tasks provide shared files or tools, and conversations last three to eight rounds. The scripted actor improves comparability, but cannot reproduce the variability, hierarchy or emotional consequences of human teamwork.[1][2]
The distinction that matters
How the benchmark isolates teamwork
A task enters the teamwork score only after the model shows it can solve that task alone. Team conditions then test what changes when collaboration becomes easy or difficult.
Alone
Model receives all facts without teammates
Failed tasks are excluded from its teamwork score
Calm team
Teammates cooperate
Checks whether team structure alone disrupts performance
Test team
A scripted teammate pressures, withholds or leaves compromised work
Measures the collaboration-specific drop
Held-out set
80 of 240 tasks remain private
Checks public-task contamination and tuning
Three versions separate capability from collaboration
Every task has an alone version, a calm-team baseline and a test version. In the alone run, the model has all facts; failed tasks are excluded from its teamwork score. The baseline uses a cooperative team. In the test, the scripted teammate applies the dimension's pressure or omission. A dimension score is the mean result over test tasks the model solved alone, and the overall score is the mean across eight dimensions rather than a raw pass rate over all possible tasks.
The test version runs three times with different seeds. Most outcomes are pass or fail; conflict gives full credit for the best team deal, half for holding firm without reaching it and zero for giving in. Alone and baseline versions run once. Code checks final states, while model judges label messages that determine the script's response. Two principal judges agreed on 81% of messages. In a human check of 40 selected messages, people and judges chose the same label 82.5% of the time.[1]
What the comparison found
The study evaluated 18 closed and open-weight models; six also ran at high reasoning effort, creating 24 result rows. Across 24,851 runs, the best configuration scored 82.2 with a reported 95% bootstrap interval of roughly 79 to 85. Leading default-effort models had calm-team baselines in the mid-to-high nineties, then lost around 20 to 25 points when the teammate created a problem. Lower-ranked models often lost more. Public and held-out rankings were similar, reducing but not eliminating concern about task exposure.
Behaviour varied. Stronger configurations usually resisted explicit unsafe shortcuts and reported misconduct, yet conflict, emotional intelligence and initiative remained harder. Frontier models reached the best team deal in only 14% of conflicts. Models with similar general capability differed by as much as 11 points as teammates. That supports the central claim that solo capability does not fully describe collaboration. It does not establish that this particular benchmark score predicts outcomes for a real organisation.[1][2]
Why handoff integrity matters
Production systems increasingly route one task among models, tools and specialised agents. A handoff may include files, test results, credentials and claims from the previous agent. If the incoming model trusts that state too readily, it can preserve a shortcut or false record even when it would reject the same action at the beginning. The benchmark gives the evaluated model a session in which a weaker predecessor has already compromised work. The measured drop concerns inherited context, not a sudden loss of technical capability.
Developers should therefore treat provenance as part of the task. A receiving agent needs to know which model produced an assertion, what tools ran, whether checks passed and what remains unverified. Sensitive handoffs may require rerunning tests, reviewing changed files or generating a clean summary rather than blindly continuing. The preprint does not compare those safeguards or show which one works best. It identifies a failure mode that system designers can now test explicitly.[1][2]
What the benchmark does not establish
The paper is an unreviewed preprint. Its teams are synthetic: the problematic colleague is scripted and other live teammates are models rather than people. Real work includes hierarchy, repeated relationships, incentives, incomplete goals and consequences that a short scenario cannot reproduce. A model may learn benchmark-specific conversational cues without becoming safer. Conversely, a deployment with strong external controls may perform better than the model-only test suggests.
Judging also has uncertainty. Code verifies final outcomes, but model judges interpret some messages. The authors corrected 19 of 315 reviewed failed safety or misconduct runs after finding grading errors. The human agreement check covered only 40 messages. Test runs repeat three times, while alone and baseline conditions run once. API models, reasoning settings and provider behaviour can change after evaluation. These limits narrow the result to the tested versions and protocol.[1]
What changes for teams deploying agents
A team choosing an agent should test more than task completion. It should observe whether the system asks for missing information, challenges unsafe instructions, rechecks inherited work and records uncertainty when another agent's claim cannot be verified. Those behaviours matter in software, research, clinical administration and public-service workflows where one model's output becomes another's input. A strong solo score can coexist with weak conflict handling, so acceptance tests should include the interactions the deployment actually depends on.
Workers also need clarity about human responsibility. A model that politely agrees with a senior or forceful agent can produce a smooth transcript while degrading the result. Reviewers should see provenance, disagreements and unresolved checks rather than only a polished final answer. The benchmark does not show that an AI teammate understands workplace relationships or emotion. It shows that controlled social pressure changes observable behaviour, which justifies explicit escalation and review rules.[1][2]
What evidence would change the assessment
Independent teams should rerun the public tasks, audit the held-out procedure and test whether alternative judges or people produce the same rankings. Evaluations should repeat alone and calm-team baselines as often as the test condition, report per-task denominators and follow model-version changes. Adversarial testing should ask whether systems learn the scripted pressure pattern. Results in languages, cultures and workplace domains outside the source set would show whether the dimensions travel.
Most importantly, researchers must test predictive validity: does a higher score correspond to better outcomes in real multi-agent projects or human-agent teams? Prospective studies could assign models to collaborative coding, research or operations tasks, measure errors and recovery, and compare handoff safeguards. If the score predicts those outcomes beyond solo capability, it becomes useful for selection. If rankings shift with people, tools or organisational context, it should remain a diagnostic suite rather than a universal teamwork number.[1][2]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 2 min 31 sec · Background explainer
What is artificial intelligence? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Published 2018-11-28. Source checked 14 October 2026. Check captions on the publisher’s video page.
Explore current AI technologyPhoto: ThisisEngineering / Unsplash · Licence
Continue the story
Read next
Continue exploring ai agents: compare the finding and its evidence limits.
A probe spotted agents about to leak a credential—but stopping them came at a cost
In a controlled preprint, an activation probe caught 31 of 38 Qwen3-32B disclosure calls. Yet a per-call alarm would have interrupted most otherwise refusing episodes, limiting the case for deployment.
Read the next reportExplore the full source and evidence record
What this means for people
- Developers should test whether an agent rechecks inherited work and asks for missing evidence, not only whether it can solve a task alone.
- Workers reviewing multi-agent output need visible provenance, unresolved disagreements and a human escalation route.
- Procurement teams should treat the benchmark as a diagnostic signal, not proof that a model will collaborate safely in their workplace.
Global context
The study covers several providers and open- and closed-weight models, but its records, language, teammate design and scoring reflect specific research choices. Collaboration norms differ across cultures, occupations and safety regimes. A short English-language scenario cannot establish how an agent will behave with clinicians, engineers, teachers or public servants. The global relevance lies in the failure mode—context and pressure can change behaviour—not in treating one leaderboard as a universal workplace ranking.
What the evidence does not yet show
- The paper is an unreviewed preprint; peer-review corrections and independent replication are unavailable.
- The problematic teammate is scripted and the other live teammates are language models, not people.
- Test versions run three times, while alone and calm-team baselines run once.
- Each model is scored on tasks it solves alone, so denominators differ across models.
- Some labels rely on model judges; the human agreement check covered 40 messages, and 19 grading errors were corrected in 315 reviewed failed runs.
- The score has not been shown to predict outcomes in live workplaces or human-agent teams.
What to watch next
- Peer review, corrections and independent replication
- Repeated solo and calm-team baselines
- Results with human teammates and live workflows
- Whether provenance and mandatory rechecking improve handoffs
- Model-version drift after provider updates
- Prospective evidence that the score predicts collaboration beyond solo capability
Living evidence record
Impact record IAI-08B300V
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
3 November 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 3 November 2026
Research question, topics and regions
Whether solo capability predicts reliable teamwork when AI agents face pressure, missing information, conflict and compromised handoffs
Topics: AI agents · Multi-agent systems · Evaluation · Teamwork · Handoff safety · Preprint
Regions: United States · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.