Workplace AI agents struggled when records conflicted—what should teams verify?
Across 1,613 questions over simulated projects, the strongest tested model reached 68% accuracy with 50 retrieved records and remained weak on cross-source and historical conflicts.
The short version
- What happened
- MMEnterprise-Bench contains 1,613 questions over 7,140 simulated workplace records from 30 projects, including chats, emails, reports, slides and figures.
- Who it affects
- A worker who relies on an agent for a deadline, figure or approval could repeat an obsolete value even when the correct revision exists elsewhere in the workspace.
- What remains unproven
- The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
More retrieved evidence helped, but did not solve revision conflicts
A new South Korean benchmark finds that workplace AI agents can answer routine questions yet still falter when a team has corrected a figure, preserved an older document or spread evidence across several images. The researchers built 30 simulated collaboration projects from public papers and reports, retaining earlier versions rather than overwriting them. The resulting workspace contains 7,140 records and 1,613 questions: 1,102 evidence-reasoning tasks, 424 conflict-resolution tasks and 87 questions that should be judged unanswerable.[1]
Who it affects
A worker who relies on an agent for a deadline, figure or approval could repeat an obsolete value even when the correct revision exists elsewhere in the workspace.[1]
Keep in mind
The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
Source analysis · 1 linked source. Original source date: 20 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
More retrieved evidence helped, but did not solve revision conflicts
A new South Korean benchmark finds that workplace AI agents can answer routine questions yet still falter when a team has corrected a figure, preserved an older document or spread evidence across several images. The researchers built 30 simulated collaboration projects from public papers and reports, retaining earlier versions rather than overwriting them. The resulting workspace contains 7,140 records and 1,613 questions: 1,102 evidence-reasoning tasks, 424 conflict-resolution tasks and 87 questions that should be judged unanswerable.
Ten proprietary and open models answered with either 10 or 50 retrieved records. GPT-5.6 Sol led at 68.0% overall with 50 records, but scored 55.2% on cross-source conflicts and 56.1% on temporal conflicts. Its accuracy fell to 27.5% when a question required several images, compared with 83.3% for one text record. A larger retrieval budget improved recall, but stronger retrieval did not consistently produce a correct answer.[1]
The practical control is version-aware review, not blind retrieval
For workers, the immediate implication is procedural: an agent should expose which record version and timestamp support an answer, surface contradictions and allow a person to inspect the evidence before a deadline, metric or decision is reused. Simply giving the system more documents can also give it more obsolete or conflicting material.
The benchmark remains synthetic. LLMs generated and validated much of the workspace, GPT-5.6 Luna judged model answers, and the authors themselves reviewed a 100-item sample; all 50 automatically accepted items passed three human reviewers, but that sample cannot establish correctness for all 1,613 questions. Real-company trials with authentic permissions, revision systems and independent human adjudication would change the assessment.[1]
The distinction that matters
Why a workplace answer can be current, historical or unsupported
The same claim can appear in several sources and several versions.
Cross-source
Chat and report disagree
Agent must identify the applicable record
Temporal
A value changed over time
Question date determines the answer
Unanswerable
Required evidence is absent
Agent should decline to infer
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Evidence check
The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
Reporting basis: Source analysis. Original source date: 20 October 2026.
Continue the story
Read next
Continue exploring multimodal ai: compare the finding and its evidence limits.
What do people actually ask image-upload AI to do?
An unreviewed Microsoft-led study analysed 42,617 de-identified Copilot image-upload sessions and checked its taxonomy against 23,413 ChatGPT sessions. Users often chained recognition into writing, code and data tasks—but private, model-generated summaries limit what outsiders can verify.
Read the next reportExplore the full source and evidence record
What this means for people
- A worker who relies on an agent for a deadline, figure or approval could repeat an obsolete value even when the correct revision exists elsewhere in the workspace.
Global context
The authors are based in South Korea, but document revision and provenance are common problems in workplaces worldwide. Local privacy and access-control rules remain outside the benchmark.
What the evidence does not yet show
- The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
What to watch next
- Independent replication on permissioned real-world workspaces, stronger citation and version displays, human adjudication and controlled tests of conflict-aware retrieval.
Living evidence record
Impact record IAI-1AGAAZT
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
21 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 21 October 2026
Research question, topics and regions
Whether multimodal enterprise agents can answer questions when workplace records contain revisions, cross-source contradictions, multiple historical states or insufficient evidence
Topics: Workplace agents · Enterprise search · Document conflicts · Human verification · Multimodal AI
Regions: South Korea · Asia · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.