Can an AI agent run a neighbourhood shop? Four models fell short
In an unreviewed Indian simulation, the best model earned about one-eighth of a scripted shopkeeper's profit. The test exposes language and accounting failures, not outcomes in a live business.
The short version
3 key points
- What changed
- Four open-weight models each ran three 30-day shop simulations; the best averaged Rs 13,683 profit against Rs 103,434 for a fixed-rule reference.
- Why it matters
- Small retailers could bear the cost of missed orders, incorrect cash reports and language failures if an agent is trusted without reconciliation.
- What to watch
- Independent reproduction using the released code, data and traces
The best model remained far behind the scripted reference
Four open-weight language models struggled to run a simulated Indian neighbourhood shop for a month, according to an unreviewed preprint submitted on 16 October. Each model was tested across three 30-day seeds, producing 12 simulated shop-months and 360 model-managed days. The model filled four roles—manager, cashier, helper and bookkeeper—while customers and suppliers communicated in English, Hindi and Hinglish. Demand was calibrated using India's 2022-23 Household Consumption Expenditure Survey.[1][2]
Who it affects
Small retailers could bear the cost of missed orders, incorrect cash reports and language failures if an agent is trusted without reconciliation.[1][2]
Keep in mind
Unreviewed preprint; one simulated Delhi shop; four model builds; three seeds per model and no live customers, workers or suppliers.
The scripted reference had direct stock access and language-matching capabilities that the model manager did not share.
Simulated messages and purchasing behaviour cannot establish real profit, adoption, safety, workload or employment effects.
No funding or competing-interest statement was found in the accessible paper, and the linked trace archive could not be independently opened during review.
Source analysis · 2 linked sources. Original source date: 16 October 2026.
Read the full report · 5 min The evidence, methods, implications and limitations
The best model remained far behind the scripted reference
Four open-weight language models struggled to run a simulated Indian neighbourhood shop for a month, according to an unreviewed preprint submitted on 16 October. Each model was tested across three 30-day seeds, producing 12 simulated shop-months and 360 model-managed days. The model filled four roles—manager, cashier, helper and bookkeeper—while customers and suppliers communicated in English, Hindi and Hinglish. Demand was calibrated using India's 2022-23 Household Consumption Expenditure Survey.
A fixed-rule shopkeeper averaged Rs 103,434 profit. The best model, Qwen3.8-27B, averaged Rs 13,683—about one-eighth as much. Gemma 4 31B averaged a Rs 3,730 loss, while gpt-oss-20b and LFM2.5-8B-A1B lost Rs 193,411 and Rs 126,554 respectively. An unmanaged shop lost Rs 114,995. The three runs also varied widely: Gemma ranged from a Rs 37,226 profit to a Rs 40,762 loss, so model rankings should not be read as stable league-table positions.[1][2]
Language and reporting errors reached daily operations
The simulation makes failure concrete. In pooled supplier-email handling, Qwen failed 5% of messages, Gemma 7% and gpt-oss 46%. The last model failed on 23% of English messages, 51% of Hindi messages and 59% of Hinglish messages. LFM2.5 was excluded from that comparison because its model card did not support the Indic languages used. An English-only control recorded no failures across 459 messages, indicating that multilingual operation—not merely email volume—was a central difficulty in this setup.
Bookkeeping also differed. Qwen and Gemma reproduced daily sales exactly in the reported comparison, whereas gpt-oss misstated sales on 6% of days and LFM2.5 on 27%; the largest one-day discrepancies exceeded Rs 15,000. For a small retailer, those failures could mean missed stock, unpaid suppliers or decisions based on the wrong cash position. The study does not show that a human shopkeeper would accept those mistakes, or that a person supervising the system could not catch them.[1]
The distinction that matters
What the benchmark compared
The three conditions did not receive identical capabilities, which matters when interpreting the profit gap.
Language-model manager
Acts through simulated staff reports
Four models × three 30-day seeds
Fixed-rule reference
Reads stock directly and matches supplier language
Highest average profit
Unmanaged shop
No active replenishment or coordination
Loss-making reference
A controlled benchmark is not a field trial
The test is useful because it joins ordering, pricing, staffing, cash and communication over time rather than asking isolated questions. But it represents one simulated Delhi shop, one catalogue, one supplier schedule and one festival calendar. Customer and supplier language was itself model-generated. Only four specific open-weight builds were tested, with three seeds apiece, and the paper has not been peer reviewed. The authors did not report a funding or competing-interest statement in the accessible paper.
The fixed-rule reference is also a demanding but uneven comparator: it could read stock directly and communicate in the supplier's language, while the model manager depended on reports from its simulated staff. The result therefore shows that these configurations failed under the benchmark's constraints; it does not prove that every AI agent will fail or that the profit gap measures real-world productivity. Confidence would rise with more seeds and shops, preregistered field tests, human-supervised comparisons, representative language users and measured error recovery. Until then, shop owners should treat agent autonomy as an experiment requiring reconciled accounts and human control—not a replacement proven ready for the till.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Evidence check
Unreviewed preprint; one simulated Delhi shop; four model builds; three seeds per model and no live customers, workers or suppliers.
The scripted reference had direct stock access and language-matching capabilities that the model manager did not share.
Simulated messages and purchasing behaviour cannot establish real profit, adoption, safety, workload or employment effects.
No funding or competing-interest statement was found in the accessible paper, and the linked trace archive could not be independently opened during review.
Reporting basis: Source analysis. Original source date: 16 October 2026.
Continue the story
Read next
Also about ai agents
Do agentic tools now make up most business AI use in the EU?
Within OpenAI's Enterprise and Business customer sample, yes: Codex and ChatGPT Work produced 55% of included output tokens at the end of September. That is not a measure of all EU firms, completed work or productivity.
Read the 10 min reportAlso about ai agents
What would SAP gain from TechWolf?
SAP has signed an agreement to acquire TechWolf, whose software maps tasks, skills and labour-market signals for workforce planning. The price is undisclosed, closing remains conditional and no public evidence yet shows that deeper integration will improve hiring, reskilling or job design.
Read the 6 min reportExplore the full source and evidence record
What this means for people
- Small retailers could bear the cost of missed orders, incorrect cash reports and language failures if an agent is trusted without reconciliation.
- Workers may see more monitoring and exception-handling work even when routine ordering or reporting is automated.
- Hindi and Hinglish performance matters because an English-only test would have hidden much of the operational failure reported here.
Global context
Kirana shops are a major part of everyday retail in India, but inventory systems, credit practices, language use and supplier relationships vary across neighbourhoods and countries. A single calibrated Delhi simulation cannot stand in for that diversity.
What the evidence does not yet show
- Unreviewed preprint; one simulated Delhi shop; four model builds; three seeds per model and no live customers, workers or suppliers.
- The scripted reference had direct stock access and language-matching capabilities that the model manager did not share.
- Simulated messages and purchasing behaviour cannot establish real profit, adoption, safety, workload or employment effects.
- No funding or competing-interest statement was found in the accessible paper, and the linked trace archive could not be independently opened during review.
What to watch next
- Independent reproduction using the released code, data and traces
- More shops, products, language varieties, seasons and model seeds
- Field comparisons with shopkeepers and human-supervised agents
- Error recovery, account reconciliation, labour time and customer impact
Living evidence record
Impact record IAI-1Y1F4VA
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
20 October 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 20 October 2026
Research question, topics and regions
Whether open-weight language models can manage the daily operations of a simulated Indian neighbourhood shop
Topics: Small business · AI agents · Retail · Multilingual systems · Work & Skills
Regions: India · South Asia
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.