Did a chatbot empower tourism communities? The study never tested one
A peer-reviewed experiment found that Kimi.ai could tailor advice to six community-tourism roles. But all 36 participants were students acting out those roles, sessions lasted minutes, and the paper reports no real decision, income, cultural or environmental outcome.
The short version
3 key points
- 1.The experiment assigned 36 tourism undergraduates aged 18–22 to six simulated roles; none participated as a real official, investor, entrepreneur, business owner, marginalised resident or inbound tourist.
- 2.Each Kimi.ai session lasted about five to seven minutes and generated roughly 12 to 15 responses. Researchers coded relevance, responsiveness and alignment, but the paper does not report the numerical survey results or effect estimates needed to judge improvement.
- 3.The study shows that a chatbot can produce role-tailored text in a controlled exercise. It does not show that AI redistributed power, increased local income, protected culture, changed policy or improved environmental outcomes in an operating tourism community.
The answer: the chatbot adapted its language, but empowerment was not measured in a community
Kimi.ai produced different advice when students prompted it as government officials, tourism investors, returning entrepreneurs, family-business owners, marginalised residents and inbound tourists. The paper describes the responses as relevant to each role: officials received governance and benefit-sharing suggestions; investors saw risk and return considerations; residents were prompted around land, voice and compensation; and tourists received material about authenticity, safety and responsible behaviour. That is evidence that one language model can follow a structured role prompt and generate plausible, role-specific text.
It is not evidence that a community was empowered. The 36 participants were undergraduates aged 18–22 from six tourism-related majors. They acted out stakeholder identities that were not their occupations or lived social positions. No elected official changed a rule, no resident gained a vote, no local business recorded a sale and no community negotiated ownership of data or tourism revenue. The paper's strongest defensible finding is therefore about the chatbot's behaviour in a classroom-style simulation, not about social or economic change in community-based tourism.[1]
What the researchers actually did
The study used purposive sampling in December 2024. Six students were allocated to each of the six roles, giving 36 participants in total: 16 women and 20 men, all between 18 and 22. The researchers standardised an opening prompt and a follow-up task for every role. A student playing a district tourism officer, for example, had to outline licensing and policy priorities and then draft a notice inviting marginalised villagers to co-design a visitor code. A student playing an Indigenous resident was asked to describe hopes and fears and identify a concern about outsiders staying on communal land.
Sessions lasted about five to seven minutes and reportedly produced 12 to 15 chatbot responses. The team recorded the conversations, applied thematic coding and collected post-interaction feedback on ease of use, relevance and perceived usefulness. It also counted external resources offered by the chatbot. The protocol had prospective ethics approval from Guilin Tourism University, and participants gave written consent. Those safeguards matter for the students, but ethical approval does not solve the study's external-validity problem: a simulated concern voiced by a student is not a substitute for the knowledge, risk or bargaining position of the person whose land, work or culture is affected.[1]
The missing numbers make the headline claims hard to audit
The methods say chatbot performance was compared on response relevance, user satisfaction and problem-solving efficiency and that statistical tools made the analysis more rigorous. The published results, however, do not provide group means, distributions, confidence intervals, test statistics or a pre-interaction baseline. The paper does not show how many students rated an answer useful, how large any difference was between roles, whether coders agreed on the themes, or whether an alternative chatbot or a human facilitator performed better. Without those denominators and comparators, readers cannot assess the size, consistency or practical importance of the reported benefit.
The article links an associated Science Data Bank page, which is useful for transparency, but a repository link is not an independent validation. The paper should still state the scoring scale, coding procedure, inter-rater reliability and complete numerical outcomes in the report itself or an accessible supplement. Reproducibility is also affected by model drift: Kimi.ai in 2026 may not return the responses produced in December 2024, and the paper does not identify a locked model version or decoding settings. A future replication needs preserved prompts, timestamped outputs, model identifiers and an analysis plan fixed before results are inspected.[1][2]
Why role-play can conceal the people with the most at stake
Community-based tourism is partly about who controls decisions and who receives the benefits. Those questions cannot be answered only by asking a model to imitate stakeholder perspectives. A marginalised resident may face land insecurity, language barriers, political pressure or unequal access to a phone and reliable internet. A family running a homestay may understand seasonal cash flow, unpaid care work and local price expectations in ways a student prompt does not. Treating generated text as a proxy for these voices risks converting participation into a design exercise while leaving the underlying distribution of authority unchanged.
There is also a circularity risk. Researchers defined the roles and concerns in the prompts, then concluded that the model surfaced role-specific concerns. The output may reflect the categories embedded by the study team rather than needs discovered from a community. Even apparently inclusive suggestions can be wrong, culturally insensitive or impractical. Before a chatbot is used in planning, affected people should decide whether it is wanted, which languages and knowledge sources it may use, what information must remain private, who can correct it and whether its summaries have any formal status in meetings or funding decisions.[1]
What the result could still offer practitioners
The experiment does identify a plausible low-risk use: preparing people for a discussion. A chatbot could help a resident turn concerns into questions for an official, help a small operator compare contract clauses or give a meeting organiser a checklist of interests that might otherwise be missed. It could translate a draft, explain technical language or generate scenarios for participants to challenge. Used this way, AI is a rehearsal and accessibility tool rather than a stand-in for consent or collective decision-making.
The boundary is important. A model should not decide which cultural practices can be commercialised, infer community consent, determine a 'fair' revenue split or rank residents' claims. Its output should be visible, contestable and traceable to the human who chose to use it. Communities need non-digital routes to participate, and facilitators must record disagreements rather than compressing them into a smooth AI summary. Procurement should also avoid locking local organisations into a platform whose training data, retention rules, cost or language performance they cannot scrutinise.[1]
What evidence would justify a stronger claim
A credible next study would recruit real stakeholder groups from several operating tourism communities and compare a clearly bounded AI-assisted process with the community's usual process or a trained human facilitator. Outcomes should be chosen with residents before the trial. Useful measures might include whose proposals appear in the final plan, speaking time by group, comprehension of contracts, correction of factual errors, participation by people with low literacy or limited connectivity, time and cost, trust after the meeting and the number of unresolved disputes. Qualitative interviews should test whether participants felt more agency—not merely whether the interface was easy to use.
Longer follow-up must then examine material results: ownership and revenue distribution, changes in local employment, complaints, cultural harms, visitor pressure, environmental indicators and whether disadvantaged groups can reverse a decision. Researchers should compare multiple models and disclose errors, privacy incidents, funding and platform dependence. The current study was partly funded by Chiang Mai University and tourism-development programmes in Guangxi; the authors declared no competing interests. Its peer review and transparent limitations make it a useful starting point, but only field evidence can show whether a chatbot changes power rather than merely producing polished language about empowerment.[1]
What this means for people
- Residents and small tourism businesses may gain a useful drafting or translation aid, but the study did not test whether anyone gained decision-making power, income or protection from harm.
- Officials and facilitators should use chatbot output as a contestable prompt for discussion, never as evidence that a community has consented or that its interests have been represented.
- People without reliable connectivity, dominant-language fluency or confidence in formal meetings need non-digital participation routes so an AI layer does not deepen existing exclusion.
Global context
Community-based tourism is used across Asia, Africa, Latin America and other regions as a route to local income and cultural or environmental stewardship. The paper joins Chinese and Thai institutions and studies a Chinese-developed chatbot, but it tests students in one controlled setting. Its findings cannot be generalised to Indigenous communities, rural enterprises, public agencies or tourists in other legal, linguistic and economic contexts.
What the evidence does not yet show
- All 36 participants were tourism students in simulated roles rather than real community stakeholders.
- There was no control group, human-facilitator comparator or pre-interaction baseline.
- The results do not report the numerical survey outcomes, effect estimates, uncertainty or coder agreement described by the methods.
- Sessions lasted minutes and measured no decision, income, cultural-preservation, environmental or longitudinal outcome.
- Only Kimi.ai was tested, and the paper does not identify a locked model version or settings that would make outputs stable to reproduce.
What to watch next
- Trials with real residents, local businesses, officials and tourists who help define the outcomes
- Transparent quantitative results, preserved prompts and outputs, coder reliability and model-version records
- Comparisons with trained human facilitation and non-AI participation tools
- Long-term measures of decision power, revenue distribution, cultural harm, environmental pressure and digital exclusion
Living evidence record
Impact record IAI-06X15V5
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
14 October 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 14 October 2026
Research question, topics and regions
Whether role-specific interaction with a generative-AI chatbot can improve participation, problem-solving and empowerment in community-based tourism—and what evidence would be needed to show effects for real communities
Topics: Community tourism · Kimi.ai · Rural livelihoods · Cultural preservation · Participatory governance · Research methods
Regions: China · Thailand · Asia & Middle East · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Society & Media
Why did AI become experts’ top future risk?
In AXA’s 2026 perception survey, AI and big data rose from fourth to first among 4,345 risk professionals, while 24,000 adults ranked it second behind climate change. The result measures concern, not the probability or cost of future harm—and 89% of the expert panel worked for AXA.
8 min · 2 sources
Society & Media
Does moderate AI involvement feel more creative?
Across two Chinese online experiments with 600 mostly student participants, a workflow preserving meaningful human choices produced higher self-rated creativity than low- or high-involvement alternatives. The tasks were short, the workflows bundled several features and the result is not evidence that the finished art was objectively better.
7 min · 1 source
Society & Media
Do AI agents polarise each other?
A peer-reviewed simulation found polarization and homophilic clustering when 1,000 language-model agents repeatedly interacted. The result is a warning about synthetic agent societies, not evidence that real voters or social platforms will behave the same way.
8 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.