6,000 app reviews suggest people judge AI helpers and companions by different standards
Task usefulness dominated ChatGPT reviews while conversation and companionship dominated Replika reviews. The study analyses public review text—not representative users, wellbeing or product quality.
The short version
- What happened
- Researchers analysed 6,000 public app-store reviews: 3,000 for ChatGPT and 3,000 for Replika, using sentiment, emotion, topic modelling and statistical tests.
- Who it affects
- Designers and regulators may need different evaluation questions for an assistant that promises efficient work and a chatbot marketed around companionship. A five-star score alone can hide whether the user expected accuracy, emotional continuity or both.
- What remains unproven
- Self-selected public reviews
Keep in mind
Self-selected public reviews
3,000 reviews per app
Two apps only
Different sampling periods
Products changed over time
Lexicon-based emotion inference
Small effect sizes for emotion differences
No demographic, behavioural or wellbeing outcomes
Source analysis · 1 linked source. Primary analysed source date: 28 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Purpose showed up more clearly than overall positivity
A task assistant and a companion chatbot may both look conversational, but reviewers wrote about them as different kinds of products. The researchers sampled 3,000 public reviews for each app, then compared star ratings, sentiment words, emotion labels and topics.
The clearest split was what people discussed. Conversation and companionship accounted for 53.6% of Replika reviews and 10.9% of ChatGPT reviews. Task assistance and usefulness accounted for 30.9% of ChatGPT reviews and 1.2% of Replika reviews. ChatGPT ratings were also more concentrated at one and five stars, while Replika had more middle ratings.[1]
Review language is not a user-outcome study
The two sentiment methods did not agree on which app's reviews were more positive, and reported emotion differences were small. That makes broad claims about satisfaction inappropriate. The study instead supports a narrower point: people may evaluate instrumental and relational systems through different expectations.
Public reviewers are self-selecting, the products and policies change, and the two samples span different periods. The analysis did not verify reviewers' demographics, usage intensity, subscription status or mental-health outcomes. It compared two prominent apps rather than establishing universal categories for conversational AI.[1]
The distinction that matters
Two review frames, not two measures of product quality
The study found a strong topic split while overall sentiment remained method-sensitive.
Task frame
Reviewers discuss help and usefulness
30.9% of ChatGPT reviews; 1.2% of Replika reviews
Relationship frame
Reviewers discuss conversation and companionship
53.6% of Replika reviews; 10.9% of ChatGPT reviews
Evidence limit
Analyse posted review text
No representative user or wellbeing measure
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring sentiment analysis: compare the finding and its evidence limits.
Can zero-shot AI find what buyers value?
A peer-reviewed study turns 12,900 Amazon smartwatch reviews into product priorities without task-specific training. A 200-review human check supports the sentiment labels, but the attribute taxonomy and strategic conclusions were not independently validated.
Read the next reportExplore the full source and evidence record
What this means for people
- Designers and regulators may need different evaluation questions for an assistant that promises efficient work and a chatbot marketed around companionship. A five-star score alone can hide whether the user expected accuracy, emotional continuity or both.
Global context
App-store reviews can come from many countries, but platform access, language and product versions shape who appears in the corpus. The study should not be read as a population survey or a cross-cultural comparison.
What the evidence does not yet show
- Self-selected public reviews
- 3,000 reviews per app
- Two apps only
- Different sampling periods
- Products changed over time
- Lexicon-based emotion inference
- Small effect sizes for emotion differences
- No demographic, behavioural or wellbeing outcomes
What to watch next
- Contemporaneous samples
- Representative user surveys
- Longitudinal product changes
- Independent coding of relational expectations
- Wellbeing and dependency outcomes
- Comparisons across more assistants and companions
Living evidence record
Impact record IAI-0XCX5HL
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
28 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Scientific Reports published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 28 October 2026
Research question, topics and regions
How public app-store reviews differ between a task-oriented assistant and a companionship-oriented conversational AI
Topics: Conversational AI · Companion chatbots · App reviews · User expectations · Sentiment analysis · Topic modelling
Regions: India · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.