Can people still tell AI from humans? They struggled with saved snippets—but got every GPT-3.5 live test right
Across six constrained tasks, people often misclassified machine output. Active judges perfectly identified GPT-3.5 in one live setup, while a later Llama follow-up again produced mistakes.
The short version
- What happened
- The study assembled 36,499 answers from 636 people and 37 AI systems, then recorded 72,191 Turing-like judgements from 1,916 human judges across six language and vision tasks.
- Who it affects
- People assessing messages, reviews or media should not treat writing style alone as proof of human authorship; platform provenance and contextual verification remain more useful than confident guessing.
- What remains unproven
- Controlled tasks with balanced human/machine trials do not represent real-world prevalence
Keep in mind
Controlled tasks with balanced human/machine trials do not represent real-world prevalence
Evidence spans older datasets and model generations; the preprint began in 2022
Live tests mix language ability with timing, safety refusals and typing-style clues
Imitation is not a measure of intelligence, accuracy, deception intent or safety
Crowdsourced and laboratory samples may not transfer across languages and cultures
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Fixed snippets were difficult to classify
A peer-reviewed study found no single answer to whether people can distinguish AI from humans: performance changed sharply with the task. Researchers assembled 36,499 answers from 636 people and 37 AI systems across image captioning, word association, conversation, colour estimation, object detection and attention prediction. Another 1,916 people made 72,191 balanced human-or-machine judgements.
In fixed language samples, judges often struggled. Machine image captions and word associations were correctly labelled as machine 46% and 47% of the time; machine contributions in human–AI conversations were identified 56% of the time. Because trials were balanced, those results describe controlled discrimination—not the probability that any post encountered online is synthetic.[1]
Active questioning made GPT-3.5 easier to spot
The result reversed in a mobile test where judges questioned one person and GPT-3.5 directly: they identified both correctly in every reported conversation. Judges could probe personal details and observe response timing and style. Those clues were part of the test, so the finding does not isolate language quality alone. In a later 28-person follow-up using Llama-3-70B without safeguards, 188 trials no longer produced perfect identification, although detection improved with longer exchanges.
For readers, journalists and platforms, the practical lesson is narrower than 'AI passes the Turing test': style-only guesses from isolated content are unreliable, while interaction and provenance can add evidence. The paper's preprint first appeared in 2022 and was revised in 2025 before today's peer-reviewed version; several model generations and datasets are therefore historical. Imitation also does not establish intelligence, truthfulness, harmful intent or real-world prevalence. Replication with current systems, multiple languages and realistic base rates would strengthen the claim.[1]
The distinction that matters
Why the test format changed the answer
The experiments tested controlled imitation under different conditions, not a universal detector for real social feeds.
Saved snippets
Judges classified fixed language samples
Machine answers were often missed
Live GPT-3.5
Judges asked their own questions and saw timing cues
Every reported identity judgement was correct
Live Llama follow-up
28 judges completed 188 trials without safeguards
Perfect identification did not persist
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Continue the story
Read next
Continue exploring human judgement: compare the finding and its evidence limits.
Can people spot AI-generated images?
A preregistered experiment with 2,091 adults in Denmark found that people were slightly more accurate than random probability guesses, yet on average would have scored better by assigning every image a 50% chance of being AI-generated. The test used one 2024 image tool and does not measure belief, sharing or harm.
Read the next reportExplore the full source and evidence record
What this means for people
- People assessing messages, reviews or media should not treat writing style alone as proof of human authorship; platform provenance and contextual verification remain more useful than confident guessing.
Global context
The collaboration spans Singapore and the United States, and participants were recruited online and in laboratories. Cultural and linguistic cues can affect human judgements, so the findings should not be assumed to transfer unchanged to other languages, regions or platform contexts.
What the evidence does not yet show
- Controlled tasks with balanced human/machine trials do not represent real-world prevalence
- Evidence spans older datasets and model generations; the preprint began in 2022
- Live tests mix language ability with timing, safety refusals and typing-style clues
- Imitation is not a measure of intelligence, accuracy, deception intent or safety
- Crowdsourced and laboratory samples may not transfer across languages and cultures
What to watch next
- Blinded interactive tests using current frontier models and matched timing cues
- Replication across languages, cultures and accessibility needs
- Field studies using realistic ratios of human and synthetic content
- Whether provenance standards outperform human classification in practice
Living evidence record
Impact record IAI-0IBYA96
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
26 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Nature Human Behaviour published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 26 October 2026
Research question, topics and regions
How accurately people can distinguish human and AI responses across constrained language, vision and live interactive tasks
Topics: AI detection · Online trust · Turing tests · Human judgement
Regions: Asia · North America · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.