LLMs found depression signals in one-minute diaries—but not a diagnosis
Seven language models produced scores that moved with self-reported mood in 108 US adults. The peer-reviewed study is an early measurement test, not evidence that passive screening is accurate or safe in clinical care.
The short version
3 key points
- What changed
- The study followed 108 Pittsburgh-area adults aged 18–40 for 14 or 21 days; they completed a mean 14.9 one-minute video diaries, alongside surveys and a baseline clinical interview.
- Why it matters
- People may welcome a lower-burden way to describe how they feel, but diary text is intimate. A monitoring service could misread ordinary distress, miss serious symptoms or expose private context unless consent, security and human review are designed into the workflow.
- What to watch
- Larger preregistered multilingual replication, independent model testing, subgroup error rates, repeated clinician assessments, outcome prediction and explicit trials of consent, privacy and human escalation.
The models tracked some of the same variation as surveys
Researchers asked 108 adults to record roughly one-minute smartphone videos about the most significant event of their day. The group was 53% female, averaged 28.3 years old and included 49 people currently receiving mental-health treatment. Participants completed diaries on 77% of study days, a mean 14.9 entries each. Whisper transcribed the audio, then GPT-4.1, GPT-4.1 Mini, Claude 3.7 Sonnet, Gemini 2.5 Pro, Grok 3 Beta, Llama 4 Maverick and Qwen3-235B each scored three depression items from the text with temperature set to zero.[1]
Why a useful measurement result is not a screening licence
The study tested convergent, criterion and incremental validity—not whether an LLM could safely identify a disorder, predict deterioration or trigger care. LLM and survey scores showed similar patterns against daily affect and stress, and each retained some associations after adjustment for the other. But the clinical interviews were conducted once at baseline, inter-rater agreement was moderate, and no sensitivity, specificity, false-alarm rate or treatment outcome was reported.[1]
Who it affects
People may welcome a lower-burden way to describe how they feel, but diary text is intimate. A monitoring service could misread ordinary distress, miss serious symptoms or expose private context unless consent, security and human review are designed into the workflow.[1]
Keep in mind
One small Pittsburgh sample of 108 adults aged 18–40; diary data from 2016–2018; no diagnostic threshold, crisis detection, prospective outcome, multilingual validation or adequately powered racial-bias analysis.
Source analysis · 1 linked source. Original source date: 17 October 2026.
Continue the story
Read next
Health & Life Sciences
Medical AI bias research is growing—but it rarely reaches real clinics
A peer-reviewed review found 92 empirical studies across clinical decision support, patient communication, documentation and education. The field can demonstrate unequal model behaviour, but inconsistent definitions, narrow datasets and scarce deployment evidence still block confident comparisons.
2 min · 1 source
Health & Life Sciences
What happens when someone refuses a chatbot's crisis referral?
In 2,550 scripted conversations, all 17 chatbots recommended outside help after suicide thoughts were disclosed. After refusal, persistence ranged from almost none to nearly all ten attempts, and newer versions persisted less in every matched model line.
2 min · 1 source
Read more The full report, methods and context
The models tracked some of the same variation as surveys
Researchers asked 108 adults to record roughly one-minute smartphone videos about the most significant event of their day. The group was 53% female, averaged 28.3 years old and included 49 people currently receiving mental-health treatment. Participants completed diaries on 77% of study days, a mean 14.9 entries each. Whisper transcribed the audio, then GPT-4.1, GPT-4.1 Mini, Claude 3.7 Sonnet, Gemini 2.5 Pro, Grok 3 Beta, Llama 4 Maverick and Qwen3-235B each scored three depression items from the text with temperature set to zero.
Averaged across the seven models, the diary score correlated 0.45 with within-person changes in same-day self-reported depression and 0.61 with differences between participants' average self-reports. Model-specific correlations were similar: 0.40–0.45 within people and 0.55–0.64 between people. The averaged LLM score correlated 0.53 with both interview-rated major-depression symptom counts and interview-rated depressivity. Correlation is agreement in movement, not diagnostic accuracy.[1]
Why a useful measurement result is not a screening licence
The study tested convergent, criterion and incremental validity—not whether an LLM could safely identify a disorder, predict deterioration or trigger care. LLM and survey scores showed similar patterns against daily affect and stress, and each retained some associations after adjustment for the other. But the clinical interviews were conducted once at baseline, inter-rater agreement was moderate, and no sensitivity, specificity, false-alarm rate or treatment outcome was reported.
The authors found no age- or sex-related moderation, but the sample was too small to assess racial, linguistic or other demographic bias. Participants were 18–40, from one US city and mostly community or outpatient volunteers; more severe psychiatric populations were not tested. The diaries were recorded in 2016–2018, while the seven LLMs were applied later. That design cannot show how people would change what they say when they know an automated system is monitoring them.[1]
The distinction that matters
What the study measured—and what it did not
The paper compared model-generated mood scores with other measures; it did not validate a diagnostic or crisis-detection system.
Input
108 adults record mean 14.9 daily diaries
About one minute of speech per entry
Comparison
Seven LLM scores versus surveys and interviews
r = 0.45 within people; r = 0.61 between people
Boundary
No prospective clinical deployment
No diagnostic accuracy or patient outcome test
Privacy and consent sit outside the reported correlations
Short narratives can reveal relationships, work problems, health concerns and location clues that a three-item mood survey never collects. The research used informed consent and did not openly release recordings or transcripts. A real service would still need purpose limits, secure processing, retention rules, human review and a clear route for people to refuse or challenge an inference. None of those safeguards was evaluated here.
For clinicians and researchers, the immediate value is narrower: natural-language ratings may add a different signal to repeated self-report measures. The case would strengthen with preregistered replication in larger multilingual cohorts, independent models, repeated clinical interviews, prospective outcome prediction, subgroup error estimates and direct comparisons of burden. Until then, the study supports supervised research—not passive diagnosis from everyday speech.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
The Royal Society · 44 min 28 sec · Background explainer
Is this the AI future we want? | The Royal Society
YouTube loads only when you choose; playback starts when you press play.
A Royal Society discussion with Madhumita Murgia and biomedical engineer Alison Noble on AI’s potential in healthcare, science and culture, recorded at the 30 June 2026 event.
Evidence check: A discussion of opportunities and risks, including future possibilities. It is not evidence that every application described is available or clinically proven.
Published 2026-07-01. Source checked 14 October 2026. Check captions on the publisher’s video page.
Check the healthcare evidencePhoto: Daniel Sone / National Cancer Institute / Unsplash · Licence
Evidence check
One small Pittsburgh sample of 108 adults aged 18–40; diary data from 2016–2018; no diagnostic threshold, crisis detection, prospective outcome, multilingual validation or adequately powered racial-bias analysis.
Reporting basis: Source analysis. Original source date: 17 October 2026.
Explore the full source and evidence record
What this means for people
- People may welcome a lower-burden way to describe how they feel, but diary text is intimate. A monitoring service could misread ordinary distress, miss serious symptoms or expose private context unless consent, security and human review are designed into the workflow.
Global context
The participants and clinical measures came from one US city, while the evaluated models were supplied by companies in the United States and China. Expressions of distress, access to care and privacy law vary across languages and health systems, so transfer cannot be assumed.
What the evidence does not yet show
- One small Pittsburgh sample of 108 adults aged 18–40; diary data from 2016–2018; no diagnostic threshold, crisis detection, prospective outcome, multilingual validation or adequately powered racial-bias analysis.
What to watch next
- Larger preregistered multilingual replication, independent model testing, subgroup error rates, repeated clinician assessments, outcome prediction and explicit trials of consent, privacy and human escalation.
Living evidence record
Impact record IAI-07W30XT
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
18 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what npj Digital Medicine published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 18 October 2026
Research question, topics and regions
Whether zero-shot LLM ratings from short daily narratives track day-to-day and between-person variation in depression
Topics: Mental health · Large language models · Clinical measurement · Privacy · Bias
Regions: United States · North America · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.