Can AI spot a lie? Strong synthetic scores collapsed on real military interviews
Models reached 88%–93% accuracy on generated text but only 35.5%–48.7% on 76 real participants. The study does not support operational lie detection.
The short version
- What happened
- Four classifiers scored 88.0%–93.0% accuracy on a held-out set of 300 synthetic participant records, then only 35.5%–44.7% on 76 real military-exercise participants.
- Who it affects
- A deception label can influence investigations, employment and security decisions. Below-chance or near-chance accuracy in a small real-world sample is a reason to withhold operational use, not to tune a threshold around the same data.
- What remains unproven
- 76 real participants
Keep in mind
76 real participants
30 deceptive participants
Military training exercise, not operational screening
Synthetic training text
No language marker survived multiplicity correction
Company-linked evaluated system
Government-contract funding
Provisional typeset version
Source analysis · 1 linked source. Primary analysed source date: 28 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 2 min The evidence, methods, implications and limitations
Generated language made the task look much easier
Researchers trained logistic regression, random forest, naive Bayes and support-vector models on 1,000 synthetic participant records—8,000 generated answers in total—split 700 for training and 300 for testing. Logistic regression led the synthetic test with 93.0% accuracy and a 0.978 ROC AUC.
Applied without retraining to interviews from 76 US Army training participants, including 30 assigned to deceive, every classifier fell below 45% accuracy. FranklinAsk, a system grounded in 21 deception papers, reached 48.7%. With wide uncertainty and a small sample, those point estimates do not establish operational value.[1]
The practical warning is about validation, not better lie detection
For investigators and people being assessed, a false label can carry serious consequences. The study suggests synthetic text encoded cleaner stereotypes of deception than stressed, spontaneous speech did. No language feature survived the study's multiple-testing correction.
The US Defense Counterintelligence and Security Agency funded the work. Three authors worked for EntigenLogic, whose FranklinAsk system was evaluated. Larger independent studies with representative interviews, pre-specified thresholds and real decision outcomes would be needed before deployment claims become credible.[1]
The distinction that matters
Why the headline score did not travel
The same models faced two very different kinds of language.
Synthetic
Test on 300 generated participant records
88.0%–93.0% classifier accuracy
Real exercise
Apply models to 76 military participants
35.5%–48.7% accuracy across systems
Decision
Ask whether results support screening
No operational validation demonstrated
Continue the story
Read next
Continue exploring retrieval-augmented generation: compare the finding and its evidence limits.
Can a simple projection reduce what RAG systems leak?
Randomly compressing query and document embeddings sharply reduced extraction in four laboratory tests while retaining much of the utility. It is an empirical defence, not a privacy guarantee.
Read the next reportExplore the full source and evidence record
What this means for people
- A deception label can influence investigations, employment and security decisions. Below-chance or near-chance accuracy in a small real-world sample is a reason to withhold operational use, not to tune a threshold around the same data.
Global context
The exercise involved US Army trainees and cannot establish performance across languages, cultures, interview settings or civilian legal systems.
What the evidence does not yet show
- 76 real participants
- 30 deceptive participants
- Military training exercise, not operational screening
- Synthetic training text
- No language marker survived multiplicity correction
- Company-linked evaluated system
- Government-contract funding
- Provisional typeset version
What to watch next
- Independent preregistered replication
- Larger representative interview samples
- False-positive harms
- Cross-language testing
- Prospective operational validation
- Controlled component ablations
Living evidence record
Impact record IAI-10ISX0J
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
28 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Artificial Intelligence published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 28 October 2026
Research question, topics and regions
Topics: Deception detection · Military training · Synthetic data · Machine learning · Retrieval-augmented generation · Operational testing
Regions: United States
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.