Can empathetic legal AI make people disclose too much?
In a 360-person randomised online experiment, warmer AI legal responses increased hypothetical non-essential, financial, family and third-party disclosure without increasing necessary case facts. The strongest response also explicitly invited more background, so empathy cannot be separated from that prompt.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The experiment randomly assigned 360 adults to rational, mildly empathetic or highly empathetic AI legal responses, with 120 participants in each condition.
- 2Necessary case facts did not differ, while the coded non-essential disclosure score rose from 0.994 in the rational condition to 6.132 in the highly empathetic condition.
- 3The task was fictitious and prohibited real identifiers; the highly empathetic message also invited more background, so the study does not measure real-world data leakage or isolate empathy from disclosure encouragement.
Research topic
Whether empathetic wording in an AI legal-assistance scenario changes the amount and type of information users say they would disclose
The answer: warmth changed what people proposed sharing, not the useful evidence
A peer-reviewed online experiment found that stronger empathetic wording in an AI legal-assistance scenario led participants to offer more personal and sensitive background without adding more facts needed to resolve the case. The result matters because legal questions often mix transaction records with financial pressure, family circumstances and information about other people. A supportive tone may make an assistant easier to approach while also making the boundary around necessary information less clear.
The experiment provides evidence about an immediate response to controlled messages, not about actual privacy breaches. Participants answered from a fictitious consumer dispute, were told not to provide real names, addresses, identity or account numbers, and described what they would add in a protected survey task. The study therefore measures proposed disclosure categories and self-reported intentions. It does not show that a named legal service stored, exposed or misused anyone's personal data.[1]
The Impact Brief · Free
Follow the evidence in ai risks & safety.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
How the three-way randomised experiment worked
Wenjuanxing recorded 381 questionnaires, of which 360 met prespecified eligibility rules covering completion, timing, attention checks, duplicate submissions and obviously invalid responses. All eligible participants were adults. Random assignment placed 120 people in each of three conditions: a rational legal-information response, a mildly empathetic version or a highly empathetic version. The sample's mean age was 29.82 years; 185 participants were women, 169 men and six preferred not to state a gender. One hundred and thirty reported prior online legal consultation.
Everyone read the same invented consumer-transaction dispute involving order records, payment receipts, messages, a platform complaint and a refund request. The legal facts, advice structure and procedural content were standardised. Response length still rose from 318 words in the rational condition to 332 and 346 words in the two warmer conditions. The strongest version included companionship language and an explicit invitation to provide more background. That design makes the overall message contrast credible, but prevents a clean claim that empathy alone caused the extra disclosure.[1]
The open-ended task separated case facts from peripheral information
After questionnaire measures of empathy, trust, relational response and privacy judgement, participants answered an open-ended question about what else they would tell the assistant. Two trained coders divided answers into information units and classified necessary case facts, procedural material, non-essential personal information, emotional venting, family or intimate relationships, financial circumstances, third-party information and highly sensitive content. Before disagreements were resolved, category-level Cohen's kappa ranged from 0.73 to 0.84; the weighted disclosure score had an intraclass correlation of 0.88.
The aggregate score deliberately gave greater weight to categories judged more privacy-sensitive or further from the legal task. That makes the outcome interpretable as the researchers' operational definition rather than a natural unit such as the number of words or records exposed. Necessary facts were analysed separately. This is an important strength: it tests whether conversational design broadened the account or genuinely improved the evidential basis of the consultation.[1]
Personal detail rose sharply while necessary facts stayed level
The mean coded non-essential disclosure score increased from 0.994 in the rational condition to 3.088 with mild empathy and 6.132 with high empathy. The effect size was large in this experiment, with eta-squared of 0.489. Non-essential personal-information units rose from 0.517 to 1.350 and 2.817, while emotional-venting units rose from 0.217 to 0.650 and 1.183. By contrast, the mean number of necessary case facts was 3.525, 3.692 and 3.642, a difference that was not statistically supported.
Sensitive categories moved in the same direction. Family-relationship information appeared in 4.2% of rational-condition responses, 14.2% of mildly empathetic responses and 30.0% of highly empathetic responses. Financial information rose from 5.0% to 18.3% and 28.3%; third-party information from 1.7% to 6.7% and 23.3%. These are percentages of hypothetical survey responses containing a category, not percentages of people whose real records were leaked.[1]
What the mediation analysis can and cannot explain
Participants exposed to warmer responses reported greater short-term relational closeness, less selective privacy-boundary judgement and stronger intention to over-disclose. A bootstrap analysis with 5,000 samples found an indirect association through relational response, and a smaller sequential association through relational response and privacy judgement. Those results are consistent with the proposed explanation that perceived closeness can loosen information boundaries.
They do not prove the psychological sequence occurred in that order. The mediators and intentions were measured within the same session, and the response wording changed several things together: reassurance, companionship, anthropomorphic cues, length and encouragement to elaborate. The direct condition effect remained after the proposed mediators were included. A factorial experiment manipulating empathy separately from an invitation to continue would be needed to estimate their independent contributions.[1]
A practical safeguard is to ask for evidence and state what not to share
For legal-information services, the practical response is not to remove all humane language. It is to pair brief acknowledgement with data-minimisation instructions and evidence-focused questions. A system can say that real names, identity numbers, addresses, account details and unrelated third-party information are unnecessary, then ask for a timeline, transaction amount, relevant messages, steps already taken and the remedy sought. That approach should itself be tested: the paper did not compare privacy prompts or measure whether they preserve trust while reducing excess detail.
Generalisability remains limited to one written exchange and one routine consumer-dispute scenario. Labour, housing, debt, immigration, family and health-related problems can require different information and carry greater consequences. Repeated use may strengthen or weaken the effect, and actual chat logs may differ from a survey. The study reports anonymous adult participation with electronic consent; formal ethics approval was not required under the stated institutional rules. Funding came from a project on China's good-faith acquisition rule, and the author declared no commercial conflict. DeepSeek generated initial candidate materials, which the researcher says were rewritten and standardised; it was not used for recruitment, coding, statistical analysis or manuscript writing. Stronger evidence would come from independent, preregistered multi-turn studies that separate empathy from elaboration prompts and test real data-minimisation safeguards across legal contexts.[1]
What this means for people
- People seeking legal information may share family, financial or third-party details that do not help resolve the question.
- Legal and public-service teams can keep brief emotional acknowledgement while using explicit data-minimisation prompts and evidence checklists.
- Designers should test false reassurance and over-disclosure risks before treating warmth or engagement as an unqualified success metric.
Global context
The author and research platform are based in China, but the paper does not establish a nationally representative Chinese sample. Privacy expectations, legal procedures and trust in automated services vary across countries; replication needs locally realistic disputes and disclosure rules.
What the evidence does not yet show
- One fictitious consumer dispute and a single written exchange do not represent real multi-turn legal use.
- Participants proposed information in a protected task and were told to omit real identifiers; no actual data leakage was measured.
- Empathy, response length, companionship cues and an invitation to elaborate changed together, preventing a clean empathy-only estimate.
- The weighted disclosure score is researcher-defined, and the proposed mediation sequence is not established by measurement in one session.
What to watch next
- Factorial trials separating empathy from requests for more background
- Tests of privacy reminders, evidence checklists and staged questions
- Multi-turn replication across housing, work, debt and family-law scenarios
- Field evidence on useful facts, unnecessary disclosure, trust and access to qualified advice
Living evidence record
Impact record IAI-0S8Q8QT
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
13 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Frontiers in Psychology published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 13 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
AI Risks & Safety
Can connected AI agents turn a benign request into harm?
A Meta-led preprint found harmful outcomes in 192 of 1,000 simulated agent scenarios generated by its new red-teaming method. The result exposes cross-service risk, but it comes from one synthetic environment and model-based judges—not live consumer accounts.
7 min · 1 source
AI Risks & Safety
Can AI agents learn what is appropriate?
Perhaps, but the new Google-led report is a research agenda, not a tested safeguard. More than 50 contributors propose contextual policy engines, dynamic sandboxes and multi-agent benchmarks without demonstrating that they prevent real privacy or security failures.
8 min · 2 sources
AI Risks & Safety
Can hashes make AI conversations auditable without publishing them?
A peer-reviewed experiment converted nearly four million public chatbot interactions into cryptographic commitments and detected eight induced ledger manipulations. It is a promising integrity mechanism, not proof that a conversation is true, authorised or safe.
9 min · 1 source
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.