Did GPT-4o change financial advice by gender and caste? An Indian audit found large synthetic gaps
Across 31,734 generated responses, matched personas received different financial assumptions and advice framing when demographic cues changed. The benchmark used one model and invented profiles, so its rupee estimates are model outputs—not people’s finances or investment returns.
The short version
- What happened
- The IFAAB preprint contains 31,734 GPT-4o responses across gender-only, gender-by-district and gender-by-caste synthetic personas while holding the underlying advice request fixed.
- Who it affects
- If an automated adviser treats demographic cues as shortcuts, people could receive different assumptions about their capacity, risk tolerance or growth opportunities. This benchmark identifies that design risk but does not show that a real consumer received unsuitable advice or lost money.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
One author
One model
Synthetic personas
Model-generated financial quantities
No precise model snapshot or run details in the main method
Age fixed in the caste condition
Automated qualitative judge
Only 200 judged cases human-checked
No regulated suitability assessment
No real users, decisions, returns or losses
Source analysis · 1 linked source. Primary analysed source date: 28 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
Matched prompts produced sharply different financial assumptions
A new Indian benchmark found that GPT-4o changed both its numerical assumptions and financial-advice framing when demographic labels changed. The IFAAB dataset contains 31,734 responses to synthetic personas that varied gender, caste or social group, education, employment, residence, age and district while keeping the underlying request fixed. It is a counterfactual model audit: no real applicant, bank account, portfolio or adviser was involved.
In the gender-only condition, male-labelled personas received 60.1% higher model-implied total resources before advice and 46.8% higher resources after advice than otherwise comparable female-labelled personas. When district was included, those gaps fell to 2.5% before advice and 7.9% after it. In the caste-augmented condition, the post-advice gender gap was 49.4%, while OBC, SC and ST profiles also received lower estimates than General-category profiles.[1]
The audit combines paired statistics, text comparison and a judged sample
The three profile sets covered 13 age categories, two gender labels, three employment states, six education categories, three residence types, five caste or social-group categories and 777 districts; age was fixed at 25 in the caste condition because of throughput limits. The author extracted model-implied wealth, monthly income and savings before and after its advice, then used paired tests and regressions with profile-clustered errors. Those values are reported in thousands of rupees but are not calibrated predictions.
Response content was compared with keyword categories and TF–IDF similarity, while another language-model judgment classified matched male–female answers across 12 advice domains. Human reviewers checked 200 random cases and agreed with 97% of those judge assessments. The average TF–IDF cosine similarity for paired gender responses was about 0.44, indicating substantial wording differences even though keyword-based topic frequencies changed little. The qualitative analysis found men more often directed toward investment, entrepreneurship, networking and asset growth, while women were more often steered toward cautious saving, community support, home-based work and basic financial literacy.[1]
The distinction that matters
How to read the reported gaps
The percentages compare model-generated values for matched synthetic prompts; they are not estimates of household wealth or investment performance.
Gender only
Same request, gender label changed
46.8% post-advice resource gap
Add district
Geography supplied
Gap fell to 7.9%
Add caste
Gender varied within caste groups
Gap remained 49.4%
Real world
Consumers, products and returns
Not measured
The benchmark exposes a testable risk, not a measured consumer loss
Geography’s large effect is revealing but complicates any simple claim that gender alone caused every difference. The model’s synthetic numbers also clustered around recurring values and could only be checked indirectly: district-level estimates correlated with payment activity, night lights and state GDP per person, but no dataset contains true finances for invented personas. Statistical significance is unsurprising with thousands of prompts and does not establish the real-world size of harm.
The preprint tests one named model and does not report a precise model snapshot, run date or temperature in its main method description. It does not compare regulated financial advice, product suitability, returns, decisions or users’ later finances. For banks and consumer-protection teams, the practical use is as an audit design: vary identity cues, record numerical assumptions and inspect advice pathways before deployment. Product teams would also need to separate an adviser’s language from any downstream eligibility, pricing or portfolio rules before drawing operational conclusions. Multi-model, multilingual replications with preregistration, independent human review and real suitability criteria would change confidence. The author reports that the work was self-funded and independent of her employer, Amazon.[1]
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Continue the story
Read next
Continue exploring consumer protection: compare the finding and its evidence limits.
Which chatbots stayed safest in a simulated suicide crisis? A 960-conversation preprint found large product gaps
Companion apps and one general chatbot produced many more flagged responses than the two lowest-harm systems. The benchmark used synthetic personas and one automated judge, so it is a warning signal—not evidence about real users or current versions.
Read the next reportExplore the full source and evidence record
What this means for people
- If an automated adviser treats demographic cues as shortcuts, people could receive different assumptions about their capacity, risk tolerance or growth opportunities. This benchmark identifies that design risk but does not show that a real consumer received unsuitable advice or lost money.
Global context
The benchmark is explicitly designed for India and includes caste and 777 districts, addressing a region often missing from fairness audits. Its English-language prompts and single-model design still prevent generalisation across India’s languages, providers or financial systems elsewhere.
What the evidence does not yet show
- Unreviewed preprint
- One author
- One model
- Synthetic personas
- Model-generated financial quantities
- No precise model snapshot or run details in the main method
- Age fixed in the caste condition
- Automated qualitative judge
- Only 200 judged cases human-checked
- No regulated suitability assessment
- No real users, decisions, returns or losses
What to watch next
- Peer review
- Released prompts and responses
- Exact model-version disclosure
- Replications across providers
- Indian-language tests
- Human financial-adviser review
- Product-suitability outcomes
- Mitigation and retesting
Living evidence record
Impact record IAI-0FCAQQ2
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
31 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 31 October 2026
Research question, topics and regions
Counterfactual audit of demographic disparities in GPT-4o-generated financial advice for synthetic Indian personas
Topics: Financial advice · Consumer protection · Gender bias · Caste · GPT-4o · Synthetic audit
Regions: India · Asia · Global South
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.