When cheque fields disagreed, AI sometimes repeated one amount—how serious is the risk?
Two multimodal models were tested on 841 selected cheques. Under one redaction condition, predicted-field agreement rose sharply, but the study does not prove a live banking error or a universal copying mechanism.
The short version
- What happened
- The study evaluated GPT-4.1 and Claude 4.5 Sonnet on 243 primary cheques plus 598 strict mismatches from two Indian datasets.
- Who it affects
- A mistaken amount extraction can delay a payment, create a costly exception or move the wrong sum unless the two fields and the underlying image are checked separately.
- What remains unproven
- The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
Visual ambiguity changed whether the two extracted amounts agreed
BNY-affiliated researchers tested whether multimodal models confuse the numeric courtesy amount on a cheque with the written legal amount. Their primary cohort contained 243 selected cheques: 124 had two observed unequal amounts and 119 had one blank annotation represented as zero. Two additional Indian datasets supplied 598 strict mismatches, for 841 evaluated cheques in total.[1]
Who it affects
A mistaken amount extraction can delay a payment, create a costly exception or move the wrong sum unless the two fields and the underlying image are checked separately.[1]
Keep in mind
The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
Source analysis · 1 linked source. Original source date: 19 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
Visual ambiguity changed whether the two extracted amounts agreed
BNY-affiliated researchers tested whether multimodal models confuse the numeric courtesy amount on a cheque with the written legal amount. Their primary cohort contained 243 selected cheques: 124 had two observed unequal amounts and 119 had one blank annotation represented as zero. Two additional Indian datasets supplied 598 strict mismatches, for 841 evaluated cheques in total.
On full images, unexpected matching predictions occurred in 4.0% to 9.7% of the 722 strict two-field mismatches across models and datasets. When the courtesy field was redacted while the legal field remained visible, agreement rose to 73.7% for GPT-4.1 and 49.0% for Claude on the broader 243-case cohort. That sensitivity is a warning about visual context, but matching predictions do not prove that one field was literally copied into the other.[1]
A small prompt intervention helped unevenly
The researchers also supplied small example tables as a single image. Claude's best condition improved joint extraction accuracy by 11.5 percentage points on all 243 cheques, but its gain on the stricter 124-case subset was not statistically significant after correction. GPT's effects depended on the example set. Model-reported confidence could remain high on wrong or agreeing outputs, so confidence alone was not a reliable safety check.
For banks and payment processors, the practical response is field-level validation: compare numeric and written amounts, preserve image context and route disagreements or obscured fields to human review. The public cohorts are small, selected and handwriting-specific; the primary cases came from a training split, and confidential production data were unavailable. Independent tests on held-out operational documents and measured payment errors would materially change the assessment.[1]
The distinction that matters
Why two cheque fields need separate validation
A cheque expresses one amount twice, but each field can be ambiguous for a different reason.
Courtesy amount
Numeric field is read
Digits may be obscured or stylized
Legal amount
Written field is read
Words provide a second signal
Cross-check
Predictions and image regions are compared
Mismatch triggers review
Watch the background, then check the evidence
Illustrative photo · not a video frame
UNESCO · 11 min 27 sec · Background explainer
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
YouTube loads only when you choose; playback starts when you press play.
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Published 2024-10-14. Source checked 14 October 2026. Check captions on the publisher’s video page.
Read the AI and jobs guidePhoto: Dylan Gillis / Unsplash · Licence
Evidence check
The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
Reporting basis: Source analysis. Original source date: 19 October 2026.
Continue the story
Read next
Continue exploring operational risk: compare the finding and its evidence limits.
Should an AI agent be allowed to move a company's money?
Airwallex says its revamped business accounts let approved agents manage liquidity and transfers within rules and approvals. The 30 September release identifies important controls, but provides no independent test, loss data or deployment denominator.
Read the next reportExplore the full source and evidence record
What this means for people
- A mistaken amount extraction can delay a payment, create a costly exception or move the wrong sum unless the two fields and the underlying image are checked separately.
Global context
Cheque formats came from the United States, Singapore and India. Handwriting, layouts, payment rules and the importance of the legal amount vary by jurisdiction and institution.
What the evidence does not yet show
- The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
What to watch next
- Held-out production-style tests, independent replication, validation across more cheque formats and evidence that field-level controls reduce real processing errors.
Living evidence record
Impact record IAI-118U8I0
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
21 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 21 October 2026
Research question, topics and regions
Whether multimodal language models incorrectly make the numeric and written amount fields agree when extracting information from visually ambiguous cheques
Topics: Financial documents · Cheque processing · Multimodal models · Model confidence · Operational risk
Regions: United States · India · Singapore · Asia · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.