Giving vision models a pointing tool made their counting visible—not reliably better
Across nine models and 682 images, numbered marks exposed omissions, duplicates and phantom objects. The tool improved small-photo counts for only four models and produced a clearly positive interval for one.
The short version
- What happened
- The researchers tested nine vision-language models on 682 controlled arrays and photographs under seven tool and prompting conditions.
- Who it affects
- People using vision models to inventory objects, inspect images or support accessibility need more than a confident number. A visible trace can make mistakes easier to challenge, but the tested tool was not reliably accurate enough for consequential counting.
- What remains unproven
- This is an unreviewed workshop preprint; the developmental analogy is qualitative and does not mean that vision models learn or count like children.
Keep in mind
This is an unreviewed workshop preprint; the developmental analogy is qualitative and does not mean that vision models learn or count like children.
Models were accessed through one provider API, and the authors could not standardize reasoning settings for every endpoint.
The larger models ran only a stratified subset of conditions and images; the photo datasets cover limited object counts and categories.
No separate funding or competing-interest statement is reported; one author uses an Antigma.ai address.
Source analysis · 1 linked source. Original source date: 20 October 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 3 min The evidence, methods, implications and limitations
The marks turned a wrong answer into an inspectable failure
Four independent researchers gave vision-language models simple image tools: place a numbered tag, crop, erase or overlay a grid. The tools contained no visual intelligence. The model chose where to mark, then saw the updated image. Because the datasets supplied object locations, each tag could be classified as correct, duplicated, placed on nothing or missing an object.
The study covered nine models, 682 images and seven harness conditions. It combined 272 controlled dot arrays, 30 larger arrays, 200 FSC-147 photographs with 8–80 objects and 180 PixMo-Count photographs with 2–10. Three cheaper models ran every condition; four larger models were tested on a stratified 240-image subset, so not every model-condition comparison uses the full denominator.[1]
A visible procedure did not become a dependable skill
On small-count photographs, tags improved four of nine models and produced a paired-bootstrap interval excluding zero for one. Imprecise pointers became worse, and the tagging loop rarely beat a verbal counting routine or a list of unseen coordinates. Reversing the instructed scan order also changed answers for some models beyond ordinary repeat instability, suggesting that following a procedure did not guarantee order-independent counting.
The practical value is evaluation: a tester can see whether a model skipped objects, double counted or reported a number inconsistent with its own marks. The study used one API, temperature zero and mostly low reasoning effort; two endpoints could not disable reasoning, and the photo datasets cover bounded count ranges. Independent replication and training tests would be needed before calling this a path to reliable visual enumeration.[1]
The distinction that matters
What a visible counting trace reveals
A numbered mark can be checked against object-level ground truth.
On target
One mark lands on one object
Supports one-to-one counting
Duplicate or phantom
Extra mark repeats or misses an object
Shows over-counting
Omission
An object receives no mark
Shows where the count was lost
Watch the background, then check the evidence
Illustrative photo · not a video frame
Google DeepMind · 1 min 52 sec · Background explainer
Protein folding explained
YouTube loads only when you choose; playback starts when you press play.
A short introduction to why a protein’s shape matters. It provides background for understanding what AI structure prediction contributes to biological research.
Evidence check: This is the developer’s explanation. Predicting a structure does not by itself establish a treatment’s safety or benefit for patients.
Published 2020-11-30. Source checked 14 October 2026. Check captions on the publisher’s video page.
Explore current researchContinue the story
Read next
Continue exploring vision-language models: compare the finding and its evidence limits.
Eight vision-language models read charts differently from people—why does that matter?
Across six assessments, models were generally less accurate than people and made different patterns of mistakes. The study is broad and peer reviewed, but its model snapshot dates from 2024.
Read the next reportExplore the full source and evidence record
What this means for people
- People using vision models to inventory objects, inspect images or support accessibility need more than a confident number. A visible trace can make mistakes easier to challenge, but the tested tool was not reliably accurate enough for consequential counting.
Global context
Auditable visual reasoning could matter in warehouses, environmental monitoring, accessibility and quality control, but performance will depend on image type, object density, model access and local validation.
What the evidence does not yet show
- This is an unreviewed workshop preprint; the developmental analogy is qualitative and does not mean that vision models learn or count like children.
- Models were accessed through one provider API, and the authors could not standardize reasoning settings for every endpoint.
- The larger models ran only a stratified subset of conditions and images; the photo datasets cover limited object counts and categories.
- No separate funding or competing-interest statement is reported; one author uses an Antigma.ai address.
What to watch next
- Independent replication through other model providers
- Tests on higher-stakes counting and inspection tasks
- Training that teaches models to verify marks rather than merely produce them
- Evaluations that separate pointing precision, bookkeeping and final-answer consistency
Living evidence record
Impact record IAI-1XL2AAA
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
22 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 22 October 2026
Research question, topics and regions
Whether simple visual marking tools improve vision-language-model counting and whether the marks provide an auditable trace of serial visual reasoning
Topics: Vision-language models · Counting · Interpretability · Tool use · Evaluation
Regions: Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.