Can AI-reviewed rooftop labels support solar planning?
A peer-reviewed Czech dataset pairs 5,359 aerial images with 7,402 human review outputs and one vision-language-model proposal per image. It exposes annotation variation, but every human task began from the AI proposal and the files lack engineering-grade geometry.
The short version
- What happened
- RoofeNet-Multi contains 5,359 building-centred Czech orthophoto tiles, 7,402 human review outputs and one matched Gemini proposal per image.
- Who it affects
- Homeowners, installers and local authorities should not treat AI-assisted roof boxes as a quote or planning decision. Their value today is in improving research on annotation uncertainty and review workflows.
- What remains unproven
- All images come from Czech buildings around fifteen cities, excluding Prague; rural and international architecture may differ.
Keep in mind
All images come from Czech buildings around fifteen cities, excluding Prague; rural and international architecture may differ.
Every human task started from the same model proposal, so anchoring or inherited omissions can inflate apparent human–AI agreement.
Most images have only one human output, and the 60-image manual check intentionally selected easy, high-agreement cases.
The dataset lacks per-tile georeferencing, roof polygons and independently validated tilt or energy-yield measurements. The authors declare no competing interests and report EU programme support.
Source analysis · 2 linked sources. Primary analysed source date: 24 October 2026. Earliest linked supporting source: 14 August 2026.
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Read the full report · 4 min The evidence, methods, implications and limitations
The dataset preserves disagreement—but not an independent ground truth
A new peer-reviewed dataset makes an unusually useful limitation visible: people can interpret the same roof differently. RoofeNet-Multi contains 12,761 annotation sets for 5,359 building-centred aerial tiles—7,402 human review outputs plus one vision-language-model proposal for every image. Labels cover flat or pitched roof planes and obstructions such as chimneys, vents and skylights.
However, the final article states that every human task began from the image's model proposal, which the reviewer could edit, delete or extend. The human outputs are therefore not independent of the model or of one another. That design can speed review and reveal how people modify an AI starting point, but agreement between AI and people must not be read as independent proof that either located the roof correctly.[1][2]
The Czech sample is large but uneven
The team sampled buildings within roughly 15 kilometres of the fifteen most populous Czech cities other than Prague. Each 416-by-416-pixel tile came from the national orthophoto service. Ninety supervised university students produced the human outputs; 3,857 images received one human review and 1,502 received between two and nine. Instructors spot-checked work and sent 51 image cases back for correction, but there was no numeric gold-standard screening threshold.
The matched AI labels came from Gemini 2.5 Pro preview with non-zero sampling temperature. Human-to-human mean F1 was 0.733 for roof planes and 0.604 for obstructions across 6,012 directed comparisons covering the multiply reviewed images. Against an any-human-acceptable consensus, the model's mean F1 was 0.684 for roofs and 0.491 for obstructions across all 5,359 images. These are consistency measures under chosen matching rules, not absolute accuracy. A coarse topology check found about 2.0% of human and 2.2% of model obstruction boxes failed to overlap a roof box, but overlap alone cannot show that the box is correctly placed or classified.[1][2]
The distinction that matters
What the release can and cannot support
The same files serve different purposes, but only some claims are supported by their design.
Review
People edit one AI proposal per image
Captures revision behaviour, not independent labels
Compare
Match roof and obstruction boxes
Measures agreement under selected thresholds
Plan
Estimate installable PV area or output
Needs geometry, georeferencing and field validation
Useful for research, not a solar quotation
The release can support experiments that retain several plausible labels rather than collapsing ambiguity into one supposed truth. It also gives researchers a provenance-aware test bed for measuring what changes when humans start from an AI suggestion. That is relevant to public bodies or solar developers considering assisted mapping, because annotation shortcuts can carry the model's initial omissions into the reviewed record.
The files are not ready for engineering decisions. They use axis-aligned boxes rather than roof polygons, omit per-tile georeferencing and do not validate the model's tilt estimates against surveyed geometry. The only manual validation subset contained 60 deliberately easy, high-agreement images. Although model tilt categories often agreed with human flat-versus-pitched labels, both can share the same visual ambiguity and neither is a physical measurement. Reliable deployment would require an independent expert reference sample, geographic tests beyond Czech urban and peri-urban buildings, calibrated area and shade estimates, and comparison with site surveys.[1][2]
Continue the story
Read next
Continue exploring remote sensing: compare the finding and its evidence limits.
Can open radar data train better landslide AI?
A peer-reviewed data descriptor releases 4,910 expert annotations from 92 selected radar interferograms across the Central Alps and Northern Apennines. It expands material for training slow-movement detectors, but it does not benchmark a model across all nine classes or operate as an early-warning system.
Read the next reportExplore the full source and evidence record
What this means for people
- Homeowners, installers and local authorities should not treat AI-assisted roof boxes as a quote or planning decision. Their value today is in improving research on annotation uncertainty and review workflows.
Global context
Rooftop-solar mapping is useful for grid and local-energy planning, but aerial imagery, roof styles and permitting data vary by country. A Czech annotation resource can seed research; it cannot establish worldwide model performance.
What the evidence does not yet show
- All images come from Czech buildings around fifteen cities, excluding Prague; rural and international architecture may differ.
- Every human task started from the same model proposal, so anchoring or inherited omissions can inflate apparent human–AI agreement.
- Most images have only one human output, and the 60-image manual check intentionally selected easy, high-agreement cases.
- The dataset lacks per-tile georeferencing, roof polygons and independently validated tilt or energy-yield measurements. The authors declare no competing interests and report EU programme support.
What to watch next
- Independent expert annotation before seeing an AI proposal
- Validation across countries, rural areas and roof styles
- Polygon geometry, georeferencing and measured roof tilt
- Field comparison with installer surveys and realised energy output
Living evidence record
Impact record IAI-00BHY9B
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
24 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 24 October 2026
Research question, topics and regions
Whether a multi-annotator rooftop dataset can support uncertainty-aware research on photovoltaic assessment without being mistaken for engineering ground truth
Topics: Solar energy · Remote sensing · Data annotation · Vision-language models · Human review
Regions: Czech Republic · Europe · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.