SQuID: Satellite Quantitative Intelligence Dataset

SQuID asks vision-language models to measure satellite images: count objects, compute areas and reason about distances. Most benchmark answers are computed from reference masks. The 50 NAIP questions use human-consensus answers.

Usage

from datasets import load_dataset

bench = load_dataset("PeterAM4/SQuID", "benchmark", split="test")

ex = bench[0]
ex["question"]   # 'What percentage of the image is agricultural land within 50m of water bodies? (GSD: 0.3m)'
ex["answer"]     # '29.32'
ex["image"]      # PIL image
ex["mask"]       # its reference mask, None for NAIP

A numeric answer is correct inside its accepted range. Other answers must match exactly:

def is_correct(pred, ex):
    lo, hi = ex["acceptable_range_lower"], ex["acceptable_range_upper"]
    if lo is None:  # yes/no (1/0), connected/fragmented
        return str(pred) == ex["answer"]
    return lo <= float(pred) <= hi

The train config holds 75,687 questions whose image and mask are paths in this repository:

from huggingface_hub import snapshot_download

snapshot_download("PeterAM4/SQuID", repo_type="dataset", local_dir="data")

Data

Source	GSD	Benchmark	Train	Reference mask
DeepGlobe	0.5 m	549	35,542	land-cover classes (_label.png)
EarthVQA	0.3 m	1,304	36,190	buildings and land cover (_label2.png)
Solar panels	0.3 m	97	3,955	panels (_label.bmp)
NAIP	1.0 m	50	—	none: median of 10 human annotators
Total		2,000	75,687

• Tiers: basic (713), spatial (610) and multi-step (677) questions of 25 types. The two splits share no images.
• Tolerances: numeric answers are accepted within the spread of 500 human annotations (±1.7 points for percentages, ±19% for counts).
• EarthVQA masks: _label2 is the human label (_label) with its unlabeled background filled by a land-cover model (_pred2). Answers are computed from _label2.

License and citation

CC BY-NC 4.0. Images keep the terms of DeepGlobe, EarthVQA, PV segmentation and NAIP.

@misc{massih2026reasoningpixellevelprecisionqvlm,
  title         = {Reasoning with Pixel-level Precision: QVLM Architecture and SQuID Dataset for Quantitative Geospatial Analytics},
  author        = {Peter A. Massih and Eric Cosatto},
  year          = {2026},
  eprint        = {2601.13401},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2601.13401}
}