|
|
|
Ultra-High-Resolution Remote Sensing Understanding and Reasoning
🤗 Hugging Face · 🤖 ModelScope · 📄 Paper · 💻 Code
--- > **Mirror status (2026-09-16):** This ModelScope mirror is incomplete. `data/test-00044-of-00197.parquet` is withheld pending resolution of a source-file integrity mismatch. Statistics below describe the original Hugging Face release; do not treat this mirror as a complete evaluation set. Use the [Hugging Face release](https://huggingface.co/datasets/RL-MIND/RSHR-Bench) for the complete package. ## 📚 Introduction **RSHR-Bench** evaluates ultra-high-resolution remote-sensing visual understanding and reasoning in multimodal large language models across single-image, multi-image, and multi-turn settings. This release embeds original-resolution images directly in Parquet shards for use with Hugging Face Datasets, without a separate `images/` directory or local image-path resolution. The benchmark covers real-world scenes from public remote-sensing datasets and UAV imagery, including object-level, region-level, temporal, and anomaly-oriented understanding. It accompanies [A Benchmark for Ultra-High-Resolution Remote Sensing MLLMs](https://arxiv.org/abs/2512.17319). [XHRBench](https://huggingface.co/datasets/RL-MIND/XHRBench) belongs to the same benchmark project and uses a different package layout. ## 🎯 Release Statistics | Item | Original release | | --- | ---: | | Examples | 1,932 | | Split | `test` | | Parquet shards | 197 | | Parquet size | 210.14 GiB (225,641,067,559 bytes) | | Data language | English | | One-image examples | 1,862 | | Two-image examples | 50 | | Three-image examples | 20 | These figures describe this Parquet package, as documented in the package manifest and original dataset card; they are distinct from image totals for the full benchmark in the paper. The Hugging Face Parquet release with embedded ultra-high-resolution images was published on **2026-05-19**. ## 🌟 Task Coverage - **Multiple-choice VQA:** Visual decisions within a fixed answer space. - **Open-ended VQA:** Free-form visual understanding and compositional reasoning. - **Regional and object-level understanding:** Grounding, counting, classification, object-state judgment, relations, color, and shape. - **Multi-image and multi-turn reasoning:** Temporal change, future prediction, anomaly detection, and contrastive reasoning across regions or images. These describe the benchmark's overall task coverage; the fields and category distribution of this package are detailed below. Answers should be grounded in visual evidence. See the [paper](https://arxiv.org/abs/2512.17319) and [code](https://github.com/Yunkaidang/RSHR) for the evaluation protocol. ## 📦 Data Format ```text data/test-*.parquet # Parquet shards with embedded images annotations/*.json # Original task annotations vqa_summary.json # Merged annotation summary manifests/hf_package_summary.json # Package statistics assets/ # Dataset-card figures and logos ``` Images are stored in Hugging Face `Image()` columns at their original resolution, without resizing. | Field | Description | | --- | --- | | `id` | Original sample ID from `vqa_summary.json` | | `turn_id` | Turn ID for multi-turn samples, when available | | `category` | Task category | | `question` | Question text | | `option_a`, `option_b`, `option_c`, `option_d` | Multiple-choice options | | `options` | JSON string containing the original option mapping | | `answer` | Gold answer key | | `image_count` | Number of images used by the sample | | `image_1`, `image_2`, `image_3` | Embedded image columns | | `image_1_path`, `image_2_path`, `image_3_path` | Original relative image paths | | `source_images` | List of original relative image paths | | `source_image_sizes` | Original image byte sizes | | `ok` | Generation status, when available | | `xml` | Source XML path, when present | ### Category distribution Category identifiers are preserved exactly as released, including their original spelling. | Category (original identifier) | Examples | | --- | ---: | | `anomaly_detection` | 50 | | `color` | 200 | | `detection` | 100 | | `future_prediction_two_image` | 50 | | `multi_region_join_contrast` | 20 | | `multi_region_join_contrast_singal` | 50 | | `multitrun-future_prediction` | 150 | | `multiturn_object_state_judgement` | 132 | | `multturn_anomaly_detection` | 60 | | `object_clasfication` | 200 | | `object_counting` | 200 | | `object_grounding` | 200 | | `object_state_judgement` | 50 | | `regional_grouding` | 70 | | `regionla_counting` | 100 | | `relation` | 200 | | `shape` | 100 | | **Total** | **1,932** | ## 🚀 Quick Start After installing the required dependencies, stream a sample for an initial inspection. The following example disables image decoding; streaming still transfers the necessary shard or image bytes and is not a text-metadata-only download. ```python from datasets import Image, load_dataset # Stream a sample and keep image bytes undecoded during inspection. ds = load_dataset("RL-MIND/RSHR-Bench", split="test", streaming=True) for column in ("image_1", "image_2", "image_3"): ds = ds.cast_column(column, Image(decode=False)) first = next(iter(ds)) print(first["id"], first["category"], first["question"]) ``` Loading the full split downloads approximately **210.14 GiB** of Parquet data. Allow additional resources for caching and image decoding. ```python from datasets import load_dataset ds = load_dataset("RL-MIND/RSHR-Bench", split="test") print(ds[0]["question"]) print(ds[0]["image_1"]) ``` ### ModelScope mirror metadata The mirror currently lacks one shard; use the Hugging Face examples above for complete evaluation. The following downloads only the manifest. Its statistics describe the original release and do not certify mirror completeness. ```python from pathlib import Path from modelscope.hub.snapshot_download import dataset_snapshot_download # Inspect the package manifest without downloading image shards. local_dir = Path(dataset_snapshot_download( dataset_id="KAIWANG/RSHR-Bench", revision="master", allow_file_pattern=["manifests/hf_package_summary.json"], )) print((local_dir / "manifests/hf_package_summary.json").read_text()) ``` ## 📊 Evaluation Figures The original overview, leaderboard, and resolution-robustness analysis figures are retained below. Results correspond to the original release's evaluation configuration; see the paper for details.  ### Evaluation figures 
