|
|
|
🤗 Hugging Face · 🟣 ModelScope · 📊 Statistics
English | 中文：Hugging Face · ModelScope
## 📚 Introduction **NJU-HARD** is the deduplicated, full-resolution release of the HARD visual question answering data. It contains **1,563 valid VQA records** across **8 task types**, using **854 original aerial images**. Only images referenced by these valid questions are included. Every original JPEG is embedded **once** in a native Hugging Face Parquet image table. Questions refer to images by their full SHA-256 IDs, in the original Image 1 / Image 2 / Image 3 order. The release supports single-image and multi-image VQA without storing the same original repeatedly for different questions. This release packages the question table and original-image table together for distribution on Hugging Face and ModelScope. It is separate from the per-question embedded-image `HARD-VQA` version. ## 📊 Dataset at a Glance | Item | Value | |:---|:---| | Valid questions | 1,563 | | Unique original images | 854 distinct paths and SHA-256 hashes | | Ordered image references | 2,571 | | Images per question | 1, 2, or 3 | | Original resolution | 12,768 × 9,564 pixels | | Original JPEG bytes | 87,637,348,481 bytes, approximately 87.64 GB | | Original-image Parquet shards | 180 | | All Parquet files | 87,639,068,405 bytes, including questions and thumbnail preview | | Split | `unsplit`; no new train/validation/test split is introduced | | Preview | 8 examples, one per task, with derived thumbnails | Original images are **not resized, cropped, or re-encoded**. Thumbnail images are separate browsing aids and are not additional original training or evaluation images. | Source task label | Questions | |:---|---:| | `classification` | 250 | | `counting` | 200 | | `property` | 154 | | `spatial_relation` | 100 | | `task1_specialdetect` | 66 | | `task3_region_single` | 151 | | `task4_findit_nearbox` | 342 | | `task_2_relocate` | 300 | | **Total** | **1,563** | ## 📦 Data Organization | Configuration | Split | Contents | |:---|:---|:---| | `questions` **(default)** | `unsplit` | Question text, options, answers, ordered `image_ids`, and source annotations | | `images` | `unsplit` | One original image per row, keyed by `image_id`; JPEG bytes embedded as `Image()` | | `preview` | `sample` | Eight VQA examples with thumbnails for browsing | ```text questions/ # 1,563 question records images/ # 854 originals in 180 Parquet shards preview/ # 8 thumbnail examples provenance/ # source/checksum manifests, image index, and validation reports statistics.json ``` The question table preserves `question_id`, `question`, `options`, `answer`, `answer_text`, `task_type`, `num_images`, `source_image_paths`, `reference_bbox`, `scene`, `sequence`, and `quality_flags`. The `options` struct has A/B/C/D fields; C/D are null for binary-choice questions. Ordered `image_ids` correspond one-to-one with `source_image_paths`. The image table contains `image_id`, `image`, `source_image_paths`, `sha256`, `width`, `height`, and `byte_count`. `image_id` equals the SHA-256 of the complete original JPEG. Source paths are provenance only: loading does not require the source server or an external image directory. ## 🚀 Quick Start Load the small question table first, or inspect the thumbnail preview: ```python from datasets import load_dataset REPO = "RL-MIND/NJU-HARD" questions = load_dataset(REPO, "questions", split="unsplit") preview = load_dataset(REPO, "preview", split="sample") print(questions[0]["question"], questions[0]["image_ids"]) ``` To reconstruct a VQA example with original images: ```python from datasets import Image, load_dataset images = load_dataset(REPO, "images", split="unsplit") images = images.cast_column("image", Image(decode=False)) # Store only IDs and row numbers in the lookup, not all image bytes. id_to_row = {image_id: i for i, image_id in enumerate(images["image_id"])} question = questions[0] for image_number, image_id in enumerate(question["image_ids"], start=1): jpeg_bytes = images[id_to_row[image_id]]["image"]["bytes"] # Process this original; iteration preserves the question's image order. print(image_number, image_id, len(jpeg_bytes)) ``` Loading `questions` alone does not download the originals. A full `images` load downloads approximately 87 GB and builds a local cache; allow additional cache space. Datasets **3.6.0 and 5.0.0** have been checked with this format. After downloading a complete repository snapshot from **either platform**, local Parquet loading is also supported: ```python from pathlib import Path from datasets import load_dataset root = Path("/path/to/NJU-HARD") questions = load_dataset("parquet", data_files={ "unsplit": str(root / "questions" / "*.parquet") }, split="unsplit") images = load_dataset("parquet", data_files={ "unsplit": str(root / "images" / "*.parquet") }, split="unsplit") ``` Sequential streaming avoids downloading the entire image table up front: ```python from datasets import Image, load_dataset stream = load_dataset(REPO, "images", split="unsplit", streaming=True) stream = stream.cast_column("image", Image(decode=False)) first = next(iter(stream)) print(first["image_id"], len(first["image"]["bytes"])) ``` Streaming is sequential and does not provide random lookup by image ID. For repeated question-to-image joins, use the cached dataset and row-number index above. ## 🔍 Validation and Annotation Notes - The source has 1,565 records. IDs **476 and 496** have null questions, options, and answers and are excluded. Images used only by excluded records are not included. - ID **597** is preserved with the flag `duplicate_option_text`: A and B both contain `white`, while the source answer is B. - Question text, options, answers, ordered image references, and bounding boxes preserve source values. Structural validation does not establish that every answer is semantically correct. - Every original JPEG is checked by SHA-256 before embedding and after Parquet readback. Independent validation checks all question fields, image metadata, reference closure, and image-index rows, plus sampled original bytes through both Datasets versions. Reports are in `provenance/`. - Preview images have a 1,024-pixel longest edge and `preview_only=true`. The ordered `images` list and `image_1`/`image_2`/`image_3` columns support browsing; missing slots are null. Preview questions and boxes still use original-image coordinates. Use the originals for evaluation. - The source provides no train/validation/test assignment, so all records remain `unsplit`. If you create a split, account for shared images and adjacent frames across partitions. ## 📜 License and Provenance The source data does not include an explicit license. The dataset card therefore retains **`license: unknown`**. Packaging the data does not grant a new data-use license; consult the data owner for applicable terms. Source-image checksums, excluded records, the image-to-shard index, and validation evidence are provided in `provenance/`. The paired header uses the supplied RL-MIND group mark and a newly generated NJU-HARD dataset logo.