---
title: OVO-S-Bench
canonical_url: "https://www.modelscope.cn/datasets/JoeLeelyf/OVO-S-Bench"
md_url: "https://www.modelscope.cn/datasets/JoeLeelyf/OVO-S-Bench.md"
repository: JoeLeelyf/OVO-S-Bench
last_updated: 2026-06-24
license: mit
storage_size: "224 GB"
downloads: 62726
stars: 2
---

# OVO-S-Bench

> OVO-S-Bench - JoeLeelyf 在 ModelScope 开源的数据集。OVO-S-Bench is a hierarchical benchmark for streaming spatial intelligence in multimodal LLMs — 1,695 multiple-choice questions across 4 difficulty levels, sampled from 9 public video sources.

JoeLeelyf/OVO-S-Bench 是 ModelScope 魔搭社区上的数据集，存储大小 224 GB，采用 mit 许可。

- **Repository**: JoeLeelyf/OVO-S-Bench
- **License**: mit
- **Storage size**: 224 GB
- **Downloads**: 62726
- **Stars**: 2
- **Last updated**: 2026-06-24

Source: https://www.modelscope.cn/datasets/JoeLeelyf/OVO-S-Bench

---

# OVO-S-Bench

OVO-S-Bench is a hierarchical benchmark for **streaming spatial intelligence**
in multimodal LLMs — 1,695 multiple-choice questions across 4 difficulty
levels, sampled from 9 public video sources.

Evaluation framework: <https://github.com/InternLM/OVO-S-Bench>

## Repository layout

```
ovo_s_bench_l1_l4.parquet     # 1695 questions, ~35 MB
videos/
├── Ego4D/         *.mp4           # 71 clips,  ~88 GiB
├── annotated_videos/
│   ├── 01_CODa_full/   *.mp4
│   ├── 02_Ego4D/       *.mp4
│   ├── 03_OmniWorld/   *.mp4
│   ├── 04_RoomTour3D/  *.mp4
│   ├── honda_videos_for_modelscope/  *.mp4
│   └── RoomTour3D/     *.mp4
├── Sekai/         *.mp4           # YouTube travelogue clips
├── RoomTour3D/    *.mp4
├── CODa_full/     *.mp4
├── OmniWorld/     *.mp4
├── VSI-Bench/     *.mp4
├── honda/         *.mp4           # L4.3 trajectory matching source
├── arkitscenes/   *.mp4           # L4.3 trajectory matching source
└── edited_videos/ *.mp4           # L3.2 edited clips
```

339 unique video files, ~219 GiB total.

## Schema

| column                | type                   | notes                                                              |
| --------------------- | ---------------------- | ------------------------------------------------------------------ |
| `id`                  | string                 | L1-L3: `{subcat}_{idx}` (e.g. `1.1.1_0`); L4: `{main}_{NNN}`        |
| `source_dataset`      | string                 | Ego4D / RoomTour3D / CODa_full / OmniWorld / VSI-Bench / Sekai / YouTube / Honda / ARKitScenes |
| `video_id`            | string                 | Stable across releases                                             |
| `video_path`          | string                 | Relative to `videos/`                                              |
| `level`               | int (1-4)              |                                                                    |
| `task_main_category`  | string (e.g. `1.1`)    |                                                                    |
| `task_subcategory`    | string                 | L1-L3: dotted (`1.1.1`); L4: main_cat (`4.1`)                       |
| `task_type_name`      | string                 | Human-readable task name                                            |
| `question`            | string                 | English-translated                                                  |
| `options`             | list<[label, content]> | 2-7 options. L4.3 contents are `data:image/png;base64,...` URIs    |
| `query_times`         | list<float>            | Seconds from video start                                            |
| `evidence_times`      | list<[float, float]>   | Each evidence interval `[start, end]` in seconds                    |
| `answers`             | list<string>           | Correct option label(s)                                             |

## Task taxonomy

| Level | Subcategories | Count |
| ----- | ------------- | ----- |
| L1 — Spatial perception | 1.1.x metric / 1.2.x topological / 1.3.x dynamic | 629 |
| L2 — Scene understanding | 2.1.x recognition / 2.2.x memory / 2.3.x temporal | 513 |
| L3 — Spatial-temporal multi-hop reasoning | 3.1.x / 3.2.x / 3.3.x | 279 |
| L4 — Compositional reasoning | 4.1 allocentric direction / 4.2 topological structure / 4.3 trajectory map matching (image options) | 274 |

L4.3 questions ship their option images **embedded inline** as base64 data
URIs in the `options` column. No separate image asset download needed.

## Quick load (datasets library)

```python
from datasets import load_dataset
ds = load_dataset("InternLM/OVO-S-Bench", split="train")  # 1695 rows
print(ds[0])
```

## Quick load (pandas)

```python
import pandas as pd
df = pd.read_parquet("ovo_s_bench_l1_l4.parquet")
print(df.iloc[0])
```

## Evaluation framework

The reference inference + scoring code:

```bash
pip install -r requirements.txt
python inference.py --model gpt-4o --annotation ovo_s_bench_l1_l4.parquet
python score.py --result results/gpt-4o/ovo_s_bench_l1_l4.json
```

Full setup at <https://github.com/InternLM/OVO-S-Bench>.

## Licensing

- Annotations (this parquet): CC-BY-4.0
- Source videos: retain their respective licenses — Ego4D (Ego4D License),
  RoomTour3D (CC-BY-NC), CODa (Apache-2.0), OmniWorld (CC-BY-NC),
  VSI-Bench (mixed: ScanNet++ / ScanNet / ARKitScenes), Sekai (CC-BY),
  ARKitScenes (Apple ARKit ML), Honda HRD (CC-BY-NC).

## Citation

```bibtex
@misc{ovosbench2026,
  title  = {OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs},
  author = {OVO-S-Bench Team},
  year   = {2026},
  note   = {arXiv:TBD}
}
```