---
title: WBench
canonical_url: "https://www.modelscope.cn/datasets/meituan-longcat/WBench"
md_url: "https://www.modelscope.cn/datasets/meituan-longcat/WBench.md"
repository: meituan-longcat/WBench
last_updated: 2026-06-01
license: mit
storage_size: "1017 MB"
downloads: 87455
stars: 32
---

# WBench

> WBench - meituan-longcat 在 ModelScope 开源的数据集。WBench Dataset A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

meituan-longcat/WBench 是 ModelScope 魔搭社区上的数据集，存储大小 1017 MB，采用 mit 许可。

- **Repository**: meituan-longcat/WBench
- **License**: mit
- **Storage size**: 1017 MB
- **Downloads**: 87455
- **Stars**: 32
- **Last updated**: 2026-06-01

Source: https://www.modelscope.cn/datasets/meituan-longcat/WBench

---

<div align="center">
  <img src="assets/longcat-logo-full.png" width="300">
  <h1>WBench Dataset</h1>
  <p><i>A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation</i></p>

[![Paper](https://img.shields.io/badge/Paper-red?style=for-the-badge&logo=arxiv&logoColor=white)](https://huggingface.co/papers/2605.25874)
[![Code](https://img.shields.io/badge/Code-black?style=for-the-badge&logo=github&logoColor=white)](https://github.com/meituan-longcat/WBench)
[![Weights](https://img.shields.io/badge/Weights-FF9D00?style=for-the-badge&logo=huggingface&logoColor=white)](https://huggingface.co/meituan-longcat/WBench-weights)
[![ModelScope](https://img.shields.io/badge/ModelScope-6B4EFF?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyBmaWxsPSJ3aGl0ZSIgZmlsbC1ydWxlPSJldmVub2RkIiBoZWlnaHQ9IjFlbSIgc3R5bGU9ImZsZXg6bm9uZTtsaW5lLWhlaWdodDoxIiB2aWV3Qm94PSIwIDAgMjQgMjQiIHdpZHRoPSIxZW0iIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHRpdGxlPk1vZGVsU2NvcGU8L3RpdGxlPjxwYXRoIGQ9Ik0yLjY2NyA1LjNIOHYyLjY2N0g1LjMzM3YyLjY2NkgyLjY2N1Y4LjQ2N0guNXYyLjE2NmgyLjE2N1YxMy4zSDBWNy45NjdoMi42NjdWNS4zek0yLjY2NyAxMy4zaDIuNjY2djIuNjY3SDh2Mi42NjZIMi42NjdWMTMuM3pNOCAxMC42MzNoMi42NjdWMTMuM0g4di0yLjY2N3pNMTMuMzMzIDEzLjN2Mi42NjdoLTIuNjY2VjEzLjNoMi42NjZ6TTEzLjMzMyAxMy4zdi0yLjY2N0gxNlYxMy4zaC0yLjY2N3oiPjwvcGF0aD48cGF0aCBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0yMS4zMzMgMTMuM3YtMi42NjdoLTIuNjY2VjcuOTY3SDE2VjUuM2g1LjMzM3YyLjY2N0gyNFYxMy4zaC0yLjY2N3ptMC0yLjY2N0gyMy41VjguNDY3aC0yLjE2N3YyLjE2NnoiPjwvcGF0aD48cGF0aCBkPSJNMjEuMzMzIDEzLjN2NS4zMzNIMTZ2LTIuNjY2aDIuNjY3VjEzLjNoMi42NjZ6Ij48L3BhdGg+PC9zdmc+&logoColor=white)](https://modelscope.cn/datasets/meituan-longcat/WBench)
[![Homepage](https://img.shields.io/badge/Homepage-blue?style=for-the-badge&logo=google-chrome&logoColor=white)](https://meituan-longcat.github.io/WBench/)

</div>

---

<div align="center">
  <img src="assets/teaser.png" width="90%">
</div>

<p align="center" style="color: grey;">
<b>TL;DR</b> — WBench evaluates 20 video world models across 5 dimensions and 22 metrics.
</p>

## Overview

WBench is a comprehensive multi-turn benchmark for interactive video world model evaluation. It contains **289 multi-turn interaction cases** with 1,058 interaction turns for evaluating models across **22 metrics** and **5 dimensions**:
1. Video Quality
2. Setting Adherence
3. Interaction Adherence
4. Consistency
5. Physics Compliance

The benchmark covers diverse scenes, styles, subjects, and both first- and third-person perspectives, with four interaction types: navigation, subject action, event editing, and perspective switching.

<div align="center">
  <img src="assets/distribution.png" width="90%">
</div>

## Usage

### Download Dataset
You can download the dataset using the Hugging Face CLI:

```bash
pip install huggingface_hub
huggingface-cli download meituan-longcat/WBench --repo-type dataset --local-dir data/ --exclude "splits/*"
```

### Framework Usage
To generate multi-turn videos from a case using the WBench framework (as implemented in the [GitHub repository](https://github.com/meituan-longcat/WBench)):

```python
from src.models import get_model

# Available: wan, kling, seedance (or register your own)
model = get_model("wan")

# Generate multi-turn video from a case
# case_dict should be loaded from the 'cases/' directory
result = model.generate_multi_turn(
    case=case_dict,
    output_path="work_dirs/wan/videos/case_1_combined.mp4",
    data_root="data/",
)
```

## Structure

```
├── cases/       # 289 case JSONs (prompts + interactions + annotations)
├── images/      # 289 first-frame images
├── masks/       # 188 subject segmentation masks
```

## 🏆 Leaderboard

**20 Models — Navigation Split (5 Dimensions, sorted by average)**

| # | Model | **Average** | Quality | Setting | Interaction | Consistency | Physical |
|:---:|:---|:---:|:---:|:---:|:---:|:---:|:---:|
| 1 | Kling 3.0 | **79.2 🥇** | 83.0 🥈 | 91.0 🥈 | 70.3 | 82.5 | 69.3 🥉 |
| 2 | LingBot-World | **78.8 🥈** | 81.5 | 72.6 | 79.8 | 88.9 🥇 | 71.2 🥈 |
| 3 | Wan 2.7 | **78.5 🥉** | 82.6 🥉 | 91.4 🥇 | 66.0 | 80.5 | 71.8 🥇 |
| 4 | HY-World 1.5 | **78.4** | 80.2 | 72.2 | 87.5 🥇 | 86.0 | 66.3 |
| 5 | HY-Video 1.5 | **78.2** | 79.7 | 85.6 🥉 | 71.8 | 86.7 🥉 | 67.4 |

*Detailed leaderboard results and metric breakdowns are available on the [Project Page](https://meituan-longcat.github.io/WBench/#leaderboard).*

## Citation

```bibtex
@article{ying2026wbenchcomprehensivemultiturnbenchmark,
  title={WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation},
  author={Ying, Kaining and Hu, Hengrui and Ren, Siyu and Li, Jiamu and Chen, Fengjiao and Wang, Ziwen and Cao, Xuezhi and Cai, Xunliang and Ding, Henghui},
  journal={arXiv preprint arXiv:2605.25874},
  year={2026}
}
```