---
title: the_matrix_dataset_8M_1920_1080
canonical_url: "https://www.modelscope.cn/datasets/TheMatrixDataset/the_matrix_dataset_8M_1920_1080"
md_url: "https://www.modelscope.cn/datasets/TheMatrixDataset/the_matrix_dataset_8M_1920_1080.md"
repository: TheMatrixDataset/the_matrix_dataset_8M_1920_1080
last_updated: 2025-09-04
license: "Apache License 2.0"
storage_size: "12 TB"
downloads: 9506
stars: 1
---

# the_matrix_dataset_8M_1920_1080

> the_matrix_dataset_8M_1920_1080 - TheMatrixDataset 在 ModelScope 开源的数据集。Download The Matrix model weights at 🤗 Huggingface or 🤖 ModelScope

TheMatrixDataset/the_matrix_dataset_8M_1920_1080 是 ModelScope 魔搭社区上的数据集，存储大小 12 TB，采用 Apache License 2.0 许可。

- **Repository**: TheMatrixDataset/the_matrix_dataset_8M_1920_1080
- **License**: Apache License 2.0
- **Storage size**: 12 TB
- **Downloads**: 9506
- **Stars**: 1
- **Last updated**: 2025-09-04

Source: https://www.modelscope.cn/datasets/TheMatrixDataset/the_matrix_dataset_8M_1920_1080

---

# The Matrix

<div align="center">
  <img src="readme_src/white-logo.svg" width="50%" alt="The Matrix logo" />
</div>

<p align="center">
  Download The Matrix model weights at
  <a href="https://huggingface.co/MatrixTeam/TheMatrix" target="_blank">🤗 Huggingface</a>
  or
  <a href="https://www.modelscope.cn/models/AiurRuili/TheMatrix" target="_blank">🤖 ModelScope</a>
</p>

<p align="center">
  Download The Matrix Dataset at
  <a href="https://huggingface.co/MatrixTeam/TheMatrix" target="_blank">🤗 Huggingface</a>
  or
  <a href="https://www.modelscope.cn/models/AiurRuili/TheMatrix" target="_blank">🤖 ModelScope</a>
</p>

<p align="center">
  📚 View the
  <a href="https://arxiv.org/abs/2412.03568" target="_blank">Paper</a>,
  <a href="https://matrixteam-ai.github.io/pages/TheMatrix/" target="_blank">Website</a>,
  and
  <a href="http://matrixteam-ai.github.io/docs/TheMatrixDocs" target="_blank">Documentation</a>
</p>

<p align="center">
  👋 Say Hi to our team and members at
  <a href="https://matrixteam-ai.github.io/" target="_blank">Matrix-Team</a>
</p>

<p align="center">
  📍 (Coming Soon) Explore The Matrix playground online at <a href="">Journee</a>.
</p>

<div align="center">
  <img src="readme_src/font_30s.png" width="36%" alt="Stylized font banner" />
</div>


# Matrix Dataset

## Dataset Description
**Matrix Dataset** is a large-scale dataset designed for **interactive simulation modeling and vision-based navigation**, covering two representative open-world video game environments: **Forza Horizon 5** and **Cyberpunk 2077**. It includes high-resolution gameplay videos, fine-grained control signals, and 3D positional telemetry, making it suitable for a wide range of tasks including reinforcement learning, world modeling, and video generation.

### Dataset Overview
The dataset consists of two distinct domains:

- **Forza Horizon 5**: The dataset comprises a total of 937,900 video clips at 60 FPS, of which 262,707 include corresponding control signals. The combined duration exceeds 2861.9 hours.
- **Cyberpunk 2077**: 

### Supported Tasks
This dataset can be used for:

- World modeling
- Reinforcement and imitation learning
- Video prediction and generation
- Visual navigation and planning
- Multimodal alignment (e.g., image-control signal modeling)

## Documentation
Comprehensive documentation includes detailed installation steps, tutorials, and training instructions. The [paper](https://arxiv.org/abs/2412.03568) and [Project Page](https://matrixteam-ai.github.io/pages/TheMatrix/) offer more details about the method.


## Dataset Generation Details
### Raw Data Sources

- **Forza Horizon 5**:
  - **Stage 1** comprises 675,193 video clips (60 FPS) with an average duration of 12.8 seconds and no associated control signals.  
  - **Stage 2** comprises 208,933 clips (60 FPS) with an average duration of 6.09 seconds and a control signal at every second; on average, each clip’s control-label distribution is D 50.94 %, DR 24.67 %, DL 24.38 %, with 3.04 signal-change events per clip.  
  - **Stage 3** comprises 53,774 clips (60 FPS) with an average duration of 6.07 seconds and a control signal at every second; on average, each clip’s control-label distribution is D 51.25 %, DR 24.38 %, DL 24.37 %, with 3.02 signal-change events per clip.
- **Cyberpunk 2077**: Coming soon

### Dataset Annotations

#### Annotation Process
- **Forza Horizon 5** annotations are auto-generated using metadata logs synchronized with video and control signals.
- **Cyberpunk 2077** annotations are inferred from predefined scene types and metadata associated with the manual collection process.

#### Annotators
- **Forza Horizon 5** data is fully automated.
- **Cyberpunk 2077** data is collected by trained human annotators through remote access to cloud-hosted servers.

## Citation
If you find our work useful please consider citing:

```bibtex
@article{feng2024matrix,
  title={The matrix: Infinite-horizon world generation with real-time moving control},
  author={Feng, Ruili and Zhang, Han and Yang, Zhantao and Xiao, Jie and Shu, Zhilei and Liu, Zhiheng and Zheng, Andy and Huang, Yukun and Liu, Yu and Zhang, Hongyang},
  journal={arXiv preprint arXiv:2412.03568},
  year={2024}
}
```

## License
The Matrix Dataset is released under the Apache 2.0 License.