---
title: Orsta-Data-47k
canonical_url: "https://www.modelscope.cn/datasets/One-RL-to-See-Them-All/Orsta-Data-47k"
md_url: "https://www.modelscope.cn/datasets/One-RL-to-See-Them-All/Orsta-Data-47k.md"
repository: One-RL-to-See-Them-All/Orsta-Data-47k
last_updated: 2025-06-04
license: "Apache License 2.0"
storage_size: "14 GB"
downloads: 854
stars: 0
---

# Orsta-Data-47k

> Orsta-Data-47k - One-RL-to-See-Them-All 在 ModelScope 开源的数据集。Orsta-Data-47k Dataset

One-RL-to-See-Them-All/Orsta-Data-47k 是 ModelScope 魔搭社区上的数据集，存储大小 14 GB，采用 Apache License 2.0 许可。

- **Repository**: One-RL-to-See-Them-All/Orsta-Data-47k
- **License**: Apache License 2.0
- **Storage size**: 14 GB
- **Downloads**: 854
- **Stars**: 0
- **Last updated**: 2025-06-04

Source: https://www.modelscope.cn/datasets/One-RL-to-See-Them-All/Orsta-Data-47k

---

# Orsta-Data-47k Dataset

* 🐙 **GitHub Repo:** [MiniMax-AI/One-RL-to-See-Them-All](https://github.com/MiniMax-AI/One-RL-to-See-Them-All)
* 📜 **Paper (arXiv):** [V-Triune: One RL to See Them All (arXiv:2505.18129)](https://arxiv.org/abs/2505.18129)

## Dataset Description 📖

**Orsta-Data-47k** is a specialized dataset curated for the post-training of Vision-Language Models (VLMs) using our [V-Triune](https://github.com/MiniMax-AI/One-RL-to-See-Them-All) unified reinforcement learning system. Its primary purpose is to enable robust joint training across a diverse spectrum of both visual reasoning and visual perception tasks, powering models like [Orsta](https://huggingface.co/collections/One-RL-to-See-Them-All/one-rl-to-see-them-all-6833d27abce23898b2f9815a) to achieve advanced multimodal capabilities.

This dataset is a carefully selected aggregation from 18 publicly available datasets, refined through a rigorous filtering process to ensure high quality and suitability for RL-based fine-tuning.

## Tasks Covered 🎯

The dataset is structured to cover eight principal task categories, balanced between reasoning and perception:

* **Visual Reasoning Tasks 🤔:**
    * Mathematics (Math QA)
    * Puzzle Solving (Visual Puzzles)
    * Science Question Answering (Science QA)
    * Chart Understanding (Chart QA)
* **Visual Perception Tasks 👁️:**
    * Object Detection
    * Visual Grounding
    * Object Counting
    * Optical Character Recognition (OCR)

## Data Curation Process 🛠️

To create a high-quality corpus for effective RL post-training, we implemented a comprehensive two-stage data curation pipeline:

1.  **Rule-based Filtering:** An initial filtering stage applied a set of predefined rules to the source datasets. These rules were tailored differently for reasoning and perception tasks, aiming to remove noisy samples, questions prone to "hacking" (e.g., certain multiple-choice formats), and problematic answer formats. For perception tasks, this also involved standardizing coordinate systems and filtering based on object size or count.
2.  **Difficulty-based Filtering:** Following rule-based cleaning, a difficulty filter was applied. This stage removed samples deemed too easy (e.g., already solvable by baseline models) or excessively hard, ensuring that the remaining data provides a meaningful and efficient learning signal for the models.

This meticulous process resulted in a high-quality collection of approximately **47,700 samples**. To address potential dataset imbalances, data for certain tasks (e.g., puzzles) was strategically duplicated to ensure adequate representation.

## Dataset Composition & Structure 📊

* **Total Samples:** ~47.7K
* **Task Categories:** 8 (4 reasoning, 4 perception)
* **Aggregated From:** 18 distinct public datasets
* **Content Breakdown:**
    * Visual Perception Samples: ~20.6K
    * Visual Reasoning Samples: ~27.1K
* **Interaction Format:** The data primarily consists of single-image, single-turn conversational interactions (e.g., an image paired with a question and its corresponding answer/grounding).
* **Storage Format:** All curated data is stored in the efficient Parquet format.

## Intended Use & Training 🚀

This dataset is designed for use with the [V-Triune](https://github.com/MiniMax-AI/One-RL-to-See-Them-All) framework for reinforcement learning-based post-training of VLMs. In the training of [Orsta](https://huggingface.co/collections/One-RL-to-See-Them-All/one-rl-to-see-them-all-6833d27abce23898b2f9815a) models, all samples from this dataset were uniformly mixed and utilized.

## Dataset Usage

This section outlines how to download and use the Orsta-Data-47k dataset.

### Downloading the Dataset

You can download the dataset directly from the Hugging Face Hub using the `huggingface-cli` tool. Make sure you have `huggingface_hub` installed (`pip install huggingface_hub`).

Execute the following command in your terminal:

```bash
huggingface-cli download --repo-type dataset --resume-download One-RL-to-See-Them-All/Orsta-Data-47k --local-dir Orsta-Data-47k
```

This command will download all dataset files into a local directory named `Orsta-Data-47k`. The `--resume-download` flag is useful for resuming downloads if interrupted.

### Dataset Structure

Once downloaded, the dataset will have the following structure within the `Orsta-Data-47k` directory. All data files are in the Parquet (`.parquet`) format.

```
Orsta-Data-47k/
├── test/
│   ├── test_chart_megabench_176.parquet
......
│   └── test_science_megabench_91.parquet
└── train/
    ├── train_chart_chartqapro_498.parquet
......
    └── train_science_virl39k_2539.parquet
```

### File Naming Convention

The files within the `train/` and `test/` directories follow this naming convention:

`{split}_{task_name}_{source_name}_{num}.parquet`

Where:
* `{split}`: Indicates the dataset split, either `train` or `test`.
* `{task_name}`: Specifies the general task category.
* `{source_name}`: Denotes the specific benchmark or origin of the data.
* `{num}`: Represents the number of samples contained within that Parquet file.

### Purpose of Each Split

* **`train/` directory**: These files constitute the training corpus for the Orsta model.
* **`test/` directory**: These files contain samples specifically curated for online evaluation of the model's performance on different tasks *during* the training process. Analyzing performance on these samples helps in diagnosing the training status and understanding the model's learning progression for each task category.

### Data Format

```python
{
    'data_source': Value(dtype='string', id=None),
    'images': Sequence(feature=Image(mode=None, decode=True, id=None), length=-1, id=None),
    'prompt': [{'content': Value(dtype='string', id=None), 'role': Value(dtype='string', id=None)}],
    'ability': Value(dtype='string', id=None),
    'reward_model': {
        'answer': Value(dtype='string', id=None),
        'ground_truth': Value(dtype='string', id=None),
        'accuracy_ratio': Value(dtype='float32', id=None),
        'format_ratio': Value(dtype='float32', id=None),
        'verifier': Value(dtype='string', id=None),
        'verifier_parm': {
            'det_verifier_normalized': Value(dtype='bool', id=None),
            'det_reward_ratio': {
                'iou_max_label_first': Value(dtype='float32', id=None),
                'iou_max_iou_first': Value(dtype='float32', id=None),
                'iou_completeness': Value(dtype='float32', id=None),
                'map': Value(dtype='float32', id=None),
                'map50': Value(dtype='float32', id=None),
                'map75': Value(dtype='float32', id=None)
            }
        }
    },
    'extra_info': {'id': Value(dtype='string', id=None), 'image_path': Value(dtype='string', id=None)}
}
```

## 📊 Data Sources and Composition

The **Orsta-Data-47k** dataset is constructed entirely from publicly available, open-source datasets. These have been aggregated and curated to create a collection suitable for VLM post-training on both visual reasoning and perception tasks.

Orsta-Data-47k is compiled from 18 distinct public datasets. The primary contributing sources for each task category are as follows:

* **Math**: [mm_math](https://huggingface.co/datasets/THU-KEG/MM_Math), [geometry3k](https://huggingface.co/datasets/hiyouga/geometry3k), [mmk12](https://huggingface.co/datasets/FanqingM/MMK12)
* **Puzzle**: [PuzzleVQA](https://huggingface.co/datasets/declare-lab/PuzzleVQA), [AlgoPuzzleVQA](https://huggingface.co/datasets/declare-lab/AlgoPuzzleVQA), [VisualPuzzles](https://huggingface.co/datasets/neulab/VisualPuzzles)
* **Science**: [ScienceQA](https://huggingface.co/datasets/lmms-lab/ScienceQA), [SciVQA](https://huggingface.co/datasets/katebor/SciVQA), [ViRL39K-Science](https://huggingface.co/datasets/TIGER-Lab/ViRL39K)
* **Chart**: [ChartQAPro](https://huggingface.co/datasets/ahmed-masry/ChartQAPro), [ChartX](https://huggingface.co/datasets/U4R/ChartX), [Table-VQA-Bench](https://huggingface.co/datasets/terryoo/TableVQA-Bench), [ViRL39K-Chart](https://huggingface.co/datasets/TIGER-Lab/ViRL39K)
* **Detection**: [V3Det](https://arxiv.org/abs/2307.12813), [Object365](https://www.objects365.org/overview.html)
* **Grounding**: [D^3](https://arxiv.org/abs/2307.12813)
* **Counting**: [CLEVR](https://arxiv.org/abs/1612.06890)
* **OCR**: [LLaVA-OV Data](https://huggingface.co/datasets/lmms-lab/LLaVA-OneVision-Data), [EST-VQA](https://arxiv.org/abs/2002.10215)

For detailed information and licensing for each source dataset, please refer to their original publications and repositories. Our specific aggregation and curation methodology for Orsta-Data-47k is described in our paper: [V-Triune: One RL to See Them All (arXiv:2505.18129)](https://arxiv.org/abs/2505.18129).

## Citation Information 📜

If you use the Orsta-Data-47k dataset or our V-Triune framework in your research, please cite our accompanying paper:

```bibtex
@article{ma2025one,
      title={One RL to See Them All: Visual Triple Unified Reinforcement Learning}, 
      author={Ma, Yan and Du, Linge and Shen, Xuyang and Chen, Shaoxiang and Li, Pengfei and Ren, Qibing and Ma, Lizhuang and Dai, Yuchao and Liu, Pengfei and Yan, Junjie},
      journal={arXiv preprint arXiv:2505.18129},
      year={2025}
}
```