---
title: ReasonFlux-V2-Reasoner-DPO
canonical_url: "https://www.modelscope.cn/datasets/Gen-Verse/ReasonFlux-V2-Reasoner-DPO"
md_url: "https://www.modelscope.cn/datasets/Gen-Verse/ReasonFlux-V2-Reasoner-DPO.md"
repository: Gen-Verse/ReasonFlux-V2-Reasoner-DPO
last_updated: 2025-08-07
license: "Apache License 2.0"
storage_size: "28 MB"
downloads: 369
stars: 0
---

# ReasonFlux-V2-Reasoner-DPO

> ReasonFlux-V2-Reasoner-DPO - Gen-Verse 在 ModelScope 开源的数据集。ReasonFlux-V2 is our new template-augmented reasoning paradigm which internalize the thought templates through iterative hierarchical reinforcement learning. Specifically, we first develop an…

Gen-Verse/ReasonFlux-V2-Reasoner-DPO 是 ModelScope 魔搭社区上的数据集，存储大小 28 MB，采用 Apache License 2.0 许可。

- **Repository**: Gen-Verse/ReasonFlux-V2-Reasoner-DPO
- **License**: Apache License 2.0
- **Storage size**: 28 MB
- **Downloads**: 369
- **Stars**: 0
- **Last updated**: 2025-08-07

Source: https://www.modelscope.cn/datasets/Gen-Verse/ReasonFlux-V2-Reasoner-DPO

---

**ReasonFlux-V2** is our new template-augmented reasoning paradigm which **internalize the thought templates** through **iterative hierarchical reinforcement learning**.  Specifically, we first develop an automated pipeline to extract thought templates from the problem–solution pairs in training set. To effectively internalize these high-level thought templates and learning a more efficient reasoning paradigm,  we propose two collaborative modules: **Template Proposer**  which adaptively proposes suitable thought templates based on the input problem; and **Template Reasoner**,which exactly instantiates the proposed templates and performs precise, detailed reasoning. Building upon these modules, we iteratively conduct **hierarchical RL** on optimizing both modules. 

ReasonFlux-v2 consists of two main modules:

1. **Template Proposer**, which **adaptively** proposes suitable high-level thought templates based on the input problem. It functions as intuitive thinking process of human which helps to **narrow the exploration space** of detailed reasoning process thus **improve the solution efficiency**.
2. **Template Reasoner**, which follow the proposed high-level thought template to efficiently and effectively solve the corresponding problem.

**This dataset is the DPO dataset for Template Reasoner,** the rest of the models and dataset are available below:

[Code](https://github.com/Gen-Verse/ReasonFlux)|[Template](Gen-Verse/ReasonFlux-V2-Template)|[SFT Dataset](https://huggingface.co/datasets/Gen-Verse/ReasonFlux-V2-SFT/) |[DPO Dataset (Proposer)](https://huggingface.co/datasets/Gen-Verse/ReasonFlux-V2-DPO)|[DPO Dataset (Reasoner)](https://huggingface.co/datasets/Gen-Verse/ReasonFlux-V2-Reasoner-DPO)

## Citation

```bash
@article{yang2025reasonflux,
  title={ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates},
  author={Yang, Ling and Yu, Zhaochen and Cui, Bin and Wang, Mengdi},
  journal={arXiv preprint arXiv:2502.06772},
  year={2025}
}
```