---
title: DistillQwen-ThoughtY-8B
canonical_url: "https://www.modelscope.cn/models/PAI/DistillQwen-ThoughtY-8B"
md_url: "https://www.modelscope.cn/models/PAI/DistillQwen-ThoughtY-8B.md"
repository: PAI/DistillQwen-ThoughtY-8B
chinese_name: DistilQwen-ThoughtY-8B
last_updated: 2026-08-03
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3
architectures:
  - Qwen3ForCausalLM
parameters: 8.2B
tensor_type:
  - BF16
library_name:
  - transformer
  - safetensors
  - pytorch
frameworks:
  - Pytorch
downloads: 345
stars: 0
---

# DistillQwen-ThoughtY-8B

> DistillQwen-ThoughtY-8B - PAI 在 ModelScope 开源的模型。DistillQwen-ThoughtY: Enhanced Chain-of-Thought Reasoning Models

PAI/DistillQwen-ThoughtY-8B 是 ModelScope 魔搭社区上的 8.2B 参数text-generation模型，采用 apache-2.0 许可。

- **Repository**: PAI/DistillQwen-ThoughtY-8B
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 8.2B
- **Downloads**: 345
- **Stars**: 0
- **Last updated**: 2026-08-03

Source: https://www.modelscope.cn/models/PAI/DistillQwen-ThoughtY-8B

---

# DistillQwen-ThoughtY: Enhanced Chain-of-Thought Reasoning Models

## Key Contributions
- **Advanced Reasoning Models**: DistillQwen-ThoughtY series (4B/8B/32B) outperform previous versions (ThoughtX) and Qwen3 in thinking mode, achieving state-of-the-art results on mathematical, scientific, and coding tasks.
- **OmniThought-0528 Dataset**: New 365K high-quality Chain-of-Thought (CoT) dataset distilled from DeepSeek-R1-0528 (top-tier Chinese model) with:
  - Cognitive Difficulty (CD) and Reasoning Verbosity (RV) annotations
  - Multi-teacher integration (DeepSeek-R1, DeepSeek-R1-0528, QwQ-32B)

## Performance Highlights
| Model               | AIME2024 | MATH500 | GPQA Diamond | LiveCodeBench V2 | Avg. |
|---------------------|----------|---------|--------------|------------------|------|
| **DistillQwen-ThoughtY-4B** | **76.7** | **95.2**    | **56.1**     | **75.8**   | **76.0** |
| **DistillQwen-ThoughtY-8B** | **76.7** | **94.6**    | **62.1**     | **78.1**   | **77.9** |
| **DistillQwen-ThoughtY-32B** | **90.0** | **95.2**    | 63.6     | **76.3**   | **81.3** |
| OpenThinker2-32B | 76.7     | 90.8    | **64.1**         | 72.5             | 76.0 |
| DistillQwen-ThoughtX-32B | 80.0 | 92.6    | 64.0         | 73.4             | 77.5 |

## Quick Start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "alibaba-pai/DistillQwen-ThoughtY-4B",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Solve ∫x e^x dx. Show your reasoning step-by-step."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
)
inputs = tokenizer([text], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=32768)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Resources
- [Models on Hugging Face](https://huggingface.co/alibaba-pai)
- [OmniThought-0528 Dataset](https://huggingface.co/datasets/alibaba-pai/OmniThought-0528)
- [EasyDistill Framework](https://github.com/modelscope/easydistill)


## Reference

For more detailed information about the model, we encourage you to refer to our paper:

- **Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations**  
  Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang
  [arXiv:2505.10937](https://arxiv.org/abs/2505.10937)

You can cite the paper using the following citation format:

```bibtex
@misc{cai2025reasoningomnithoughtlargecot,
      title={Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations}, 
      author={Wenrui Cai and Chengyu Wang and Junbing Yan and Jun Huang and Xiangzhong Fang},
      year={2025},
      eprint={2505.10937},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.10937} 
}
```