---
title: DistilQwen_1M
canonical_url: "https://www.modelscope.cn/datasets/PAI/DistilQwen_1M"
md_url: "https://www.modelscope.cn/datasets/PAI/DistilQwen_1M.md"
repository: PAI/DistilQwen_1M
chinese_name: DistilQwen_1M
last_updated: 2025-12-11
license: apache-2.0
storage_size: "2.6 GB"
downloads: 270
stars: 0
---

# DistilQwen_1M

> DistilQwen_1M - PAI 在 ModelScope 开源的数据集。DistilQwen-1M: High-Quality Instruction-Tuning Dataset

PAI/DistilQwen_1M 是 ModelScope 魔搭社区上的数据集，存储大小 2.6 GB，采用 apache-2.0 许可。

- **Repository**: PAI/DistilQwen_1M
- **License**: apache-2.0
- **Storage size**: 2.6 GB
- **Downloads**: 270
- **Stars**: 0
- **Last updated**: 2025-12-11

Source: https://www.modelscope.cn/datasets/PAI/DistilQwen_1M

---

# DistilQwen-1M: High-Quality Instruction-Tuning Dataset  

## Overview  
To empower community developers in enhancing the **instruction-following capabilities** of large language models (LLMs), we open-source **`DistilQwen-1M`**, a distilled subset of the training data used for the **DistilQwen model series**. Alongside its smaller counterpart (`DistilQwen-100K`), this dataset provides diverse, high-quality samples to improve model performance in key areas.  

## Dataset Features  
- **Scale**: **1 million** meticulously distilled entries.  
- **Coverage**: Balanced mix of:  
  - **Mathematics**  
  - **Code generation & understanding**  
  - **Knowledge-based QA**  
  - **Instruction following**  
  - **Creative generation**  
- **Purpose**: Optimized for **instruction tuning**, helping models retain generalization while adapting to downstream tasks.  

## Use Cases  
- **Fine-tuning LLMs**: Mitigate *catastrophic forgetting* by combining with custom datasets.  
- **Multi-task learning**: Improve coherence in mathematical reasoning, coding, and creative tasks.  
- **Research**: Study distillation techniques or instruction-tuning efficacy.

## Reference

For more detailed information about the dataset construction process, we encourage you to refer to our paper:

- **DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models**  
  Chengyu Wang, Junbing Yan, Yuanhao Yue, Jun Huang  
  [arXiv:2504.15027](https://arxiv.org/abs/2504.15027)

You can cite the paper using the following citation format:

```bibtex
@misc{wang2025distilqwen25industrialpracticestraining,
      title={DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models}, 
      author={Chengyu Wang and Junbing Yan and Yuanhao Yue and Jun Huang},
      year={2025},
      eprint={2504.15027},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.15027}
}
```