---
title: DistilQwen_100k
canonical_url: "https://www.modelscope.cn/datasets/PAI/DistilQwen_100k"
md_url: "https://www.modelscope.cn/datasets/PAI/DistilQwen_100k.md"
repository: PAI/DistilQwen_100k
chinese_name: DistilQwen_100k
last_updated: 2025-12-11
license: apache-2.0
storage_size: "118 MB"
downloads: 316
stars: 3
---

# DistilQwen_100k

> DistilQwen_100k - PAI 在 ModelScope 开源的数据集。To support community developers in avoiding the phenomenon of "catastrophic forgetting" when fine-tuning the DistilQwen2.5 model, we have open-sourced a portion of the dataset used for model training.

PAI/DistilQwen_100k 是 ModelScope 魔搭社区上的数据集，存储大小 118 MB，采用 apache-2.0 许可。

- **Repository**: PAI/DistilQwen_100k
- **License**: apache-2.0
- **Storage size**: 118 MB
- **Downloads**: 316
- **Stars**: 3
- **Last updated**: 2025-12-11

Source: https://www.modelscope.cn/datasets/PAI/DistilQwen_100k

---

To support community developers in avoiding the phenomenon of "catastrophic forgetting" when fine-tuning the DistilQwen2.5 model, we have open-sourced a portion of the dataset used for model training.  

These datasets are designed to provide a solid foundation for model fine-tuning, helping to enhance the model's adaptability to new tasks while maintaining its performance on previous ones.  

The released data covers various domains, including mathematics, coding, knowledge-based Q&A, instruction following, and creative generation, with a total volume of 10k samples.  

When fine-tuning the model with their own data, users can incorporate DistilQwen_100k to ensure strong performance on downstream tasks without compromising the model's general capabilities, thereby preserving its generalization ability.


## Reference

For more detailed information about the dataset construction process, we encourage you to refer to our paper:

- **DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models**  
  Chengyu Wang, Junbing Yan, Yuanhao Yue, Jun Huang  
  [arXiv:2504.15027](https://arxiv.org/abs/2504.15027)

You can cite the paper using the following citation format:

```bibtex
@misc{wang2025distilqwen25industrialpracticestraining,
      title={DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models}, 
      author={Chengyu Wang and Junbing Yan and Yuanhao Yue and Jun Huang},
      year={2025},
      eprint={2504.15027},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.15027}
}
```