---
title: DistilQwen2.5-3B-Instruct
canonical_url: "https://www.modelscope.cn/models/PAI/DistilQwen2.5-3B-Instruct"
md_url: "https://www.modelscope.cn/models/PAI/DistilQwen2.5-3B-Instruct.md"
repository: PAI/DistilQwen2.5-3B-Instruct
last_updated: 2025-08-10
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
parameters: 3.1B
tensor_type:
  - F32
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 1113
stars: 2
---

# DistilQwen2.5-3B-Instruct

> DistilQwen2.5-3B-Instruct - PAI 在 ModelScope 开源的模型。DistilQwen2.5-3B is a distilled version of Qwen2.5-3B-Instruct, designed to distill the capabilities of stronger LLMs into smaller ones. To achieve this, we utilized a diverse range of datasets for the…

PAI/DistilQwen2.5-3B-Instruct 是 ModelScope 魔搭社区上的 3.1B 参数text-generation模型，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: PAI/DistilQwen2.5-3B-Instruct
- **Tasks**: text-generation
- **Parameters**: 3.1B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 1113
- **Stars**: 2
- **Last updated**: 2025-08-10

Source: https://www.modelscope.cn/models/PAI/DistilQwen2.5-3B-Instruct

---

## 📖 Introduction

**DistilQwen2.5-3B** is a distilled version of **Qwen2.5-3B-Instruct**, designed to distill the capabilities of stronger LLMs into smaller ones. To achieve this, we utilized a diverse range of datasets for the distillation process, including well-known open-source collections such as Magpie, Openhermes, and Mammoth 2, as well as proprietary synthetic datasets.

The training data primarily consists of instructions in Chinese and English. To enhance the quality and diversity of the instruction data, we implemented a difficulty scoring system and task-related resampling techniques.

For difficulty scoring, we employed the LLM-as-a-Judge paradigm, using the teacher model to evaluate responses based on accuracy, relevance, helpfulness, and level of detail. We then calculated the Model Fitting Difficulty (MFD) Score by subtracting the teacher model's score from the student model's score. A higher MFD Score indicates that the instruction is more valuable for distillation training. This approach allowed us to remove low-difficulty instructions from the training set, focusing on more challenging and informative examples.

After performing black-box data distillation on the model, we further conducted white-box distillation (teacher model logits distillation). Black-box knowledge distillation relies solely on the highest probability token output by the teacher model, while white-box knowledge distillation focuses more on the distribution of logits output by the teacher model, thereby providing richer information for the student model. By mimicking the logits distribution of the teacher model, white-box distillation can transfer knowledge more effectively, further enhancing the performance of the student model.

This careful curation and scoring process ensures that **DistilQwen2.5-3B** achieves high performance after the distillation process.

## 🚀 Quick Start

Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "alibaba-pai/DistilQwen2.5-3B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("alibaba-pai/DistilQwen2.5-3B-Instruct")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=2048，
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

## Reference

For more detailed information about the model, we encourage you to refer to our paper:

- **DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models**  
  Chengyu Wang, Junbing Yan, Yuanhao Yue, Jun Huang  
  [arXiv:2504.15027](https://arxiv.org/abs/2504.15027)

You can cite the paper using the following citation format:

```bibtex
@misc{wang2025distilqwen25industrialpracticestraining,
      title={DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models}, 
      author={Chengyu Wang and Junbing Yan and Yuanhao Yue and Jun Huang},
      year={2025},
      eprint={2504.15027},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.15027}
}
```