---
title: YiXin-Distill-Qwen-72B-AWQ
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ.md"
repository: AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ
last_updated: 2025-03-26
license: apache-2.0
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
parameters: 11.9B
tensor_type:
  - BF16
  - F16
  - I32
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 574
stars: 0
tags:
  - awq
  - 4-bits
---

# YiXin-Distill-Qwen-72B-AWQ

> YiXin-Distill-Qwen-72B-AWQ - AI-ModelScope 在 ModelScope 开源的模型。YiXin-Distill-Qwen-72B-AWQ

AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ 是 ModelScope 魔搭社区上的 11.9B 参数机器学习模型，采用 apache-2.0 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ
- **License**: apache-2.0
- **Parameters**: 11.9B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Tags**: awq, 4-bits
- **Downloads**: 574
- **Stars**: 0
- **Last updated**: 2025-03-26

Source: https://www.modelscope.cn/models/AI-ModelScope/YiXin-Distill-Qwen-72B-AWQ

---

# YiXin-Distill-Qwen-72B-AWQ

<div style="text-align: center;">
  <h1>YiXin-Distill-Qwen-72B-AWQ</h1>
  <img src="./fig/logo.png" alt="YiXin Logo">
</div>

## Model Overview

**YiXin-Distill-Qwen-72B: A High-Performance Distilled Model for Mathematical and General Reasoning**, derived from Qwen2.5-72B using reinforcement learning. It is specifically optimized for mathematical reasoning and general knowledge tasks. Leveraging advanced distillation techniques, this model enhances reasoning capabilities while maintaining computational efficiency. Built upon the robust Qwen model foundation, it aims to achieve state-of-the-art performance across various benchmark evaluations.Our benchmark evaluations demonstrate that YiXin-Distill-Qwen-72B delivers strong performance, showing improvements over comparable distilled models in key mathematical and general reasoning tasks, with observed average improvements of 5 to 11 percentage points.


## Run locally with vllm

For instance, you can easily start a service using [vLLM](https://github.com/vllm-project/vllm):

```shell
vllm serve /models/YiXin-Distill-Qwen-72B-AWQ --tensor-parallel-size 2  --quantization=awq
```

Then you can access the Chat API by:

```bash
curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "YiXin-AILab/YiXin-Distill-Qwen-72B-AWQ",
    "messages": [
        {"role": "system", "content": "You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step."},
        {"role": "user", "content": "8+8=?"}
    ]
    }'
```
## Citation

If you use YiXin-Distill-Qwen-72B-AWQ in your research, please cite this work appropriately:

```bibtex
@article{yixin2025,
  title={YiXin-Distill-Qwen-72B-AWQ: A High-Performance Distilled Model for Mathematical and General Reasoning},
  author={Your Name},
  year={2025},
  journal={Preprint}
}
```

## Acknowledgments

We acknowledge the contributions of the open-source community and researchers who have developed and maintained the Qwen and DeepSeek models. Their work has significantly advanced the field of large language model distillation and reasoning capabilities.