---
title: Qwen2-7B-DI
canonical_url: "https://www.modelscope.cn/models/dash-infer/Qwen2-7B-DI"
md_url: "https://www.modelscope.cn/models/dash-infer/Qwen2-7B-DI.md"
repository: dash-infer/Qwen2-7B-DI
last_updated: 2024-06-06
license: "Apache License 2.0"
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
library_name:
  - other
frameworks:
  - other
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 189
stars: 1
---

# Qwen2-7B-DI

> Qwen2-7B-DI - dash-infer 在 ModelScope 开源的模型。通义千问推出的最新一代Qwen2-7B模型的DashInfer格式版本。方便通过DashInfer推理引擎在CPU/ARM等机器上使用。

dash-infer/Qwen2-7B-DI 是 ModelScope 魔搭社区上的text-generation模型，采用 Apache License 2.0 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: dash-infer/Qwen2-7B-DI
- **License**: Apache License 2.0
- **Tasks**: text-generation
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 189
- **Stars**: 1
- **Last updated**: 2024-06-06

Source: https://www.modelscope.cn/models/dash-infer/Qwen2-7B-DI

---

# 

本模型是[qwen/Qwen2-7B](https://modelscope.cn/models/qwen/Qwen2-7B/summary)的[DashInfer](https://github.com/modelscope/dash-infer)格式模型。

## requirements

```bash
pip install modelscope dashinfer jinja2 tabulate torch transformers
```

## 下载模型

```python
from modelscope import snapshot_download
model_dir = snapshot_download("dash-infer/Qwen2-7B-DI")
```

## 模型推理

参考如下python代码。

```python
import copy
import random

from modelscope import snapshot_download
from dashinfer.helper import EngineHelper, ConfigManager

model_path = snapshot_download("dash-infer/Qwen2-7B-DI")

config_file = model_path + "/" + "di_config.json"
config = ConfigManager.get_config_from_json(config_file)
config["model_path"] = model_path

## init EngineHelper class
engine_helper = EngineHelper(config)
engine_helper.verbose = True
engine_helper.init_tokenizer(model_path)

## init engine
engine_helper.init_engine()

## prepare inputs and generation configs
user_input = "浙江的省会在哪"
prompt = "<|im_start|>" + "system\n" + "You are a helpful assistant." + "<|im_end|>\n" + \
         "<|im_start|>" + "user\n" + user_input + "<|im_end|>\n" + \
         "<|im_start|>" + "assistant\n"
gen_cfg = copy.deepcopy(engine_helper.default_gen_cfg)
gen_cfg["seed"] = random.randint(0, 10000)
request_list = engine_helper.create_request([prompt], [gen_cfg])

## inference
engine_helper.process_one_request(request_list[0])
engine_helper.print_inference_result_all(request_list)

engine_helper.uninit_engine()