---
title: Qwen2-14B-merge-GPTQ-Int4
canonical_url: "https://www.modelscope.cn/models/tclf90/Qwen2-14B-merge-GPTQ-Int4"
md_url: "https://www.modelscope.cn/models/tclf90/Qwen2-14B-merge-GPTQ-Int4.md"
repository: tclf90/Qwen2-14B-merge-GPTQ-Int4
chinese_name: "通义千问2-14B-Merge-GPTQ-Int4-量化修复"
last_updated: 2024-06-12
license: qwen
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
parameters: 2.7B
tensor_type:
  - F16
  - I32
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 243
stars: 4
tags:
  - qwen2
  - gptq
  - int4
  - "量化修复"
  - vLLM
---

# Qwen2-14B-merge-GPTQ-Int4

> Qwen2-14B-merge-GPTQ-Int4 - tclf90 在 ModelScope 开源的模型。通义千问2-14B-Merge-GPTQ-Int4-量化修复 HF原模型 paperplanedeemo/Qwen2-14B-merge

tclf90/Qwen2-14B-merge-GPTQ-Int4 是 ModelScope 魔搭社区上的 2.7B 参数text-generation模型，采用 qwen 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: tclf90/Qwen2-14B-merge-GPTQ-Int4
- **License**: qwen
- **Tasks**: text-generation
- **Parameters**: 2.7B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Tags**: qwen2, gptq, int4, 量化修复, vLLM
- **Downloads**: 243
- **Stars**: 4
- **Last updated**: 2024-06-12

Source: https://www.modelscope.cn/models/tclf90/Qwen2-14B-merge-GPTQ-Int4

---

# 通义千问2-14B-Merge-GPTQ-Int4-量化修复
HF原模型 [paperplanedeemo/Qwen2-14B-merge](https://huggingface.co/paperplanedeemo/Qwen2-14B-merge)


### 【模型更新日期】
``` 2024-06-13 ```

### 【模型大小】
`7.9GB`

### 【修复内容】

1. 对GPTQ量化的校准做了额外优化；减少量化模型的 `1.乱吐字`、`2.无限循环`、`3.长文能力丢失`等情况。

### 【更新日志】

```
2024-06-13
首次commit
```

### 【介绍】
<i>注意：非官方模型</i>

Qwen2-14B-merge is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
```yaml
dtype: float16
merge_method: passthrough
slices:
  - sources:
      - layer_range: [0, 6]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [3, 9]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [6, 12]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [9, 15]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [12, 18]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [15, 21]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [18, 24]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [21, 28]
        model: Qwen/Qwen2-7B-Instruct
```

[更多详情...](https://huggingface.co/paperplanedeemo/Qwen2-14B-merge)

### 【同期量化修复模型】
待工作完成后补充...

### 【模型下载】
```python
from modelscope import snapshot_download
model_dir = snapshot_download('tclf90/模型名', cache_dir="本地路径")
```

### 【[vLLM](https://github.com/vllm-project/vllm)推理（目前仅限Linux）】
#### 1. Python 简易调试

待工作完成后补充...

#### 2. 类ChatGPT RESTFul API Server
```
>>> python -m vllm.entrypoints.openai.api_server --model 本地路径/tclf90/模型名称
```