---
title: midashenglm-7b-1021-w4a16-gptq
canonical_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-w4a16-gptq"
md_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-w4a16-gptq.md"
repository: midasheng/midashenglm-7b-1021-w4a16-gptq
last_updated: 2025-10-31
license: apache-2.0
model_type:
  - midashenglm
architectures:
  - MiDashengLMModel
base_model:
  - mispeech/dasheng-0.6B
  - Qwen/Qwen2.5-Omni-7B
base_model_relation: finetune
parameters: 2.6B
tensor_type:
  - BF16
  - I64
  - I32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
language:
  - en
  - zh
  - th
  - id
  - vi
downloads: 177
stars: 0
tags:
  - multimodal
  - audio-language-model
  - audio
---

# midashenglm-7b-1021-w4a16-gptq

> midashenglm-7b-1021-w4a16-gptq - midasheng 在 ModelScope 开源的模型。MiDashengLM-7B-1021 (4bit, GPTQ quantized)

midasheng/midashenglm-7b-1021-w4a16-gptq 是 ModelScope 魔搭社区上的 2.6B 参数机器学习模型，采用 apache-2.0 许可，基于 mispeech/dasheng-0.6B、Qwen/Qwen2.5-Omni-7B 构建。

- **Repository**: midasheng/midashenglm-7b-1021-w4a16-gptq
- **License**: apache-2.0
- **Parameters**: 2.6B
- **Base model**: mispeech/dasheng-0.6B, Qwen/Qwen2.5-Omni-7B
- **Tags**: multimodal, audio-language-model, audio
- **Downloads**: 177
- **Stars**: 0
- **Last updated**: 2025-10-31

Source: https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-w4a16-gptq

---

# MiDashengLM-7B-1021 (4bit, GPTQ quantized)

The 4bit (w4a16) weights for [mispeech/midashenglm-7b-1021-fp32](https://huggingface.co/mispeech/midashenglm-7b-1021-fp32), quantized by GPTQ.

An ideal choice for resource-constrained environments. It offers broad GPU compatibility and a smaller memory footprint, making it suitable for deployment where VRAM, memory, or storage is limited, provided that a slight trade-off in quality is acceptable.

## Usage

### Load Model

```python
from transformers import AutoModelForCausalLM, AutoProcessor, AutoTokenizer

model_id = "mispeech/midashenglm-7b-1021-w4a16-gptq"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
```

### Construct Prompt

```python
user_prompt = "Caption the audio."  # You may try any other prompt

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": user_prompt},
            {
                "type": "audio",
                "path": "/path/to/example.wav",
                # or "url": "https://example.com/example.wav"
                # or "audio": np.random.randn(16000)
            },
        ],
    },
]
```

### Generate Output

```python
import torch

with torch.no_grad():
    model_inputs = processor.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        add_special_tokens=True,
        return_dict=True,
    ).to(device=model.device, dtype=model.dtype)
    generation = model.generate(**model_inputs)
    output = tokenizer.batch_decode(generation, skip_special_tokens=True)  # ["An engine is idling."]
```

## Citation

MiDashengLM is under the Apache License 2.0, and we encourage its use in **both research and business applications**.

If you find MiDashengLM useful in your research, please consider citing our work:

```bibtex
@techreport{midashenglm7b,
  title      = {MiDashengLM: Efficient Audio Understanding with General Audio Captions},
  author     = {{Horizon Team, MiLM Plus}},
  institution= {Xiaomi Inc.},
  year       = {2025},
  note       = {Contributors: Heinrich Dinkel et al. (listed alphabetically in Appendix B)},
  url        = {https://arxiv.org/abs/2508.03983},
  eprint     = {2508.03983},
}
```