---
title: midashenglm-7b-1021-fp8
canonical_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp8"
md_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp8.md"
repository: midasheng/midashenglm-7b-1021-fp8
chinese_name: MiDashengLM-7B-1021-fp8
last_updated: 2025-10-31
license: apache-2.0
model_type:
  - midashenglm
architectures:
  - MiDashengLMModel
base_model:
  - mispeech/dasheng-0.6B
  - Qwen/Qwen2.5-Omni-7B
base_model_relation: finetune
parameters: 8.3B
tensor_type:
  - BF16
  - F8_E4M3
  - I64
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - Pytorch
language:
  - en
  - zh
  - th
  - id
  - vi
downloads: 203
stars: 0
tags:
  - multimodal
  - audio-language-model
  - audio
---

# midashenglm-7b-1021-fp8

> midashenglm-7b-1021-fp8 - midasheng 在 ModelScope 开源的模型。MiDashengLM-7B-1021 (FP8)

midasheng/midashenglm-7b-1021-fp8 是 ModelScope 魔搭社区上的 8.3B 参数机器学习模型，采用 apache-2.0 许可，基于 mispeech/dasheng-0.6B、Qwen/Qwen2.5-Omni-7B 构建。

- **Repository**: midasheng/midashenglm-7b-1021-fp8
- **License**: apache-2.0
- **Parameters**: 8.3B
- **Base model**: mispeech/dasheng-0.6B, Qwen/Qwen2.5-Omni-7B
- **Tags**: multimodal, audio-language-model, audio
- **Downloads**: 203
- **Stars**: 0
- **Last updated**: 2025-10-31

Source: https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp8

---

# MiDashengLM-7B-1021 (FP8)

The FP8 weights for [mispeech/midashenglm-7b-1021-fp32](https://huggingface.co/mispeech/midashenglm-7b-1021-fp32).

Optimized for Hopper-class (H100 and newer) GPUs, leveraging hardware support for enhanced performance and memory savings. While older GPUs may see limited performance gains, FP8 can still be used to conserve VRAM, and storage.

## Usage

### Load Model

```python
from transformers import AutoModelForCausalLM, AutoProcessor, AutoTokenizer

model_id = "mispeech/midashenglm-7b-1021-fp8"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
```

### Construct Prompt

```python
user_prompt = "Caption the audio."  # You may try any other prompt

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": user_prompt},
            {
                "type": "audio",
                "path": "/path/to/example.wav",
                # or "url": "https://example.com/example.wav"
                # or "audio": np.random.randn(16000)
            },
        ],
    },
]
```

### Generate Output

```python
import torch

with torch.no_grad():
    model_inputs = processor.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        add_special_tokens=True,
        return_dict=True,
    ).to(device=model.device, dtype=model.dtype)
    generation = model.generate(**model_inputs)
    output = tokenizer.batch_decode(generation, skip_special_tokens=True)  # ["An engine is idling."]
```

## Citation

MiDashengLM is under the Apache License 2.0, and we encourage its use in **both research and business applications**.

If you find MiDashengLM useful in your research, please consider citing our work:

```bibtex
@techreport{midashenglm7b,
  title      = {MiDashengLM: Efficient Audio Understanding with General Audio Captions},
  author     = {{Horizon Team, MiLM Plus}},
  institution= {Xiaomi Inc.},
  year       = {2025},
  note       = {Contributors: Heinrich Dinkel et al. (listed alphabetically in Appendix B)},
  url        = {https://arxiv.org/abs/2508.03983},
  eprint     = {2508.03983},
}
```