--- title: midashenglm-7b-1021-fp32 canonical_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp32" md_url: "https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp32.md" repository: midasheng/midashenglm-7b-1021-fp32 last_updated: 2025-10-31 license: apache-2.0 model_type: - midashenglm architectures: - MiDashengLMModel base_model: - mispeech/dasheng-0.6B - Qwen/Qwen2.5-Omni-7B base_model_relation: finetune parameters: 8.3B tensor_type: - F32 - I64 library_name: - pytorch - transformer - safetensors frameworks: - pytorch language: - en - zh - th - id - vi downloads: 1429 stars: 0 tags: - multimodal - audio-language-model - audio --- # midashenglm-7b-1021-fp32 > midashenglm-7b-1021-fp32 - midasheng 在 ModelScope 开源的模型。MiDashengLM is an efficient audio-language model that achieves holistic audio understanding through caption-based alignment. It achieves state-of-the-art performance on multiple audio understanding… midasheng/midashenglm-7b-1021-fp32 是 ModelScope 魔搭社区上的 8.3B 参数机器学习模型，采用 apache-2.0 许可，基于 mispeech/dasheng-0.6B、Qwen/Qwen2.5-Omni-7B 构建。 - **Repository**: midasheng/midashenglm-7b-1021-fp32 - **License**: apache-2.0 - **Parameters**: 8.3B - **Base model**: mispeech/dasheng-0.6B, Qwen/Qwen2.5-Omni-7B - **Tags**: multimodal, audio-language-model, audio - **Downloads**: 1429 - **Stars**: 0 - **Last updated**: 2025-10-31 Source: https://www.modelscope.cn/models/midasheng/midashenglm-7b-1021-fp32 --- # MiDashengLM-7B-1021 MiDashengLM is an efficient audio-language model that achieves holistic audio understanding through caption-based alignment. It achieves state-of-the-art performance on multiple audio understanding benchmarks while maintaining high inference efficiency—delivering 3.2× throughput speedup and supporting batch sizes up to 512. 📖 For more detailed introduction and technical report, please visit our [GitHub repository](https://github.com/xiaomi-research/dasheng-lm). Note that for most applications, we strongly recommend using the BF16 version ([mispeech/midashenglm-7b-1021-bf16](https://huggingface.co/mispeech/midashenglm-7b-1021-bf16)) for optimal performance and efficiency. ## Usage ### Load Model ```python from transformers import AutoModelForCausalLM, AutoProcessor, AutoTokenizer model_id = "mispeech/midashenglm-7b-1021-fp32" # Only for exact reproduction; otherwise strongly recommend "mispeech/midashenglm-7b-1021-bf16" model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained(model_id) processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) ``` ### Construct Prompt ```python user_prompt = "Caption the audio." # You may try any other prompt messages = [ { "role": "user", "content": [ {"type": "text", "text": user_prompt}, { "type": "audio", "path": "/path/to/example.wav", # or "url": "https://example.com/example.wav" # or "audio": np.random.randn(16000) }, ], }, ] ``` ### Generate Output ```python import torch with torch.no_grad(): model_inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, add_special_tokens=True, return_dict=True, ).to(device=model.device, dtype=model.dtype) generation = model.generate(**model_inputs) output = tokenizer.batch_decode(generation, skip_special_tokens=True) # ["An engine is idling."] ``` ## Results The following evaluation results are based on the model version: `mispeech/midashenglm-7b-1021-fp32`. ### Audio Captioning Results | Domain | Dataset | MiDashengLM | Qwen2.5-Omni-7B | Kimi-Audio-Instruct | |:--------:|:--------------:|:--------------:|:----------------:|:-------------------:| | Music | MusicCaps | **59.11** | 43.71 | 35.43 | | Music | Songdescriber | **46.42** | 45.31 | 44.63 | | Sound | AudioCaps | **62.13** | 60.79 | 49.00 | | Sound | ClothoV2 | **49.35** | 47.55 | 48.01 | | Sound | AutoACD | **67.13** | 55.93 | 44.76 | *Metrics: FENSE (higher is better).* ### Audio and Paralinguistic Classification | Dataset | Metric | MiDashengLM | Qwen2.5-Omni-7B | Kimi-Audio-Instruct | |:----------------:|:------:|:--------------:|:----------------:|:------------------:| | VoxCeleb1 | ACC↑ | **92.66** | 59.71 | 82.72 | | VoxLingua107 | ACC↑ | **93.72** | 51.03 | 73.65 | | VoxCeleb-Gender | ACC↑ | 97.72 | **99.82** | 99.69 | | VGGSound | ACC↑ | **52.19** | 0.97 | 2.20 | | Cochlscene | ACC↑ | **75.81** | 23.88 | 18.34 | | NSynth | ACC↑ | **80.32** | 60.45 | 38.09 | | FMA | ACC↑ | 62.94 | **66.77** | 27.91 | | FSDKaggle2018 | ACC↑ | **73.38** | 31.38 | 24.75 | | AudioSet | mAP↑ | **9.90** | 6.48 | 3.47 | | FSD50K | mAP↑ | **38.10** | 23.87 | 27.23 | ### ASR Performance | Dataset | Language | MiDashengLM | Qwen2.5-Omni-7B | Kimi-Audio-Instruct | |:------------------:|:------------:|:-------------:|:------------:|:-------------------:| | LibriSpeech test-clean | English | 3.6 | 1.7 | **1.3** | | LibriSpeech test-other | English | 5.9 | 3.4 | **2.4** | | People's Speech | English | 26.12 | 28.6 | **22.3** | | AISHELL2 Mic | Chinese | 3.2 | **2.5** | 2.7 | | AISHELL2 iOS | Chinese | 2.9 | **2.6** | **2.6** | | AISHELL2 Android | Chinese | 3.1 | 2.7 | **2.6** | | GigaSpeech2 | Indonesian | 22.3 | **21.2** | >100 | | GigaSpeech2 | Thai | **38.4** | 53.8 | >100 | | GigaSpeech2 | Viet | **17.7** | 18.6 | >100 | *Metrics: WER/CER (lower is better).* ### Question Answering Results | Dataset | Subset | Metric | MiDashengLM | Qwen2.5-Omni-7B | Kimi-Audio-Instruct | |:--------------:|:------------------:|:------:|:--------------:|:----------------:|:-------------------:| | MMAU-Pro | IF | ACC↑ | 37.93 | **61.30** | 42.30 | | MMAU-Pro | Multi-Audio | ACC↑ | **42.33** | 24.30 | 17.20 | | MMAU-Pro | Music | ACC↑ | **62.20** | 61.50 | 57.60 | | MMAU-Pro | Open-ended | ACC↑ | **63.21** | 52.30 | 34.50 | | MMAU-Pro | Sound | ACC↑ | **58.36** | 47.60 | 46.00 | | MMAU-Pro | Sound–Music | ACC↑ | 42.00 | 40.00 | **46.00** | | MMAU-Pro | Sound–Music–Speech | ACC↑ | **71.43** | 28.50 | 42.80 | | MMAU-Pro | Spatial | ACC↑ | 18.77 | 41.20 | **43.70** | | MMAU-Pro | Speech | ACC↑ | **61.17** | 57.40 | 52.20 | | MMAU-Pro | Speech–Music | ACC↑ | **58.70** | 53.20 | 54.30 | | MMAU-Pro | Speech–Sound | ACC↑ | 51.14 | **60.20** | 48.90 | | MMAU-Pro | Voice | ACC↑ | 54.83 | **60.00** | 50.60 | | MMAU-Pro | Average | ACC↑ | **55.92** | 52.20 | 46.60 | | MMAU-v05.15.25 | Sound | ACC↑ | 77.48 | **78.10** | 75.68 | | MMAU-v05.15.25 | Music | ACC↑ | **70.96** | 65.90 | 66.77 | | MMAU-v05.15.25 | Speech | ACC↑ | **76.28** | 70.60 | 62.16 | | MMAU-v05.15.25 | Average | ACC↑ | **74.90** | 71.50 | 68.20 | | MuChoMusic | | ACC↑ | **73.04** | 64.79 | 67.40 | | MusicQA | | FENSE↑ | **61.56** | 60.60 | 40.00 | | AudioCaps-QA | | FENSE↑ | **54.20** | 53.28 | 47.34 | *Metrics: Higher is better.* ## Citation MiDashengLM is under the Apache License 2.0, and we encourage its use in **both research and business applications**. If you find MiDashengLM useful in your research, please consider citing our work: ```bibtex @techreport{midashenglm7b, title = {MiDashengLM: Efficient Audio Understanding with General Audio Captions}, author = {{Horizon Team, MiLM Plus}}, institution= {Xiaomi Inc.}, year = {2025}, note = {Contributors: Heinrich Dinkel et al. (listed alphabetically in Appendix B)}, url = {https://arxiv.org/abs/2508.03983}, eprint = {2508.03983}, } ```