---
title: midashenglm-spatial
canonical_url: "https://www.modelscope.cn/models/midasheng/midashenglm-spatial"
md_url: "https://www.modelscope.cn/models/midasheng/midashenglm-spatial.md"
repository: midasheng/midashenglm-spatial
last_updated: 2026-10-15
license: apache-2.0
model_type:
  - midashenglm_spatial
architectures:
  - MiDashengLMSpatialModel
parameters: 8.4B
tensor_type:
  - I64
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
downloads: 54
stars: 0
tags:
  - spatial-audio-understanding
  - multimodal
  - audio-language-model
  - audio
  - dasheng
---

# midashenglm-spatial

> midashenglm-spatial - midasheng 在 ModelScope 开源的模型。MiDashengLM-Spatial Unifying General Audio Understanding and Spatial Awareness

midasheng/midashenglm-spatial 是 ModelScope 魔搭社区上的 8.4B 参数机器学习模型，采用 apache-2.0 许可。

- **Repository**: midasheng/midashenglm-spatial
- **License**: apache-2.0
- **Parameters**: 8.4B
- **Tags**: spatial-audio-understanding, multimodal, audio-language-model, audio, dasheng
- **Downloads**: 54
- **Stars**: 0
- **Last updated**: 2026-10-15

Source: https://www.modelscope.cn/models/midasheng/midashenglm-spatial

---

<div align="center">
    <h1>
    MiDashengLM-Spatial
    </h1>
    <b><em>Unifying General Audio Understanding and Spatial Awareness</em></b>
    <p>
    </p>
</div>

[![arXiv](https://img.shields.io/badge/arXiv-2610.11156-b31b1b)](https://arxiv.org/abs/2610.11156)&nbsp;&nbsp;[![HuggingFace Model](https://img.shields.io/badge/HuggingFace-Model-FFD21E)](https://huggingface.co/mispeech/midashenglm-spatial)&nbsp;&nbsp;[![ModelScope](https://img.shields.io/badge/ModelScope-7B-7448ce)](https://modelscope.cn/models/midasheng/midashenglm-spatial)&nbsp;&nbsp;[![Demo Page](https://img.shields.io/badge/Demo-Page-0366d6)](https://jinbo-hu.github.io/midashenglm-spatial-demo)&nbsp;&nbsp;[![GitHub](https://img.shields.io/badge/GitHub-Repo-181717)](https://github.com/xiaomi-research/midashenglm-spatial)

## Overview

**MiDashengLM-Spatial**, to our knowledge, is the **first open-source end-to-end** **unified** audio-language model that supports both **general audio understanding** and **spatial awareness** within a single architecture. It extends [MiDashengLM](https://github.com/xiaomi-research/dasheng-lm) with **Spatial-Dasheng** through a **hierarchical semantic-to-spatial conditioning (HSSC)** module to separately capture and effectively integrate semantic and spatial audio information. This design enables binaural perception and spatial audio understanding capabilities while preserving general audio understanding capabilities of the base model.

## Architecture

![Architecture](./assets/MiDashengLM-Spatial.png)

## Installation

```bash
pip install torch torchaudio "transformers==4.57.6" einops
```

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "mispeech/midashenglm-spatial"  # or a local package directory
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "path": "example/example.wav"}, 
            # {"type": "audio", "audio": np.random.randn(2, 160000)}, 
            {"type": "text", "text": "Write a general audio caption describing the sound source and its spatial information."},
        ],
    },
]

with torch.no_grad():
    inputs = processor.apply_chat_template(
        messages, tokenize=True, add_generation_prompt=True,
        add_special_tokens=True, return_dict=True,
    ).to(model.device)
    generation = model.generate(**inputs, max_new_tokens=256, do_sample=False)
    print(processor.tokenizer.batch_decode(generation, skip_special_tokens=True))
```

Put audio **before** text (the trained order), and feed real binaural `(2, T)` audio — mono still runs but carries no direction.

## Citation

```bibtex
@article{hu2026midashenglmspatial,
      title={MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness}, 
      author={Jinbo Hu and Hang Su and Lichun Fan and Heinrich Dinkel and Gang Li and Zhanchen Dai and Yiru Zhang and Chang Liu and Peng Wang and Junnan Wu and Jian Luan and Cong Zou and Heng Qu},
      year={2026},
      journal={arXiv preprint arxiv:2610.11156},
}
```

## License

Released under the Apache License 2.0 (see [LICENSE](LICENSE)), for both research and commercial use.