---
title: MOSS-Music-8B-Thinking-8bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/MOSS-Music-8B-Thinking-8bit"
md_url: "https://www.modelscope.cn/models/mlx-community/MOSS-Music-8B-Thinking-8bit.md"
repository: mlx-community/MOSS-Music-8B-Thinking-8bit
last_updated: 2026-06-28
license: apache-2.0
model_type:
  - moss_music
architectures:
  - MossMusicModel
base_model:
  - OpenMOSS-Team/MOSS-Music-8B-Thinking
base_model_relation: finetune
parameters: 3.0B
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 74
stars: 0
tags:
  - mlx
  - audio
  - music
  - music-understanding
  - audio-language-model
  - moss
  - moss-music
---

# MOSS-Music-8B-Thinking-8bit

> MOSS-Music-8B-Thinking-8bit - mlx-community 在 ModelScope 开源的模型。MOSS-Music-8B-Thinking · MLX 8-bit

mlx-community/MOSS-Music-8B-Thinking-8bit 是 ModelScope 魔搭社区上的 3.0B 参数机器学习模型，采用 apache-2.0 许可，基于 OpenMOSS-Team/MOSS-Music-8B-Thinking 构建。

- **Repository**: mlx-community/MOSS-Music-8B-Thinking-8bit
- **License**: apache-2.0
- **Parameters**: 3.0B
- **Base model**: OpenMOSS-Team/MOSS-Music-8B-Thinking
- **Tags**: mlx, audio, music, music-understanding, audio-language-model, moss, moss-music
- **Downloads**: 74
- **Stars**: 0
- **Last updated**: 2026-06-28

Source: https://www.modelscope.cn/models/mlx-community/MOSS-Music-8B-Thinking-8bit

---

# MOSS-Music-8B-Thinking · MLX 8-bit

[![Base model](https://img.shields.io/badge/Base-MOSS--Music--8B--Thinking-ff9e2c?logo=huggingface&logoColor=white)](https://huggingface.co/OpenMOSS-Team/MOSS-Music-8B-Thinking)
[![MLX](https://img.shields.io/badge/MLX-Apple%20Silicon-black)](https://github.com/ml-explore/mlx)
[![License](https://img.shields.io/badge/License-Apache--2.0-green)](https://www.apache.org/licenses/LICENSE-2.0)
[![Backend](https://img.shields.io/badge/Backend-moss__music__mlx-blue?logo=github)](https://github.com/OpenMOSS/MOSS-Music/pull/3)

An **8-bit [MLX](https://github.com/ml-explore/mlx) quantization** of
[OpenMOSS-Team/MOSS-Music-8B-Thinking](https://huggingface.co/OpenMOSS-Team/MOSS-Music-8B-Thinking)
for music understanding (captioning, key / tempo / chord, structure, lyrics ASR, long-form QA)
that runs locally on Apple Silicon Macs.

> Community conversion, not an official release. All model credit goes to the OpenMOSS Team.

Other sizes: [6-bit](https://huggingface.co/mlx-community/MOSS-Music-8B-Thinking-6bit) · [4-bit](https://huggingface.co/mlx-community/MOSS-Music-8B-Thinking-4bit)

## Why this exists

On the stock PyTorch + MPS path several audio-encoder ops fall back to CPU, and local
generation is effectively unusable (under 0.3 tok/s, often hanging). This MLX build runs
properly on a Mac:

| | PyTorch / MPS (bf16) | **This model (MLX 8-bit)** |
|---|---|---|
| Size on disk | 18 GB | **~10 GB** |
| Load time | ~17 s | **~1.5 s** |
| One 75 s song | stalls (>13 min) | **~34 s** |
| Throughput | <0.3 tok/s | **~23 tok/s** |

*(Indicative single-run numbers on an M4, 24 GB.)*

## Usage

MOSS-Music is a custom multimodal (audio + text) model, so it does **not** load with
`mlx_lm` / `mlx_vlm` directly. Use the `moss_music_mlx` backend:

- Backend code: https://github.com/dthinkr/MOSS-Music/tree/feat/mlx-backend/mlx
- Upstream PR: https://github.com/OpenMOSS/MOSS-Music/pull/3

```python
from huggingface_hub import snapshot_download
from moss_music_mlx import load_pretrained, generate
from src.processing_moss_music import MossMusicProcessor

path = snapshot_download("mlx-community/MOSS-Music-8B-Thinking-8bit")
model = load_pretrained(path)
proc = MossMusicProcessor.from_pretrained(path, trust_remote_code=True, enable_time_marker=True)

print(generate(model, proc,
               "Analyze this track: genre, key, BPM, structure.",
               audio_path="song.mp3"))
```

Or from the command line:

```bash
python -m moss_music_mlx.generate --model <downloaded_path> --audio song.mp3 \
  --prompt "Describe this music."
```

See the backend `mlx/README.md` for full setup and the parity tests.

## How it was converted

- **8-bit, group size 64.** The audio encoder is kept at **bf16** to preserve audio
  fidelity; quantization is applied to the Qwen3 layers, token embeddings and `lm_head`.
- Converted with `mlx==0.31.2`, `mlx-lm==0.29.1`.

## Accuracy

| Comparison | Result |
|---|---|
| 8-bit vs **fp32 reference** — prefill next token | argmax identical, logit cosine **0.99999** |
| 8-bit vs **bf16** — prefill, 5 mixed-genre clips | argmax **5 / 5**, mean cosine **0.99998** |

Greedy decoding; long sampled generations may still diverge after a near-tie token, as
expected for 8-bit quantization.

## License & credit

Apache-2.0, inherited from the base model. This repository provides only the
MLX-quantized weights. Please cite the original authors:

```bibtex
@misc{mossmusic2026,
  title  = {MOSS-Music Technical Report},
  author = {OpenMOSS Team},
  year   = {2026},
  howpublished = {\url{https://github.com/OpenMOSS/MOSS-Music}}
}
```