---
title: MOSS-Audio-4B-Instruct-GGUF
canonical_url: "https://www.modelscope.cn/models/cloudlnk/MOSS-Audio-4B-Instruct-GGUF"
md_url: "https://www.modelscope.cn/models/cloudlnk/MOSS-Audio-4B-Instruct-GGUF.md"
repository: cloudlnk/MOSS-Audio-4B-Instruct-GGUF
last_updated: 2026-06-09
license: apache-2.0
base_model:
  - OpenMOSS-Team/MOSS-Audio-4B-Instruct
base_model_relation: quantized
library_name:
  - gguf
language:
  - zh
  - en
downloads: 18
stars: 0
tags:
  - audio
  - speech-recognition
  - audio-understanding
  - audio-qa
  - transcription
  - ggml
  - gguf
  - moss-audio
  - qwen3
  - deepstack
---

# MOSS-Audio-4B-Instruct-GGUF

> MOSS-Audio-4B-Instruct-GGUF - cloudlnk 在 ModelScope 开源的模型。MOSS-Audio-4B-Instruct -- GGUF (ggml-quantised)

cloudlnk/MOSS-Audio-4B-Instruct-GGUF 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可，基于 OpenMOSS-Team/MOSS-Audio-4B-Instruct 构建。

- **Repository**: cloudlnk/MOSS-Audio-4B-Instruct-GGUF
- **License**: apache-2.0
- **Base model**: OpenMOSS-Team/MOSS-Audio-4B-Instruct
- **Tags**: audio, speech-recognition, audio-understanding, audio-qa, transcription, ggml, gguf, moss-audio, qwen3, deepstack
- **Downloads**: 18
- **Stars**: 0
- **Last updated**: 2026-06-09

Source: https://www.modelscope.cn/models/cloudlnk/MOSS-Audio-4B-Instruct-GGUF

---

# MOSS-Audio-4B-Instruct -- GGUF (ggml-quantised)

GGUF / ggml conversions of [`OpenMOSS-Team/MOSS-Audio-4B-Instruct`](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-4B-Instruct) for use with `crispasr --backend moss-audio` from **[CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR)**.

MOSS-Audio-4B-Instruct is OpenMOSS's ~4.6 B parameter **audio-understanding** model:

- **First audio-understanding backend in CrispASR** -- not just ASR but also audio QA, scene description, music analysis, meeting summarisation
- **Mandarin + English** speech recognition and audio understanding
- **DeepStack cross-layer feature injection** -- multi-resolution encoder taps at layers 8/16/24 injected into the LM's early layers for fine-grained prosody + semantic awareness
- **Time-aware ASR** with explicit time-marker tokens for word-level and sentence-level timestamps
- **Apache-2.0** licence

## Files

| File | Size | Notes |
| --- | ---: | --- |
| `moss-audio-4b-instruct-f16.gguf` | 9.73 GB | F16, full precision |
| `moss-audio-4b-instruct-q4_k.gguf` | 2.75 GB | **Q4_K -- recommended default** |

## Quick Start

```bash
# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -G Ninja -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=OFF
cmake --build build -j$(nproc) --target crispasr-cli

# 2. Download a quantisation
huggingface-cli download cstr/MOSS-Audio-4B-Instruct-GGUF \
    moss-audio-4b-instruct-q4_k.gguf --local-dir .

# 3. Transcribe audio
./build/bin/crispasr \
    -m moss-audio-4b-instruct-q4_k.gguf \
    -f your-audio.wav \
    --backend moss-audio -t 4

# 4. Audio understanding (custom prompt)
./build/bin/crispasr \
    -m moss-audio-4b-instruct-q4_k.gguf \
    -f your-audio.wav \
    --backend moss-audio \
    --prompt "Describe the sounds in this audio clip."
```


## Verified end-to-end output

**JFK sample (samples/jfk.wav, 11s):**
> And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

Verified on Q4_K (3.8 GB, F16 encoder + Q4_K LLM). All 6 crispasr-diff stages PASS at cos >= 0.999.

## Architecture

| Component | Details |
| --- | --- |
| Audio encoder | 32-layer Whisper-style transformer (d=1280, 20 heads, head_dim=64, FFN=5120, GELU, LayerNorm, eps=1e-5) |
| Conv stem | 3x Conv2d(stride=2, channels=480, kernel=3x3, pad=1) -> 8x temporal downsample (128 mel bins -> 16 freq bins) |
| Stem projection | Linear(480x16=7680 -> 1280) + sinusoidal positional embedding (max 1500 positions) |
| DeepStack taps | Encoder layers [8, 16, 24] -> 3 independent GatedMLP(1280 -> 8192 -> 2560, SiLU) |
| DeepStack injection | Residual add at LM layers [0, 1, 2] at audio-token positions |
| Audio adapter | GatedMLP(1280 -> 8192 -> 2560, SiLU) for final encoder output |
| LM backbone | 36-layer Qwen3 (hidden=2560, 32 Q-heads / 8 KV-heads, head_dim=128, QK-norm, SwiGLU FFN=9728, RoPE theta=1M) |
| Output head | Linear(2560 -> 151936), untied from embedding |
| Vocab | 151936 Qwen3 BPE (151643 regular + 293 special tokens) |
| Audio input | 16 kHz mono, 128 mel bins, n_fft=400, hop=160 |
| Audio tokens | 12.5 Hz after 8x conv downsample, time markers every 2 seconds |
| Parameters | ~4.6 B total (encoder ~650M + adapter/deepstack ~120M + LM ~3.8B) |

### Special tokens

| Token | ID | Purpose |
| --- | --- | --- |
| `<\|AUDIO\|>` | 151654 | Audio frame placeholder (replaced by encoder embeddings) |
| `<\|audio_bos\|>` | 151669 | Audio segment start marker |
| `<\|audio_eos\|>` | 151670 | Audio segment end marker |
| `<\|im_start\|>` | 151644 | Chat turn start |
| `<\|im_end\|>` | 151645 | Chat turn end / EOS |

## How this was made

1. **Inspect** the HF safetensors: 3 shards, 901 tensors total -- audio encoder (conv stem + 32 transformer layers + layer_norm), audio adapter (1 GatedMLP), deepstack mergers (3 GatedMLPs), language model (embedding + 36 Qwen3 layers + final norm + lm_head).

2. **Convert** with [`models/convert-moss-audio-to-gguf.py`](https://github.com/CrispStrobe/CrispASR/blob/feature/moss-audio/models/convert-moss-audio-to-gguf.py): stream BF16 tensors one-at-a-time via `safe_open`, remap HF tensor names (`audio_encoder.layers.N.self_attn.q_proj` -> `enc.blk.N.attn.q`, `deepstack_audio_merger_list.N.gate_proj` -> `deepstack.N.gate`, `language_model.layers.N.mlp.gate_proj` -> `llm.blk.N.ffn.gate`, etc.), write F16 + F32 (norms/biases). BPE vocab + merges from `vocab.json` + `merges.txt` + `added_tokens.json`.

3. **Quantize** with `crispasr-quantize`: F16 -> Q4_K (2D+ tensors quantised, 1D biases/norms kept F32).

4. **C++ runtime** in [`src/moss_audio.{h,cpp}`](https://github.com/CrispStrobe/CrispASR/blob/feature/moss-audio/src/moss_audio.cpp): GGUF mmap, encoder graph (conv stem + 32 WhisperEncoderLayers with bidirectional attention + DeepStack tap capture), adapter/merger GatedMLP graphs, per-layer DeepStack injection into LM via pre-scattered residuals, KV-cached Qwen3 decode with `core_attn::kv_self_attn` (QK-norm, RoPE, GQA), greedy decode with chat-template prompt builder.

## Upstream

- Model: [`OpenMOSS-Team/MOSS-Audio-4B-Instruct`](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-4B-Instruct) (Apache-2.0)
- Code: [`OpenMOSS/MOSS-Audio`](https://github.com/OpenMOSS/MOSS-Audio)
- Runtime: [`CrispStrobe/CrispASR`](https://github.com/CrispStrobe/CrispASR) branch `feature/moss-audio`