--- title: MOSS-Transcribe-Diarize-gguf canonical_url: "https://www.modelscope.cn/models/ukk0708/MOSS-Transcribe-Diarize-gguf" md_url: "https://www.modelscope.cn/models/ukk0708/MOSS-Transcribe-Diarize-gguf.md" repository: ukk0708/MOSS-Transcribe-Diarize-gguf last_updated: 2026-10-23 license: apache-2.0 base_model: - OpenMOSS-Team/MOSS-Transcribe-Diarize base_model_relation: quantized library_name: - pytorch - transformer - gguf language: - en - zh downloads: 2 stars: 0 tags: - gguf - transcribe.cpp - asr - speech-to-text - moss - audio-llm - whisper-encoder - qwen3 - diarization --- # MOSS-Transcribe-Diarize-gguf > MOSS-Transcribe-Diarize-gguf - ukk0708 在 ModelScope 开源的模型。MOSS-Transcribe-Diarize: transcribe.cpp GGUF ukk0708/MOSS-Transcribe-Diarize-gguf 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可，基于 OpenMOSS-Team/MOSS-Transcribe-Diarize 构建。 - **Repository**: ukk0708/MOSS-Transcribe-Diarize-gguf - **License**: apache-2.0 - **Base model**: OpenMOSS-Team/MOSS-Transcribe-Diarize - **Tags**: gguf, transcribe.cpp, asr, speech-to-text, moss, audio-llm, whisper-encoder, qwen3, diarization - **Downloads**: 2 - **Stars**: 0 - **Last updated**: 2026-10-23 Source: https://www.modelscope.cn/models/ukk0708/MOSS-Transcribe-Diarize-gguf --- # MOSS-Transcribe-Diarize: transcribe.cpp GGUF GGUF conversions of [OpenMOSS-Team/MOSS-Transcribe-Diarize](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) for use with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp). Ported from upstream commit [d7231bb](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize/commit/d7231bb), pinned 2026-07-12. Validated against the MOSS author repo (OpenMOSS/MOSS-Transcribe-Diarize) reference at transcribe.cpp commit [3f5e15c](https://github.com/handy-computer/transcribe.cpp/tree/3f5e15c) on 2026-07-12. Offline English/Chinese speech-to-text with speaker diarization. A 0.9B audio-LLM: a Whisper-Medium encoder (24 layers, d_model=1024) feeds a 4x temporal merge + VQAdaptor bridge into a Qwen3-0.6B decoder (28 layers) via audio-token injection. The model emits `[start][Sxx]text[end]`; the runtime parses those generated markers into clean text and segment rows. Speaker attribution is opt-in (`--diarize`) and returns structured speaker ids/turns. Not a streaming model. ## Downloads | Quantization | Download | Size | WER (LibriSpeech test-clean) | | --- | --- | ---: | ---: | | BF16 | [MOSS-Transcribe-Diarize-BF16.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-BF16.gguf) | 1.83 GB | 2.08% | | F16 | [MOSS-Transcribe-Diarize-F16.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-F16.gguf) | 1.83 GB | 2.07% | | Q8_0 | [MOSS-Transcribe-Diarize-Q8_0.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-Q8_0.gguf) | 987 MB | 1.93% | | Q6_K | [MOSS-Transcribe-Diarize-Q6_K.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-Q6_K.gguf) | 768 MB | 1.96% | | Q5_K_M | [MOSS-Transcribe-Diarize-Q5_K_M.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-Q5_K_M.gguf) | 700 MB | 1.99% | | Q4_K_M | [MOSS-Transcribe-Diarize-Q4_K_M.gguf](https://huggingface.co/handy-computer/MOSS-Transcribe-Diarize-gguf/resolve/main/MOSS-Transcribe-Diarize-Q4_K_M.gguf) | 617 MB | 2.59% | WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 1, timestamps none. Figures without a commit were published before provenance was recorded. Scored with the Whisper-style English text normalizer and jiwer 3.x. MOSS emits the diarized format `[start][Sxx]text[end]`; the bracket spans are metadata and are de-diarized to a space (for both hypothesis and reference) before scoring, matching the author-repo reference runner. These values describe this dataset only, not a general quality ranking: a quant that scores slightly better here is not necessarily better in real-world use, because dataset-specific decoding near-ties can make quantization noise help or hurt individual utterances. The same-manifest MOSS author-repo reference (bf16, greedy) lands at **2.07%** with 95% bootstrap CI [1.82%, 2.40%]. The BF16 port lands at 2.08% (within +0.01 of the reference, well inside the CI band); the lower-bit presets sit between 1.93% and 1.99% (statistical noise) except Q4_K_M at 2.59%, whose excess is a handful of 4-bit tail failures (6 empty outputs, 5 English->Chinese language-drift utterances, 1 timestamp-token repetition loop) rather than broad degradation. Prefer Q5_K_M or higher if those tail failures matter. Reproduce with `scripts/wer/run.py` + `scripts/wer/score.py --dediarize`; public `full_text` applies equivalent marker removal. ## Usage Build transcribe.cpp from source: ```bash git clone git@github.com:handy-computer/transcribe.cpp.git cd transcribe.cpp cmake -B build && cmake --build build ``` Run on a 16 kHz mono WAV: ```bash build/bin/transcribe-cli \ -m MOSS-Transcribe-Diarize-Q8_0.gguf \ input.wav ``` If your audio isn't already 16 kHz mono WAV, convert it first: ```bash ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav ``` See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/moss-transcribe-diarize.md) for performance numbers, numerical validation, and reproduction steps. ## License Inherited from the base model: **Apache-2.0**. See the [upstream model card](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) for full terms. --- ## Original Model Card > The section below is reproduced from > [OpenMOSS-Team/MOSS-Transcribe-Diarize](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) at commit > `d7231bb` for offline reference. The upstream card is the > authoritative source. # MOSS-Transcribe-Diarize
MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness. Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond. ## News * 2026-07-09: Released MOSS-Transcribe-Diarize 0.9B. ## Contents - [Introduction](#introduction) - [Model Architecture](#model-architecture) - [Evaluation](#evaluation) - [Quickstart](#quickstart) - [Environment Setup](#environment-setup) - [Python Usage](#python-usage) - [Custom Prompt and Hotwords](#custom-prompt-and-hotwords) - [Serve with vLLM and SGLang](#serve-with-vllm-and-sglang) - [Subtitle Web App](#subtitle-web-app) - [Output Format](#output-format) - [More Information](#more-information) - [License](#license) - [Citation](#citation) ## Introduction MOSS-Transcribe-Diarize 0.9B turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, it jointly performs speech transcription and speaker diarization, producing time-aligned text with consistent speaker labels. The model is built for meetings, calls, podcasts, interviews, lectures, videos, and other long or messy multi-speaker recordings. It can also emit acoustic event annotations, giving downstream systems a richer view of what happened, who spoke, and when. Core capabilities: * **Long-form transcription**: Converts long audio or video recordings into timestamped text. * **Speaker-aware diarization**: Assigns anonymous speaker labels such as `[S01]` and `[S02]` without a separate diarization pipeline. * **Promptable generation**: Supports custom transcription instructions, hotwords, and acoustic event annotations. ## Model Architecture

MOSS-Transcribe-Diarize 0.9B model architecture

| Component | Specification | |---|---| | Text backbone | Qwen3-0.6B style causal decoder | | Audio encoder | Whisper-Medium encoder configuration | | Audio frontend | `WhisperFeatureExtractor`, 16 kHz, 80 mel bins, 30 s chunks | | Audio-text bridge | 4x temporal merge + MLP adaptor | | Fusion | Audio features replace <|audio_pad|> embeddings via `masked_scatter` | | Output format | Compact `[start][Sxx]text[end]` transcript with speaker tags such as `[S01]` | This Hugging Face repository includes the custom Transformers remote code required to load the model with `trust_remote_code=True`. ## Evaluation We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Error Rate (CER), concatenated minimum-permutation Character Error Rate (cpCER), and Delta-cp. Lower is better for all metrics. A dash (`-`) indicates that the result is unavailable.
Model AISHELL‑4 Alimeeting Podcast Movies
CER↓cpCER↓Δcp↓ CER↓cpCER↓Δcp↓ CER↓cpCER↓Δcp↓ CER↓cpCER↓Δcp↓
Doubao 18.1827.869.68 25.2537.5712.31 7.9310.542.61 9.9430.8820.94
ElevenLabs 19.5837.9518.36 25.7036.6910.99 8.5011.342.85 11.4917.856.37
GPT-4o --- --- --- 14.3723.679.31
Gemini 2.5 Pro 42.7053.4210.72 27.4341.6414.21 7.3810.232.85 15.4624.158.69
Gemini 3 Pro 22.7527.434.68 26.7532.846.09 --- 8.6214.736.11
VIBEVOICE ASR 21.4024.993.59 27.4029.331.93 27.9448.3020.36 14.5942.5427.94
MOSS Transcribe Diarize 0.9B 14.8415.830.99 24.8622.17-2.69 5.977.371.40 6.3612.766.40
MOSS Transcribe Diarize Pro 13.7814.020.24 18.2213.94-4.27 4.466.972.51 5.8611.785.92
## Quickstart ### Environment Setup Use a clean Python environment. The model uses custom Transformers code, so load the model and processor with `trust_remote_code=True`. ```bash conda create -n moss-transcribe-diarize python=3.12 -y conda activate moss-transcribe-diarize git clone https://github.com/OpenMOSS/MOSS-Transcribe-Diarize.git cd MOSS-Transcribe-Diarize pip install --index-url https://download.pytorch.org/whl/cu128 torch torchaudio pip install -e . ``` The GitHub package provides helper utilities such as audio/video loading, transcription message construction, transcript parsing, CLI inference, and the subtitle web app. The model weights and remote-code model files are loaded from this Hugging Face repository. ### Python Usage ```python import torch from transformers import AutoModelForCausalLM, AutoProcessor from moss_transcribe_diarize import parse_transcript from moss_transcribe_diarize.inference_utils import ( build_transcription_messages, generate_transcription, resolve_device, ) model_id = "OpenMOSS-Team/MOSS-Transcribe-Diarize" audio_path = "audio.wav" device = resolve_device("auto") dtype = torch.bfloat16 if device.type == "cuda" else torch.float32 model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, dtype="auto", ).to(dtype=dtype).to(device).eval() processor = AutoProcessor.from_pretrained( model_id, trust_remote_code=True, ) messages = build_transcription_messages(audio_path) result = generate_transcription( model, processor, messages, max_new_tokens=2048, do_sample=False, device=device, dtype=dtype, ) print(result["text"]) for segment in parse_transcript(result["text"]): print(segment.start, segment.end, segment.speaker, segment.text) ``` The message flow follows the common Qwen multimodal pattern: 1. `processor.apply_chat_template(messages, tokenize=False)` renders text with audio placeholders. 2. The helper utilities load audio waveforms from the same messages. 3. `processor(text=text, audio=audios)` computes Whisper input features and expands audio placeholders. 4. `model.generate(...)` produces timestamped transcription and diarization text. ### Custom Prompt and Hotwords The default prompt is optimized for timestamped transcription and speaker diarization: ```text 请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。 ``` To add hotwords, append a short hint to the default prompt: ```text 请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。热词提示：热词1, 热词2, 热词3 ``` More prompt recipes are available in the GitHub repository: ### Serve with vLLM and SGLang MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`. ```bash uv pip install -U vllm \ --torch-backend=auto \ --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129 ``` or: ```bash uv pip install -U vllm \ --torch-backend=auto \ --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130 ``` ```bash vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code ``` ```bash curl http://localhost:8000/v1/audio/transcriptions \ -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \ -F file=@"audio.wav" \ -F response_format="json" \ -F temperature="0" ``` The recommended way to serve MOSS-Transcribe-Diarize is [SGLang Omni](https://github.com/sgl-project/sglang-omni) through the OpenAI-compatible `/v1/audio/transcriptions` endpoint. Install `sglang-omni` by following the [installation guide](https://github.com/sgl-project/sglang-omni/blob/main/docs/get_started/installation.md), then download the model: ```bash hf download OpenMOSS-Team/MOSS-Transcribe-Diarize ``` Serve the model: ```bash sgl-omni serve \ --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \ --port 8000 \ --max-running-requests 16 \ --cuda-graph-max-bs 16 \ --mem-fraction-static 0.80 ``` Use `response_format=verbose_json` when you need parsed speaker segments. `json` returns the raw transcript text only. ```bash curl -X POST http://localhost:8000/v1/audio/transcriptions \ -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \ -F file=@audio.wav \ -F response_format=verbose_json ``` ```python import requests with open("audio.wav", "rb") as f: resp = requests.post( "http://localhost:8000/v1/audio/transcriptions", data={ "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize", "response_format": "verbose_json", }, files={"file": ("audio.wav", f, "audio/wav")}, timeout=300, ) resp.raise_for_status() payload = resp.json() print(payload["text"]) for segment in payload.get("segments", []): print(f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}") ``` For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript: ```bash curl -X POST http://localhost:8000/v1/audio/transcriptions \ -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \ -F file=@audio.wav \ -F response_format=verbose_json \ -F max_new_tokens=65536 ``` | Parameter | Type | Default | Description | |---|---|---|---| | `file` | file | required | Audio file uploaded as multipart form data | | `model` | string | server default | Model identifier | | `language` | string | unset | Optional language hint | | `response_format` | string | `json` | `json`, `verbose_json`, or `text` | | `temperature` | float | model default (`0.0`) | Sampling temperature | | `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio, for example `65536` | | `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt | For benchmarking, performance numbers, and implementation details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md). The following single-H100 results are reported for short- and long-sequence multi-speaker ASR tasks. `movies` short-sequence ASR: | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s | |---:|---:|---:|---:|---:| | 1 | 2.57 | 0.388 | 0.0612 | 29.76 | | 2 | 4.89 | 0.409 | 0.0659 | 56.55 | | 4 | 6.62 | 0.513 | 0.0790 | 76.64 | | 8 | 6.80 | 0.533 | 0.0810 | 78.70 | | 16 | 7.08 | 0.659 | 0.0922 | 81.98 | `aishell4_long` long-sequence ASR: | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s | |---:|---:|---:|---:|---:| | 1 | 0.022 | 45.2 | 0.0197 | 50.64 | | 2 | 0.032 | 60.7 | 0.0265 | 74.25 | | 4 | 0.036 | 105.6 | 0.0461 | 81.64 | | 8 | 0.040 | 172.6 | 0.0754 | 90.62 | | 16 | 0.043 | 282.8 | 0.1237 | 98.83 | ### Subtitle Web App The source package includes a local subtitle workflow for upload, review, subtitle export, and optional FFmpeg burn-in: ```bash mtd-subtitle-web \ --model OpenMOSS-Team/MOSS-Transcribe-Diarize \ --host 127.0.0.1 \ --port 7860 ``` Open `http://127.0.0.1:7860`, upload an audio/video file, review the parsed subtitle segments, then download JSON/SRT/ASS or burn an MP4 if `ffmpeg` and `ffprobe` are available on `PATH`. For batch processing: ```bash mtd-subtitle /path/to/input.mp4 \ --model OpenMOSS-Team/MOSS-Transcribe-Diarize \ --out-dir runs/example \ --render ``` ## Output Format The canonical output format is: ```text [start_time][Sxx]transcribed speech[end_time] ``` Example: ```text [0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready for evaluation[13.81][14.36][S01]Great, include the diarization results in the report[18.76] ``` In this format: * `start_time` and `end_time` are timestamps in seconds. * `[S01]`, `[S02]`, and similar labels are anonymous model-generated speaker labels. * Speaker labels are relative labels within the input audio and should not be interpreted as real speaker identities. ## More Information * **GitHub**: * **MOSI.AI**: * **OpenMOSS**: ## License MOSS-Transcribe-Diarize 0.9B is licensed under the Apache License 2.0. ## Citation If you use MOSS-Transcribe-Diarize 0.9B, please cite the technical report: ```bibtex @misc{moss_transcribe_diarize_2026, title={MOSS Transcribe Diarize Technical Report}, author={{MOSI.AI}}, year={2026}, eprint={2601.01554}, archivePrefix={arXiv}, primaryClass={cs.SD}, url={https://arxiv.org/abs/2601.01554} } ```