---
title: Qwen2.5-Omni-3B-Thinker-8bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Qwen2.5-Omni-3B-Thinker-8bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Qwen2.5-Omni-3B-Thinker-8bit.md"
repository: mlx-community/Qwen2.5-Omni-3B-Thinker-8bit
last_updated: 2026-11-02
license: other
model_type:
  - qwen2_5_omni_thinker_encoder
base_model:
  - Qwen/Qwen2.5-Omni-3B
base_model_relation: finetune
parameters: 1.3B
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 47
stars: 4
tags:
  - mlx
  - mlx-swift
  - qwen2.5-omni
---

# Qwen2.5-Omni-3B-Thinker-8bit

> Qwen2.5-Omni-3B-Thinker-8bit - mlx-community 在 ModelScope 开源的模型。mlx-community/Qwen2.5-Omni-3B-Thinker-8bit

mlx-community/Qwen2.5-Omni-3B-Thinker-8bit 是 ModelScope 魔搭社区上的 1.3B 参数机器学习模型，采用 other 许可，基于 Qwen/Qwen2.5-Omni-3B 构建。

- **Repository**: mlx-community/Qwen2.5-Omni-3B-Thinker-8bit
- **License**: other
- **Parameters**: 1.3B
- **Base model**: Qwen/Qwen2.5-Omni-3B
- **Tags**: mlx, mlx-swift, qwen2.5-omni
- **Downloads**: 47
- **Stars**: 4
- **Last updated**: 2026-11-02

Source: https://www.modelscope.cn/models/mlx-community/Qwen2.5-Omni-3B-Thinker-8bit

---

# mlx-community/Qwen2.5-Omni-3B-Thinker-8bit

The **thinker** of [Qwen/Qwen2.5-Omni-3B](https://huggingface.co/Qwen/Qwen2.5-Omni-3B): the Qwen2 language model
(36 × 2048) and the Whisper-shaped audio tower. It is converted to MLX as the semantic encoder of
[AuK](https://huggingface.co/mlx-community/AuK-bf16) in [xocialize/mlx-auk-swift](https://github.com/xocialize/mlx-auk-swift).
The vision tower, LM head, talker and token2wav are not included. This is an encoder for its hidden states, not a chat
model.

The 36 LLM layers and 32 audio-tower layers have their Linears quantised to affine 8-bit, group size 64 (4.26 GB). Embeddings, norms and projections are bf16.

## ⚠️ Licence

**Qwen RESEARCH LICENSE AGREEMENT**: research or evaluation use only, non-commercial. The full text is in
[LICENSE](LICENSE), and the attribution notice in [NOTICE](NOTICE). For commercial use, request a licence from Alibaba
Cloud.

These are **modified files**: converted (and, for -8bit / -4bit, quantised) from the original.

**Built with Qwen.**

## Files

- `encoder.safetensors` with `thinker.model.*` and `thinker.audio_tower.*` keys; audio conv weights in MLX (O, K, I)
  layout.
- `config.json` (text, audio and token ids, plus `quantization` when quantised).
- The original tokenizer, chat template and preprocessor files, unchanged.

## Parity (as AuK's encoder)

At fp32 against the transformers 4.57.1 reference:
- token ids identical;
- Whisper features relL2 1.5e-6;
- audio tower 3.9e-6;
- last hidden state 2.2e-5;
- AuK's fused 36-layer condition 3.8e-6.

As AuK's encoder, the bf16, 8-bit and 4-bit tiers all pass AuK's pre-registered quality gate (error rate, CAM++ SIM,
DNSMOS against fp32) in English. On Mandarin content the 4-bit tier fails it: AuK's de-accent CER goes from 3.6 % to
17.9 %, and content words are lost. Use the 8-bit tier for Mandarin edits.