---
title: Qwen3.5-9B-MTP-5bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Qwen3.5-9B-MTP-5bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Qwen3.5-9B-MTP-5bit.md"
repository: mlx-community/Qwen3.5-9B-MTP-5bit
last_updated: 2026-06-02
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3_5_mtp
base_model:
  - Qwen/Qwen3.5-9B
base_model_relation: quantized
parameters: 45.6M
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 25
stars: 0
tags:
  - mlx
  - mlx-vlm
  - qwen3_5_mtp
  - qwen3.5
  - qwen3.5-9b
  - qwen
  - mtp
  - speculative-decoding
  - draft-model
  - 5-bit
---

# Qwen3.5-9B-MTP-5bit

> Qwen3.5-9B-MTP-5bit - mlx-community 在 ModelScope 开源的模型。This repository contains Multi-Token Prediction (MTP) drafter weights split from Qwen/Qwen3.5-9B for use with mlx-vlm speculative decoding.

mlx-community/Qwen3.5-9B-MTP-5bit 是 ModelScope 魔搭社区上的 45.6M 参数text-generation模型，采用 apache-2.0 许可，基于 Qwen/Qwen3.5-9B 构建。

- **Repository**: mlx-community/Qwen3.5-9B-MTP-5bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 45.6M
- **Base model**: Qwen/Qwen3.5-9B
- **Tags**: mlx, mlx-vlm, qwen3_5_mtp, qwen3.5, qwen3.5-9b, qwen, mtp, speculative-decoding, draft-model, 5-bit
- **Downloads**: 25
- **Stars**: 0
- **Last updated**: 2026-06-02

Source: https://www.modelscope.cn/models/mlx-community/Qwen3.5-9B-MTP-5bit

---

# Qwen3.5-9B-MTP-5bit

This repository contains Multi-Token Prediction (MTP) drafter weights split from `Qwen/Qwen3.5-9B` for use with `mlx-vlm` speculative decoding.

This is not a standalone chat or text-generation model. Load it as the draft model alongside a compatible Qwen3.5 9B target checkpoint.

## Use with mlx-vlm

```bash
uv run mlx_vlm.generate \
  --model mlx-community/Qwen3.5-9B-5bit \
  --draft-model mlx-community/Qwen3.5-9B-MTP-5bit \
  --prompt "Hi, how are you?" \
  --max-tokens 256 \
  --enable-thinking
```

For local weights:

```bash
uv run mlx_vlm.generate \
  --model /path/to/target-model \
  --draft-model /path/to/Qwen3.5-9B-mtp-5bit \
  --prompt "Hi, how are you?" \
  --max-tokens 256 \
  --enable-thinking
```

## Model Details

- Model type: `qwen3_5_mtp`
- MTP block size: `2`
- Target architecture: Qwen3.5 9B
- Precision: MLX affine 5-bit, group size 64
- Runtime: MLX / `mlx-vlm`
- Format: Safetensors with MLX-compatible config and tokenizer files

The stored tensors use MLX affine 5-bit quantization as described in `config.json`.

## Intended Use

Use this repo only as a speculative decoding drafter for compatible Qwen3.5 9B checkpoints. The target model verifies drafted tokens, while this MTP model proposes candidate tokens per decoding step.

## Limitations

This checkpoint requires runtime support for Qwen/DeepSeek MTP draft models in `mlx-vlm`. Standard standalone generation through generic Transformers APIs is not expected to work with this repository by itself.

Please refer to the upstream `Qwen/Qwen3.5-9B` model card and license terms for model usage constraints.