---
title: Hy3-preview-MTP-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Hy3-preview-MTP-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Hy3-preview-MTP-4bit.md"
repository: mlx-community/Hy3-preview-MTP-4bit
last_updated: 2026-07-12
license: other
base_model:
  - tencent/Hy3-preview
base_model_relation: finetune
parameters: 586.5M
tensor_type:
  - F32
  - U32
  - BF16
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 11
stars: 0
tags:
  - mlx
  - speculative-decoding
  - mtp
  - multi-token-prediction
  - hunyuan
---

# Hy3-preview-MTP-4bit

> Hy3-preview-MTP-4bit - mlx-community 在 ModelScope 开源的模型。Hy3-preview-MTP-4bit

mlx-community/Hy3-preview-MTP-4bit 是 ModelScope 魔搭社区上的 586.5M 参数机器学习模型，采用 other 许可，基于 tencent/Hy3-preview 构建。

- **Repository**: mlx-community/Hy3-preview-MTP-4bit
- **License**: other
- **Parameters**: 586.5M
- **Base model**: tencent/Hy3-preview
- **Tags**: mlx, speculative-decoding, mtp, multi-token-prediction, hunyuan
- **Downloads**: 11
- **Stars**: 0
- **Last updated**: 2026-07-12

Source: https://www.modelscope.cn/models/mlx-community/Hy3-preview-MTP-4bit

---

# Hy3-preview-MTP-4bit

Native **Multi-Token-Prediction (MTP)** sidecar for
[`mlx-community/Hy3-preview-4bit`](https://huggingface.co/mlx-community/Hy3-preview-4bit),
for use as a self-speculative draft head with
[rapid-mlx](https://github.com/machinefi/rapid-mlx).

## What this is

Tencent's Hunyuan 3 (`model_type=hy_v3`) ships a **DeepSeek-V3-style native MTP
head** as the final decoder layer (`model.layers.80.*`) of the full-precision
[`tencent/Hy3-preview`](https://huggingface.co/tencent/Hy3-preview) checkpoint.
The 4-bit MLX conversion `mlx-community/Hy3-preview-4bit` keeps only layers
`0..79` (the backbone) and **strips** the MTP head.

This repo re-supplies just that head, quantized to match the base checkpoint,
as a single-file sidecar (`model-mtp.safetensors`, 44 tensors). rapid-mlx loads
the base 4-bit backbone and grafts this head at boot to run **self-speculative
decoding** (one draft token per verify step, K=1 chain MTP).

## Provenance

Extracted from `tencent/Hy3-preview` shards `model-00111-of-00112` +
`model-00112-of-00112` (the two shards holding layer 80). The 593 layer-80
tensors are remapped to the rapid-mlx MTP param tree:

* `enorm` / `hnorm` — RMSNorms on the next-token embedding and previous hidden
  state (DeepSeek-V3 convention).
* `eh_proj` — the `2H -> H` fused projection, applied as
  `eh_proj(concat([enorm(embed_next), hnorm(prev_hidden)], -1))` (embedding
  first, confirmed against vLLM `deepseek_mtp.py` and SGLang `hunyuan` nextn).
* one HY3 `DecoderLayer` on the MoE branch (QK-norm attention + sigmoid-router
  SwitchGLU MoE over 192 experts + a shared expert).
* `norm` — the head's final RMSNorm (upstream `final_layernorm`).

## Quantization

Matches the base checkpoint: **4-bit `group_size=64` affine** for every Linear
(`eh_proj`, attention projections, `switch_mlp.*`, `shared_mlp.*`); **8-bit
`group_size=64`** for `mlp.router.gate`; all RMSNorms and `router.expert_bias`
kept in full precision.

## Usage

```bash
rapid-mlx serve hy3-preview-4bit --speculative-config '{"method":"mtp"}'
```

rapid-mlx auto-resolves and downloads this sidecar. The base 4-bit backbone
loads normally; the MTP head is grafted at boot for self-speculative decoding.

## Measured

* Draft accept rate ~58% (K=1) across code / chinese / reasoning / list prompts.
* Greedy output is batched-consistent lossless vs the MTP-off reference.

The projection tree is byte-identical to a quantized backbone MoE layer, so the
sidecar param names line up 1:1 with the runtime module.