---
title: Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4
canonical_url: "https://www.modelscope.cn/models/lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4"
md_url: "https://www.modelscope.cn/models/lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4.md"
repository: lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4
last_updated: 2026-10-15
license: other
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - qwen4_exp
architectures:
  - Qwen4ExpForConditionalGeneration
base_model:
  - Qwen/Qwen3.8-Flash-Next
base_model_relation: quantized
parameters: 118.3B
tensor_type:
  - I64
  - F32
  - U8
  - F8_E4M3
  - BF16
library_name:
  - safetensors
downloads: 1936
stars: 0
tags:
  - ModelOpt
  - Qwen3.8
  - quantized
  - MXFP4
  - mxfp4
  - FP4
  - fp4
  - FP8
  - fp8
  - mixed-precision
  - blockwise
  - speculative-decoding
  - MTP
  - sglang
---

# Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4

> Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4 - lovedheart 在 ModelScope 开源的模型。## ⚠️ Requires a patched SGLang — stock builds emit GARBLED OUTPUT This is a mixed-precision checkpoint with three quantization domains (see config.json →…

lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4 是 ModelScope 魔搭社区上的 118.3B 参数image-text-to-text模型，采用 other 许可，基于 Qwen/Qwen3.8-Flash-Next 构建。

- **Repository**: lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4
- **License**: other
- **Tasks**: image-text-to-text
- **Parameters**: 118.3B
- **Base model**: Qwen/Qwen3.8-Flash-Next
- **Tags**: ModelOpt, Qwen3.8, quantized, MXFP4, mxfp4, FP4, fp4, FP8, fp8, mixed-precision, blockwise, speculative-decoding, MTP, sglang
- **Downloads**: 1936
- **Stars**: 0
- **Last updated**: 2026-10-15

Source: https://www.modelscope.cn/models/lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4

---

> ## ⚠️ Requires a patched SGLang — stock builds emit GARBLED OUTPUT
>
> This is a **mixed-precision** checkpoint with three quantization domains
> (see `config.json` → `quantization_config.quantized_layers`):
>
> | Domain                                       | quant_algo       | Details                                                |
> |----------------------------------------------|------------------|--------------------------------------------------------|
> | 48 main-model routed MoE experts             | `W4A8_MXFP4_FP8` | MXFP4 (E2M1, group 32) weights × MXFP8 dynamic activations |
> | 1 MTP draft layer routed MoE experts         | `W4A8_MXFP4_FP8` | same layout as the main experts                         |
> | attention / `linear_attn` projections (156)  | `FP8_PB_WO`      | FP8 E4M3 2D-blockwise (128×128) weight-only             |
>
> **Stock SGLang dispatches neither `W4A8_MXFP4_FP8` nor `FP8_PB_WO` for
> `Qwen4ExpForConditionalGeneration`.** MXFP4 expert layers fall through to the
> plain FP8 MoE method (wrong scales → garbage), and `FP8_PB_WO` layers fall
> through to `UnquantizedLinearMethod`, which loads packed FP8 bytes as BF16
> **silently — the server starts without any error and produces garbage text.**
>
> Run it on the **`qsa-on-main`** branch of
> **https://github.com/lovedheart/sglang** (sgl-project/sglang based; this
> checkpoint is served in production from `qsa-on-main` @ `dfbe88e91d`,
> 2026-10-14, with the additive `W4A8_MXFP4_FP8` wiring: modelopt_mixed →
> `use_mxfp8 + is_fp4_experts` → MXFP4 flashinfer-CUTLASS MoE method, plus the
> SM120 auto-override to `--moe-runner-backend flashinfer_mxfp4`).
>
> Startup sanity check: log shows `quant=modelopt_mixed` and the MoE layers
> (including `mtp.layers.0.mlp.experts`) resolve to the MXFP4 CUTLASS method;
> **zero** `not found in params_dict, skip loading` lines for `mtp.*` or for
> any `*.weight_scale_inv`.

# Model Overview

## Description:
Qwen3.8-Flash-Next **MXFP4-FP8 with an MXFP4 MTP draft** is the quantized
version of [Qwen/Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next),
a hybrid-architecture multimodal Mixture-of-Experts model. The main-model
routed experts use the **MXFP4 (W4A8) recipe** (same layout as DeepSeek-V4's
MXFP4 path) instead of NVFP4, and — unlike the sibling
`Qwen3.8-Flash-Next-MXFP4-FP8` — **the MTP draft layer's routed experts are
quantized to MXFP4 as well** (`model-mtp-mxfp4.safetensors`,
`mtp.layers.0.mlp.experts` only; draft attention and shared expert stay BF16),
so speculative decoding runs the draft itself in low precision.

This checkpoint is a private candidate release.

## References
NVIDIA Model Optimizer: https://github.com/NVIDIA/TensorRT-Model-Optimizer

### License/Terms of Use:
See the [source model](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) for
license terms.

### Use Case:
Developers evaluating MXFP4-quantized serving (W4A8 MoE + FP8 attention) of
Qwen3.8-Flash-Next with MXFP4 MTP speculative decoding, for agentic systems,
chat, coding, and multimodal reasoning workloads.

## Model Architecture:
**Architecture Type:** Transformer (hybrid GDN + QSA sparse attention,
multi-hyperconnection streams, PLE n-gram injection) <br>
**Network Architecture:** Multimodal MoE — 48 decoder layers, 512 routed
experts per MoE layer (top-10 routing) + shared expert, 1 MTP layer <br>
**Number of Model Parameters:** ~180B in total (360 GB BF16 source) <br>

## Input:
**Input Type(s):** Text, Image, Video <br>
**Input Format(s):** String, RGB, Video <br>
**Other Properties Related to Input:** Context length up to 262K <br>

## Output:
**Output Type(s):** Text <br>
**Output Format:** String <br>

## Software Integration:
**Supported Runtime Engine(s):** <br>
* SGLang (with `qwen4_exp` model support **and** the W4A8_MXFP4_FP8 /
  FP8_PB_WO dispatch from the `qsa-on-main` branch; see the warning above) <br>

**Supported Hardware Microarchitecture Compatibility:** <br>
* NVIDIA Blackwell SM120 (validated on RTX PRO 6000 Blackwell via the
  flashinfer MXFP4 CUTLASS MoE path) <br>

**Preferred Operating System(s):** <br>
* Linux <br>

## Model Version(s):
MXFP4 candidate, quantized with nvidia-modelopt v0.46.0 (snapshot
`87c9f8cf83021957d1a1a575c90c9a4eaaf7ef0c`; CUDA 13.0 / torch 2.13.0+cu130 —
see `conversion_environment.json`). The MTP-MXFP4 requantization was applied
on 2026-10-14 (report: `mtp_mxfp4_quant_report.json`).

## Post Training Quantization
**Only the routed experts are quantized to MXFP4** (48 main-model layers +
the MTP draft layer; per-expert packed E2M1 U8 `[N, K/2]` + E8M0 block scales
U8 `[N, K/32]`, low nibble = even K index — byte layout identical to the
DeepSeek-V4 MXFP4 path; W4A8 with dynamic MXFP8 activations). Attention /
GDN projections are FP8 E4M3 128×128-block weight-only (`FP8_PB_WO`,
per-tensor rel-err in `fp8_quant_report.csv`). Routers, shared experts, mHC,
embeddings, LM head, and the MTP attention/shared-expert tensors remain BF16.
The PLE n-gram embedding tables use the FP8-quantized versions from the
`Qwen/Qwen3.8-Flash-Next-FP8` revision (declared via
`text_config.ple_embedding_dtype = float8_e4m3fn`). No KV-cache quantization
metadata. Checkpoint size: ~125 GB (from 360 GB BF16).

## Usage

Serve with SGLang (`qsa-on-main` branch build; production command, trimmed to
the quantization-relevant flags):

```sh
python -m sglang.launch_server \
  --model-path lovedheart/Qwen3.8-Flash-Next-MXFP4-FP8-MTP-MXFP4 \
  --trust-remote-code \
  --moe-runner-backend flashinfer_mxfp4 \
  --speculative-algo NEXTN --speculative-num-steps 3 \
  --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
  --speculative-moe-runner-backend flashinfer_mxfp4 \
  --kv-cache-dtype nvfp4 \
  --page-size 64 --chunked-prefill-size 4096 --max-prefill-tokens 16384 \
  --mamba-backend triton --mamba-ssm-dtype bfloat16
```

(On SM120 the branch's arch override selects `flashinfer_mxfp4` automatically
when `--moe-runner-backend` is left at `auto`.)

## Evaluation
**Caveat: the `gsm8k_metrics.json`, `aime26_metrics.json`, and
`smoke_report.json` shipped in this repo were recorded on earlier revisions of
this model line, not on this MXFP4 checkpoint.** They are kept as pipeline
references only. This checkpoint is characterized so far by weight-level
validation — per-layer MXFP4 relative error vs BF16 in
`mxfp4_validate_report.json` (~11.7–12.1% per projection) and
`mtp_mxfp4_quant_report.json` — plus deterministic serving smoke; end-to-end
GSM8K/AIME runs on the MXFP4 expert layout had not been re-recorded at
publish time.

## Integrity Evidence
- Structural audit: 152,245 tensor entries / 207 weight shards, all files
  referenced by `model.safetensors.index.json` present (0 missing / 0 extra).
- Unchanged-content byte-equality audit: 1,562 tensors / 118.4 GB compared
  against source precision, verdict **pass** (`audit_unchanged_report.json`).
- MXFP4 per-layer scale validation: `mxfp4_validate_report.json`; FP8
  projection rel-err table: `fp8_quant_report.csv`; MTP requant scope and
  rel-err: `mtp_mxfp4_quant_report.json`.
- 192 per-shard conversion receipts (`layer-*.complete.json`) are intentionally
  not published to this repo (intermediate artifacts, same policy as the
  sibling checkpoints).

## Model Limitations:
The base model was trained on data that contains toxic language and societal
biases originally crawled from the internet. Therefore, the model may amplify
those biases and return toxic responses especially when prompted with toxic
prompts. The model may generate answers that may be inaccurate, omit key
information, or include irrelevant or redundant text producing socially
unacceptable or undesirable text, even if the prompt itself does not include
anything explicitly offensive.