---
title: Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken
canonical_url: "https://www.modelscope.cn/models/T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken"
md_url: "https://www.modelscope.cn/models/T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken.md"
repository: T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken
last_updated: 2026-10-10
license: apache-2.0
model_type:
  - qwen4_exp
architectures:
  - Qwen4ExpForConditionalGeneration
base_model:
  - Qwen/Qwen3.8-Flash-Next
base_model_relation: quantized
parameters: 180.0B
tensor_type:
  - F32
  - BF16
  - I64
  - I8
library_name:
  - safetensors
downloads: 44
stars: 0
---

# Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken

> Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken - T-HEAD 在 ModelScope 开源的模型。Release Date: 2026-10-09 Model Architecture: Qwen4ExpForConditionalGeneration Input: Image-Text Output: Text Inference Engine: vLLM Quantization: moe: upproj,…

T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken 是 ModelScope 魔搭社区上的 180.0B 参数机器学习模型，采用 apache-2.0 许可，基于 Qwen/Qwen3.8-Flash-Next 构建。

- **Repository**: T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken
- **License**: apache-2.0
- **Parameters**: 180.0B
- **Base model**: Qwen/Qwen3.8-Flash-Next
- **Downloads**: 44
- **Stars**: 0
- **Last updated**: 2026-10-10

Source: https://www.modelscope.cn/models/T-HEAD/Qwen3.8-Flash-Next-MoE-Quant-W-INT8-PerChannel-A-INT8-PerToken

---

# Model Overview

- **Release Date:** 2026-10-09
- **Model Architecture:** Qwen4ExpForConditionalGeneration
  - **Input:** Image-Text
  - **Output:** Text
- **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
- **Quantization:**
  - **moe:** `up_proj`, `gate_proj`, `down_proj`
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **shared_expert:** `up_proj`, `gate_proj`, `down_proj` 
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **linear_attn:** `in_proj_qkv`, `in_proj_z`, `out_proj`
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **self_attn:** `q_proj`, `k_proj`, `v_proj`, `o_proj`
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token

This model was built with Qwen3.8-Flash-Next model by applying W8A8-INT8 quantization.

# Deployment
### Use with vLLM

This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.

## Evaluation
The model was evaluated on C-EVAL GSM8K Humaneval_plus benchmarks. 

### Accuracy

<table>
  <tr>
   <td><strong>Benchmark</strong>
   </td>
   <td><strong>Qwen3.8-Flash-Next</strong>
   </td>
   <td><strong>Qwen3.8-Flash-Next-Quant-W-INT8-PerChannel-A-INT8-PerToken (this model)</strong>
   </td>
   <td><strong>Recovery</strong>
   </td>
  </tr>
  <tr>
   <td>C-EVAL
   </td>
   <td>93.31
   </td>
   <td>93.83
   </td>
   <td>100.56%
   </td>
  </tr>
  <tr>
   <td>GSM8K
   </td>
   <td>97.42
   </td>
   <td>98.03
   </td>
   <td>100.63%
   </td>
  </tr>
  <tr>
   <td>Humaneval_plus
   </td>
   <td>95.73
   </td>
   <td>95.12
   </td>
   <td>99.36%
   </td>
  </tr>
</table>

### Reproduction

The C-EVAL GSM8K Humaneval_plus results were obtained using the `EvalScope` and `vLLM` framework.

Install T-head vLLM `(Version: 0.28.0)` and EvalScope `(Version: 1.5.1)` in container first.

#### Launching server
```
MODEL_PATH=<Path to Qwen3.8-Flash-Next-Quant-W-INT8-PerChannel-A-INT8-PerToken>
VLLM_USE_MODELSCOPE=true vllm serve $MODEL_PATH \
  --trust-remote-code --enable-auto-tool-choice --tensor-parallel-size 8 \
  --max-model-len auto --reasoning-parser qwen3 --tool-call-parser qwen3_xml \
  --max-num-seqs 256 --gpu-memory-utilization 0.95 --served-model-name Qwen3.8-Flash-Next \
  --enable-prefix-caching --port 8999
```

#### Evaluating model in a new terminal
```
evalscope eval \
  --model Qwen3.8-Flash-Next --api-url http://127.0.0.1:8999/v1  --api-key EMPTY \
  --eval-type openai_api --datasets gsm8k \
  --dataset-args '{"gsm8k": {"few_shot_num": 0, "filters": {"remove_until": "</think>"}}}' \
  --generation-config '{"max_tokens": 32768, "temperature": 1, "top_p": 0.95, "n": 1, "min_p": "0.0", "presence_penalty": "0.0", "repetition_penalty": "1.0", "top_k": 20}' \
  --timeout 60000 --stream --work-dir outputs
```

# License
Copyright (c) 2026, T-Head (Shanghai) Semiconductor Co., Ltd.