---
title: GLM-5-MXFP4-FP8
canonical_url: "https://www.modelscope.cn/models/Apsara-Stack/GLM-5-MXFP4-FP8"
md_url: "https://www.modelscope.cn/models/Apsara-Stack/GLM-5-MXFP4-FP8.md"
repository: Apsara-Stack/GLM-5-MXFP4-FP8
last_updated: 2026-08-31
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - glm_moe_dsa
architectures:
  - GlmMoeDsaForCausalLM
base_model:
  - zai-org/GLM-5
base_model_relation: quantized
parameters: 386.6B
tensor_type:
  - F32
  - U8
  - F8_E4M3
  - BF16
library_name:
  - safetensors
downloads: 31
stars: 0
---

# GLM-5-MXFP4-FP8

> GLM-5-MXFP4-FP8 - Apsara-Stack 在 ModelScope 开源的模型。📢 Note: This repository was renamed from GLM-5-MXFP4-FP8 on 2026-08-21. The weights are unchanged; only the model name has been updated.

Apsara-Stack/GLM-5-MXFP4-FP8 是 ModelScope 魔搭社区上的 386.6B 参数text-generation模型，采用 mit 许可，基于 zai-org/GLM-5 构建。

- **Repository**: Apsara-Stack/GLM-5-MXFP4-FP8
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 386.6B
- **Base model**: zai-org/GLM-5
- **Downloads**: 31
- **Stars**: 0
- **Last updated**: 2026-08-31

Source: https://www.modelscope.cn/models/Apsara-Stack/GLM-5-MXFP4-FP8

---

📢 Note: This repository was renamed from GLM-5-MXFP4-FP8 on 2026-08-21. The weights are unchanged; only the model name has been updated.

# Model Overview

- **Model Architecture:** GlmMoeDsaForCausalLM
  - **Input:** Text
  - **Output:** Text
- **Inference Engine:** [SGLang](https://docs.sglang.io/)
- **Quantization:**
  - **dense (layers 0-2):** `up_proj`, `gate_proj`, `down_proj` 
    - **Weight quantization:** FP8, Static, Per-Channel
    - **Activation quantization:** FP8, Dynamic, Per-Token
  - **moe:** `up_proj`, `gate_proj`, `down_proj` 
    - **Weight quantization:** OCP MXFP4, Static
    - **Activation quantization:** OCP MXFP4, Dynamic
  - **shared_experts:** `up_proj`, `gate_proj`, `down_proj` 
    - **Weight quantization:** FP8, Static, Per-Channel
    - **Activation quantization:** FP8, Dynamic, Per-Token
  - **self_attn:** `q_a_proj`, `q_b_proj`, `kv_a_proj_with_mqa`, `kv_b_proj`, `o_proj`, `wq_b`, `wk`
    - **Weight quantization:** FP8, Static, Per-Channel
    - **Activation quantization:** FP8, Dynamic, Per-Token

This model was built with GLM-5 model by applying MXFP4 and FP8 quantization.

# Deployment
### Use with SGLang

This model can be deployed efficiently using the [SGLang](https://docs.sglang.io/) backend.

## Evaluation
The model was evaluated on IFEval GSM8K C-EVAL benchmarks. 

### Accuracy

<table>
  <tr>
   <td><strong>Benchmark</strong>
   </td>
   <td><strong>GLM-5</strong>
   </td>
   <td><strong>GLM-5-MXFP4-FP8 (this model)</strong>
   </td>
   <td><strong>Recovery</strong>
   </td>
  </tr>
  <tr>
   <td>IFEval
   </td>
   <td>92.42
   </td>
   <td>92.8
   </td>
   <td>100.41%
   </td>
  </tr>
  <tr>
   <td>GSM8K
   </td>
   <td>97.88
   </td>
   <td>97.5
   </td>
   <td>99.61%
   </td>
  </tr>
  <tr>
   <td>C-Eval
   </td>
   <td>92.57
   </td>
   <td>92.2
   </td>
   <td>99.6%
   </td>
  </tr>
</table>

### Reproduction

The IFEval GSM8K C-EVAL results were obtained using the `EvalScope` and `SGLang` framework.

Install T-head SGLang `(Version: 0.5.10)` and EvalScope `(Version: 1.5.0)` in container first.

#### Launching server
```
MODEL_PATH=<Path to GLM-5-MXFP4-FP8>
SGLANG_USE_MODELSCOPE=true python3 -m sglang.launch_server --model-path $MODEL_PATH \
  --tp-size 8 \
  --mem-fraction-static 0.85 \
  --disable-radix-cache \
  --trust-remote-code \
  --decode-attention-backend flashmla --prefill-attention-backend fa3 \
  --cuda-graph-max-bs 16 \
  --disable-shared-experts-fusion \
  --watchdog-timeout 6000 --dist-timeout 6000 \
  --served-model-name GLM-5
```

#### Evaluating model in a new terminal
```
evalscope eval \
  --model GLM-5 --api-url http://127.0.0.1:8000/v1  --api-key EMPTY \
  --eval-type openai_api --datasets gsm8k \
  --dataset-args '{"gsm8k": {"few_shot_num": 0, "filters": {"remove_until": "</think>"}}}' \
  --eval-batch-size 128 \
  --generation-config '{"max_tokens": 32768, "temperature": 0.6, "top_p": 0.95, "n": 1, "top_k": 20}' \
  --timeout 60000 --stream --work-dir outputs
```

# License
Copyright (c) 2026, T-Head (Shanghai) Semiconductor Co., Ltd.