---
title: GadflyII-GLM-4.7-Flash-NVFP4
canonical_url: "https://www.modelscope.cn/models/hf/GadflyII-GLM-4.7-Flash-NVFP4"
md_url: "https://www.modelscope.cn/models/hf/GadflyII-GLM-4.7-Flash-NVFP4.md"
repository: hf/GadflyII-GLM-4.7-Flash-NVFP4
last_updated: 2026-01-22
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - glm4_moe_lite
architectures:
  - Glm4MoeLiteForCausalLM
base_model:
  - zai-org/GLM-4.7-Flash
base_model_relation: quantized
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 2252
stars: 1
tags:
  - moe
  - nvfp4
  - quantized
  - vllm
  - glm
  - 30b
---

# GadflyII-GLM-4.7-Flash-NVFP4

> GadflyII-GLM-4.7-Flash-NVFP4 - hf 在 ModelScope 开源的模型。Note: If you have a multi-GPU SM120 Blackwell system (RTX 50/Pro), try my vLLM fork to resolve P2P / TP=2 issues (Pending PR into upstream). https://github.com/Gadflyii/vllm/tree/main

hf/GadflyII-GLM-4.7-Flash-NVFP4 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 zai-org/GLM-4.7-Flash 构建。

- **Repository**: hf/GadflyII-GLM-4.7-Flash-NVFP4
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: zai-org/GLM-4.7-Flash
- **Tags**: moe, nvfp4, quantized, vllm, glm, 30b
- **Downloads**: 2252
- **Stars**: 1
- **Last updated**: 2026-01-22

Source: https://www.modelscope.cn/models/hf/GadflyII-GLM-4.7-Flash-NVFP4

---

# Note: If you have a multi-GPU SM120 Blackwell system (RTX 50/Pro), try my vLLM fork to resolve P2P / TP=2 issues (Pending PR into upstream). 
https://github.com/Gadflyii/vllm/tree/main

# GLM-4.7-Flash NVFP4 (Mixed Precision)

This is a **mixed precision NVFP4 quantization** of [zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash), a 30B-A3B (30B total, 3B active) Mixture-of-Experts model.

## Quantization Strategy

This model was made via custom quantization and calibration (128 samples, 2048 max seq len, neuralmagic/calibration, all 64 experts) scripts based on NVIDIA's approach for DeepSeek-V3. It uses **mixed precision** to preserve accuracy:

| Component | Precision | Rationale |
|-----------|-----------|-----------|
| MLP Experts | FP4 (E2M1) | 64 routed experts, 4 active per token |
| Dense MLP | FP4 (E2M1) | First layer dense MLP |
| **Attention (MLA)** | **BF16** | Low-rank compressed Q/KV projections are sensitive |
| Norms, Gates, Embeddings | BF16 | Standard practice |

## Performance

| Metric | BF16 | Uniform FP4 | **This Model** |
|--------|------|-------------|----------------|
| MMLU-Pro | 24.83% | 16.84% | **23.55%** |
| Size | 62.4 GB | 18.9 GB | **20.4 GB** |
| Compression | 1x | 3.3x | **3.1x** |
| Accuracy Loss | - | -8.0% | **-1.3%** |


## Usage

### Requirements

- **vLLM**: 0.14.0+ (for compressed-tensors NVFP4 support)
- **transformers**: 5.0.0+ (for `glm4_moe_lite` architecture)
- **GPU**: NVIDIA GPU with FP4 tensor core support (Blackwell, Hopper, Ada Lovelace)

### Installation

```bash
pip install vllm>=0.14.0
pip install git+https://github.com/huggingface/transformers.git
```

### Inference with vLLM

```python
from vllm import LLM, SamplingParams

model = LLM(
    "GadflyII/GLM-4.7-Flash-NVFP4",
    tensor_parallel_size=1,
    max_model_len=4096,
    trust_remote_code=True,
    gpu_memory_utilization=0.85,
)

params = SamplingParams(temperature=0.7, max_tokens=512)
outputs = model.generate(["Explain quantum computing in simple terms."], params)
print(outputs[0].outputs[0].text)
```

### Serving with vLLM

```bash
vllm serve GadflyII/GLM-4.7-Flash-NVFP4 \
    --tensor-parallel-size 1 \
    --max-model-len 4096 \
    --trust-remote-code
```

## Model Details

- **Base Model**: [zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash)
- **Architecture**: `Glm4MoeLiteForCausalLM`
- **Parameters**: 30B total, 3B active per token (30B-A3B)
- **MoE Configuration**: 64 routed experts, 4 active, 1 shared expert
- **Layers**: 47
- **Context Length**: 202,752 tokens (max)
- **Languages**: English, Chinese

## Quantization Details

- **Format**: compressed-tensors (NVFP4)
- **Block Size**: 16
- **Scale Format**: FP8 (E4M3)
- **Calibration**: 128 samples from neuralmagic/calibration dataset
- **Full Expert Calibration**: All 64 experts calibrated per sample

## Evaluation

### MMLU-Pro Overall Results

| Model | Accuracy | Correct | Total |
|-------|----------|---------|-------|
| **BF16 (baseline)** | **24.83%** | 2988 | 12032 |
| **NVFP4 (this model)** | **23.55%** | 2834 | 12032 |
| **Difference** | **-1.28%** | -154 | - |

### MMLU-Pro by Category

| Category | BF16 | NVFP4 | Difference |
|----------|------|-------|------------|
| Social Sciences | 32.70% | 31.43% | -1.27% |
| Other | 31.57% | 30.08% | -1.49% |
| Humanities | 23.78% | 22.56% | -1.22% |
| STEM | 19.94% | 18.70% | -1.24% |

### MMLU-Pro by Subject

| Subject | BF16 | NVFP4 | Difference |
|---------|------|-------|------------|
| Biology | 50.35% | 47.42% | -2.93% |
| Psychology | 44.99% | 42.48% | -2.51% |
| Economics | 36.37% | 34.48% | -1.89% |
| Health | 35.21% | 34.84% | -0.37% |
| History | 33.60% | 30.71% | -2.89% |
| Philosophy | 31.46% | 30.06% | -1.40% |
| Other | 28.35% | 25.87% | -2.48% |
| Computer Science | 26.10% | 21.46% | -4.64% |
| Business | 16.35% | 16.98% | +0.63% |
| Law | 16.89% | 16.35% | -0.54% |
| Engineering | 16.00% | 14.04% | -1.96% |
| Physics | 15.32% | 14.70% | -0.62% |
| Math | 14.06% | 14.29% | +0.23% |
| Chemistry | 14.13% | 13.34% | -0.79% |

## Citation

If you use this model, please cite the original GLM-4.7-Flash:

```bibtex
@misc{glm4flash2025,
  title={GLM-4.7-Flash},
  author={Zhipu AI},
  year={2025},
  howpublished={\url{https://huggingface.co/zai-org/GLM-4.7-Flash}}
}
```

## License

This model inherits the Apache 2.0 license from the base model.