---
title: DeepSeek-R1-fp4
canonical_url: "https://www.modelscope.cn/models/qingcheng-ai/DeepSeek-R1-fp4"
md_url: "https://www.modelscope.cn/models/qingcheng-ai/DeepSeek-R1-fp4.md"
repository: qingcheng-ai/DeepSeek-R1-fp4
last_updated: 2025-07-10
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - deepseek_v3
architectures:
  - DeepseekV3ForCausalLM
base_model:
  - deepseek-ai/DeepSeek-R1
base_model_relation: quantized
parameters: 378.3B
tensor_type:
  - BF16
  - U8
  - F8_E4M3
  - F32
library_name:
  - safetensors
inference_backends:
  - "deploy_task text/emb"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 61
stars: 0
tags:
  - deepseek
  - fp4
---

# DeepSeek-R1-fp4

> DeepSeek-R1-fp4 - qingcheng-ai 在 ModelScope 开源的模型。All linear in DeepSeek-R1 are quantized to FP4 precision.

qingcheng-ai/DeepSeek-R1-fp4 是 ModelScope 魔搭社区上的 378.3B 参数text-generation模型，采用 apache-2.0 许可，基于 deepseek-ai/DeepSeek-R1 构建，可用 deploy_task text/emb、sglang 0.5.2、vllm 0.9.2 部署。

- **Repository**: qingcheng-ai/DeepSeek-R1-fp4
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 378.3B
- **Base model**: deepseek-ai/DeepSeek-R1
- **Inference backends**: deploy_task text/emb, sglang 0.5.2, vllm 0.9.2
- **Tags**: deepseek, fp4
- **Downloads**: 61
- **Stars**: 0
- **Last updated**: 2025-07-10

Source: https://www.modelscope.cn/models/qingcheng-ai/DeepSeek-R1-fp4

---

# Model Overview

## Description

All linear in DeepSeek-R1 are quantized to FP4 precision.

## Evaluation


The table below summarizes our results on the MMLU benchmark. 

To expedite testing, we randomly selected 25 samples from each subset of the MMLU dataset. 

Our experiments show that the **FP4 quantized model achieves accuracy on par with the BF16 version while delivering faster inference.**


| Data Format | MMLU Score |
|:---|:---|
| BF16 | 84.36 |
| FP4 | 84.84 |
| AWQ (INT4) | 53.68 |

## Quickstart
We recommend using the Chitu inference framework(https://github.com/thu-pacman/chitu) to run this model.

Here provides a simple command to show you how to run DeepSeek-R1-fp4.

```bash
torchrun \
    --nproc_per_node 8 \
    --master_port=22525 \
    -m chitu \
    serve.port=21002 \
    infer.cache_type=paged \
    infer.attn_type=flash_mla \
    infer.mla_absorb=absorb \
    models=DeepSeek-R1-fp4-all \
    models.ckpt_dir="your model path" \
    infer.tp_size=8 \
    infer.max_reqs=2 \
    infer.max_seq_len=2048 \
    request.max_new_tokens=512 \
    infer.use_cuda_graph=True
```

## Contact

solution@qingcheng.ai