---
title: DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken
canonical_url: "https://www.modelscope.cn/models/T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken"
md_url: "https://www.modelscope.cn/models/T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken.md"
repository: T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken
last_updated: 2026-11-12
license: mit
model_type:
  - deepseek_v41
architectures:
  - DeepseekV41ForCausalLM
base_model:
  - deepseek-ai/DeepSeek-V4.1-Flash-1017
base_model_relation: quantized
parameters: 763.4B
tensor_type:
  - I8
  - F32
  - BF16
  - F8_E8M0
  - F8_E4M3
library_name:
  - safetensors
downloads: 284
stars: 6
---

# DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken

> DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken - T-HEAD 在 ModelScope 开源的模型。Release Date: 2026-11-05 Model Architecture: DeepseekV41ForCausalLM Input: Image-Text Output: Text Inference Engine: SGLang Quantization: moe: w1, w2, w3 - Weight…

T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken 是 ModelScope 魔搭社区上的 763.4B 参数机器学习模型，采用 mit 许可，基于 deepseek-ai/DeepSeek-V4.1-Flash-1017 构建。

- **Repository**: T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken
- **License**: mit
- **Parameters**: 763.4B
- **Base model**: deepseek-ai/DeepSeek-V4.1-Flash-1017
- **Downloads**: 284
- **Stars**: 6
- **Last updated**: 2026-11-12

Source: https://www.modelscope.cn/models/T-HEAD/DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken

---

# Model Overview

- **Release Date:** 2026-11-05
- **Model Architecture:** DeepseekV41ForCausalLM
  - **Input:** Image-Text
  - **Output:** Text
- **Inference Engine:** [SGLang](https://docs.sglang.io/)
- **Quantization:**
  - **moe:** `w1`, `w2`, `w3` 
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **shared_experts:** `w1`, `w2`, `w3` 
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **attn:** `wq_a`, `wq_b`, `wkv`, `wo_a`, `wo_b`, `indexer.wq_b` 
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token
  - **engram:** `embed`
    - **Weight quantization:** MXFP8, Static, Per-Group
  - **mtp:** `main_proj` 
    - **Weight quantization:** INT8, Static, Per-Channel
    - **Activation quantization:** INT8, Dynamic, Per-Token

This model was built with DeepSeek-V4.1-Flash-1017 model by applying W8A8-INT8 quantization.

# Deployment
### Use with SGLang

This model can be deployed efficiently using the [SGLang](https://docs.sglang.io/) backend.

## Evaluation
The model was evaluated on C-EVAL GSM8K MM_Star Math_Vista benchmarks. 

### Accuracy

<table>
  <tr>
   <td><strong>Benchmark</strong>
   </td>
   <td><strong>DeepSeek-V4.1-Flash-1017</strong>
   </td>
   <td><strong>DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken (this model)</strong>
   </td>
   <td><strong>Recovery</strong>
   </td>
  </tr>
  <tr>
   <td>C-EVAL
   </td>
   <td>94.35
   </td>
   <td>94.5
   </td>
   <td>100.16%
   </td>
  </tr>
  <tr>
   <td>GSM8K
   </td>
   <td>97.19
   </td>
   <td>97.04
   </td>
   <td>99.85%
   </td>
  </tr>
  <tr>
   <td>MM_Star
   </td>
   <td>80.53
   </td>
   <td>80.07
   </td>
   <td>99.43%
   </td>
  </tr>
  <tr>
   <td>Math_Vista
   </td>
   <td>83.9
   </td>
   <td>84.2
   </td>
   <td>100.36%
   </td>
  </tr>
</table>

### Reproduction

The C-EVAL GSM8K MM_Star Math_Vista results were obtained using the `EvalScope` and `SGLang` framework.

Install T-head SGLang `(Version: 0.5.21)` and EvalScope `(Version: 1.13.0)` in container first.

#### Launching server
```
MODEL_PATH=<Path to DeepSeek-V4.1-Flash-1017-Quant-W-INT8-PerChannel-A-INT8-PerToken>
SGLANG_USE_MODELSCOPE=true sglang serve --model-path $MODEL_PATH \
  --tp-size 16 \
  --mem-fraction-static 0.9 \
  --page-size 64 \
  --quantization w8a8_int8 \
  --trust-remote-code \
  --chunked-prefill-size 19968 \
  --dist-timeout 60000 \
  --enable-prefill-cp \
  --cp-strategy interleave \
  --watchdog-timeout 60000 \
  --served-model-name DeepSeek-V4.1-Flash-1017 \
  --enable-cache-report \
  --reasoning-parser auto \
  --tool-call-parser auto \
  --log-level info \
  --enable-metrics \
  --decode-log-interval 1 \
  --cuda-graph-max-bs-decode 64 \
  --disable-custom-all-reduce \
  --disable-shared-experts-fusion \
  --host 0.0.0.0 \
  --port 8999 \
```

#### Evaluating model in a new terminal
```
evalscope eval \
  --model DeepSeek-V4.1-Flash-1017 \
  --api-url http://127.0.0.1:8999/v1 \
  --api-key EMPTY \
  --datasets gsm8k \
  --dataset-args '{"gsm8k": {"filters": {"remove_until": "</think>"}}}' \
  --eval-type openai_api \
  --eval-batch-size 64 \
  --generation-config '{"max_tokens": 256000, "temperature": 1, "top_p": 0.95, "n": 1, "extra_body": {"reasoning_effort": "high", "chat_template_kwargs": {"thinking": true}}}' \
  --timeout 60000 \
  --stream \
  --work-dir outputs
```

# License
Copyright (c) 2026, T-Head (Shanghai) Semiconductor Co., Ltd.
