---
title: DeepSeek-V4.1-Flash-1017-MLX
canonical_url: "https://www.modelscope.cn/models/rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX"
md_url: "https://www.modelscope.cn/models/rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX.md"
repository: rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX
last_updated: 2026-10-22
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - deepseek_v41
architectures:
  - DeepseekV41ForCausalLM
base_model:
  - deepseek-ai/DeepSeek-V4.1-Flash-1017
base_model_relation: finetune
parameters: 98.0B
tensor_type:
  - U32
  - F32
  - BF16
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 2850
stars: 1
tags:
  - mlx
  - deepseek
  - moe
  - mixed-precision
  - gptq
  - apple-silicon
  - rapid-mlx
---

# DeepSeek-V4.1-Flash-1017-MLX

> DeepSeek-V4.1-Flash-1017-MLX - rapid-mlx 在 ModelScope 开源的模型。DeepSeek-V4.1-Flash-1017 — Rapid-MLX GPTQ mixed-precision (256 GB Macs)

rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX 是 ModelScope 魔搭社区上的 98.0B 参数text-generation模型，采用 mit 许可，基于 deepseek-ai/DeepSeek-V4.1-Flash-1017 构建。

- **Repository**: rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 98.0B
- **Base model**: deepseek-ai/DeepSeek-V4.1-Flash-1017
- **Tags**: mlx, deepseek, moe, mixed-precision, gptq, apple-silicon, rapid-mlx
- **Downloads**: 2850
- **Stars**: 1
- **Last updated**: 2026-10-22

Source: https://www.modelscope.cn/models/rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX

---

# DeepSeek-V4.1-Flash-1017 — Rapid-MLX GPTQ mixed-precision (256 GB Macs)

A mixed-precision MLX build of
[deepseek-ai/DeepSeek-V4.1-Flash-1017](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017)
sized to run on a single **256 GB Apple Silicon Mac** with
[Rapid-MLX](https://github.com/raullenchai/Rapid-MLX).

This is **v2** of this repository. It replaces round-to-nearest rounding with
**GPTQ** (error-compensating quantization driven by calibration data). Against the
same source-precision reference, mean next-token KL divergence drops by 35–44%
and top-1 agreement rises by 3.5–5.9 points on every ruler we measured.
The previous build stays available as the `v1` tag.

![Closest to the original: top-1 agreement and KL vs the source model](chart.png)

*Higher top-1 agreement and lower KL mean closer to the original model. Details in [Comparison with a community 2-bit MLX quant](#comparison-with-a-community-2-bit-mlx-quant).*

- **No experts removed.** All 40 × 384 routed experts are kept.
- **GPTQ for every quantized weight matrix in the 40 layers**, calibrated on
  36 × 40,960 tokens, with routed-expert Hessians weighted by the router. The
  Engram lookup tables are the exception: they are the same 4-bit tables as v1.
- **~215.8 GB resident** (weights the model keeps in RAM), plus 122.9 GB of
  4-bit Engram tables that stay on SSD. The download is 338.8 GB in total.

## Quick start

> Requires Rapid-MLX 0.16.6 or newer; 0.16.7 or newer is recommended (up to ~1.9x faster decode). Older releases keep loading the `v1` revision.

```bash
pip install -U rapid-mlx
rapid-mlx serve deepseek-v41-flash-1017
```

This starts an OpenAI-compatible server on `http://127.0.0.1:8000/v1`. The
first run downloads the pinned revision and verifies its file hashes.

## Requirements and current limits

| Item | Value |
|---|---|
| Unified memory | 256 GB Mac (the server refuses to start below the guard) |
| Free disk | ~340 GB (Engram tables are read from SSD on demand) |
| Decoding | greedy only (`temperature` 0); `seed` is rejected |
| Input | text only, ≤ 8,192 prompt tokens |
| Output | ≤ 4,096 tokens per request |
| Tools / function calling | not yet supported by this serving lane |
| Speculative decoding | built-in DSpark draft (`dspark/`, 4-bit experts) |

Sampling, tool calling, and longer inputs and outputs are planned follow-ups for
the serving lane. They are not limits of the weights.

## What is in this build

| Component | Precision (group size 64 unless noted) |
|---|---|
| Routed experts, gate/up | 2-bit; 3-bit on layers 15 and 35–39 |
| Routed experts, down | 3-bit, all layers |
| Attention and shared experts | 4-bit; 8-bit on layers 6, 14–18, 26, 39 |
| Indexer, compressor, attention output A | 8-bit |
| Embeddings, LM head, router | BF16 |
| Engram (layers 1, 14) | 4-bit, group 32, round-to-nearest (same as v1), offloaded to SSD |
| DSpark draft head | 8-bit dense, 4-bit experts |

### How it was built

1. **Reference.** The source FP4/FP8 checkpoint is streamed one layer at a time.
   Every ruler below compares against the source-precision model, never against
   another quantized build.
2. **Calibration.** 36 slices × 40,960 tokens covering chat, code, math,
   tool-call transcripts, agentic terminal sessions, professional domains and
   many languages, including low-resource ones. Calibration shares no corpus
   file or token window with the held-out slices, and no 64-token window with
   the WikiText or long-context rulers.
3. **GPTQ, layer by layer.** Each layer is quantized on the outputs of the
   already-quantized layers before it, so later layers correct for earlier
   error. Routed experts are processed in batches, with each token's
   contribution to an expert's Hessian weighted by its router weight.
4. **Bit widths.** Sensitivity was scanned on all 40 layers. The eight most
   sensitive layers keep 8-bit attention and shared experts, and the late layers
   get 3-bit gate/up projections, within a 216 GB resident budget.
5. **Engram tables.** The SSD-resident Engram tables are taken unchanged from
   v1 (4-bit), which keeps every file under 50 GB. A 6-bit variant measured
   3.5–6.5% lower mean KL and 0.3–0.6 points higher top-1 agreement on the
   rulers below, at 49 GB more disk.

## Quality

Teacher-forced next-token KL divergence vs. the source-precision model; lower is
better. Top-1 agreement is the share of positions where the build's most likely
next token equals the source model's. Same evaluator and same reference for both
builds.

| Ruler | Tokens | Build | Mean KL (nats) | p99 KL | Top-1 agreement |
|---|---|---|---|---|---|
| 38 held-out slices × 512 | 19,456 | v1 (round-to-nearest) | 0.439 | 4.17 | 79.2% |
| | | **v2 (this build)** | **0.260** | **3.30** | **85.1%** |
| WikiText-2 test, 24 × 1,100 | 26,400 | v1 | 0.444 | 4.44 | 78.6% |
| | | **v2** | **0.288** | **3.26** | **83.2%** |
| Long context, 2 × 17,000 | 34,000 | v1 | 0.269 | 4.49 | 90.5% |
| | | **v2** | **0.150** | **3.29** | **94.0%** |

The held-out slices cover chat, code, math, tool calls, agentic terminal
sessions, law, medicine, finance, science and 20 languages; v2 improves on every
one of the 38 slices. The long-context ruler scores every position of two full
17,000-token contexts (an English and a Chinese public-domain novel). For the
WikiText and long-context rulers, KL is computed over the reference's top-128
tokens plus one tail bucket, which slightly underestimates full-vocabulary KL;
top-1 agreement is exact.

Per-domain KL on the held-out slices:

| Domain | v1 | v2 |
|---|---|---|
| code | 0.128 | 0.055 |
| math | 0.179 | 0.063 |
| agentic terminal | 0.156 | 0.097 |
| tool-call transcripts | 0.187 | 0.093 |
| general chat (EN) | 0.251 | 0.177 |
| general chat (ZH) | 0.246 | 0.141 |
| hardest slice (Chinese folk-customs text) | 0.749 | 0.546 |

Non-English and long-tail knowledge text still drifts more than code, math and
agentic text.

### Evaluation method

Teacher-forced top-1 agreement and KL divergence are measured against the source
FP4/FP8 weights dequantized to BF16, at source revision
`1bae0e1088993b9b82de08f59de5be5a34e42170`. Top-1 agreement measures agreement
with the reference, not task accuracy.

The rulers are held-out 38 × 512 tokens, WikiText-2 test 24 × 1,100 tokens, and
long-context 2 × 17,000 tokens (English and Chinese). The long-context ruler is
an offline weight evaluation. Held-out KL uses the full vocabulary; WikiText-2
and long-context KL use the reference's top-128 tokens plus one tail bucket,
a lower bound on full-vocabulary KL. Top-1 agreement is exact.

### Comparison with a community 2-bit MLX quant

| Ruler | Build | Top-1 % | Mean KL | p99 KL |
|---|---|---:|---:|---:|
| heldout38 | Rapid-MLX v2 (ours) | 85.1 | 0.259901 | 3.304570 |
| heldout38 | Community 2-bit MLX quant | 80.4 | 0.393205 | 3.953150 |
| heldout38 | Rapid-MLX v1 | 79.2 | 0.439326 | 4.173760 |
| wiki1100 | Rapid-MLX v2 (ours) | 83.2 | 0.288168 | 3.261922 |
| wiki1100 | Community 2-bit MLX quant | 79.3 | 0.402463 | 4.096575 |
| wiki1100 | Rapid-MLX v1 | 78.6 | 0.444330 | 4.438421 |
| long17k | Rapid-MLX v2 (ours) | 94.0 | 0.150136 | 3.292282 |
| long17k | Community 2-bit MLX quant | not available | not available | not available |
| long17k | Rapid-MLX v1 | 90.5 | 0.269110 | 4.486991 |

Resident weight sizes are approximately **216 GB for this build** and **190 GB
for the Community 2-bit MLX quant**, excluding SSD-resident Engram tables. These
are weight payloads, not peak process-memory measurements.


### Speed and memory (Mac Studio M3 Ultra, 256 GB)

Decode speed with Rapid-MLX 0.16.7 or newer, single stream, default settings
(built-in DSpark draft at depth 2), greedy, 256 output tokens, warm server,
~0.7–1k-token chat prompts:

| Prompt class | Decode (tok/s), default | With `RAPID_MLX_DSV41_DRAFT_DEPTH=4` |
|---|---|---|
| Long prose (1,008 tokens) | 35.2 | 31.2 |
| Question answering | 34.3 | 33.1 |
| Summarize | 36.2 | 33.6 |
| Code | 42.7 | 49.0 |
| Structured output (JSON) | 42.1 | 46.5 |

On the same machine and prompts, 0.16.6 decoded at 17.6–33.9 tok/s and 0.16.7
at 32.6–44.9 tok/s, up to about 1.9x faster
([same-machine A/B method and reproduce commands](https://github.com/raullenchai/Rapid-MLX/blob/main/docs/engineering/performance/2026-10-21-dsv41-0167-same-machine-ab.md)). For code-heavy workloads set
`RAPID_MLX_DSV41_DRAFT_DEPTH=4` (or `auto`). The target model still verifies
every emitted token, so greedy output quality is unchanged. Method and raw
tables: [measured defaults](https://github.com/raullenchai/Rapid-MLX/blob/main/docs/engineering/performance/2026-10-21-dsv41-draft-depth.md#measured-defaults).

**Memory.** The weights are loaded into RAM up front: ~215.8 GB resident, plus
working buffers. The measured peak process footprint is 213 GB (8k-token
prompt, otherwise idle Mac Studio). On a 256 GB Mac, quit other memory-heavy apps
before starting the server. If macOS runs out of memory and swap during loading,
it terminates the server.

## Files

GPTQ provenance ships with the weights: `gptq-provenance.json` (recipe,
calibration and source identities), `gptq-layer-provenance.json`,
`gptq-statistics/` (per-layer calibration and held-out error reports) and
`gptq-heldout-drift.jsonl` (cumulative hidden-state drift after each layer).
The Engram table swap is recorded under `post_assembly_overrides` in
`gptq-provenance.json`, with file hashes.

## Previous version

The round-to-nearest build is the `v1` tag of this repository
(`revision="v1"`), ~206.5 GB resident.

## License

MIT, same as the base model. See `LICENSE`.