---
title: Spark-X2.6-4B-DSpark-GGUF
canonical_url: "https://www.modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF"
md_url: "https://www.modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF.md"
repository: XHToken/Spark-X2.6-4B-DSpark-GGUF
last_updated: 2026-10-31
license: apache-2.0
pipeline_tag: text-generation
tasks:
- text-generation
base_model:
- XHToken/Spark-X2.6-4B
base_model_relation: quantized
library_name:
- gguf
language:
- en
- zh
downloads: 1
stars: 1
tags:
- gguf
- llama.cpp
- sparkx2_6
- dspark
- speculative-decoding
- draft-model
---
# Spark-X2.6-4B-DSpark-GGUF
> Spark-X2.6-4B-DSpark-GGUF - XHToken 在 ModelScope 开源的模型。Spark-X2.6-4B-DSpark-GGUF
XHToken/Spark-X2.6-4B-DSpark-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 XHToken/Spark-X2.6-4B 构建。
- **Repository**: XHToken/Spark-X2.6-4B-DSpark-GGUF
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: XHToken/Spark-X2.6-4B
- **Tags**: gguf, llama.cpp, sparkx2_6, dspark, speculative-decoding, draft-model
- **Downloads**: 1
- **Stars**: 1
- **Last updated**: 2026-10-31
Source: https://www.modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF
---
# Spark-X2.6-4B-DSpark-GGUF
**Spark-X2.6-4B-DSpark-GGUF** provides a BF16 GGUF draft model for speculative decoding with [Spark-X2.6-4B-GGUF](https://modelscope.cn/models/XHToken/Spark-X2.6-4B-GGUF). The target model can use BF16 or quantized GGUF weights.
DSpark accelerates generation through speculative decoding: the draft proposes up to seven tokens per drafting pass, and the target model verifies the proposed tokens in parallel, reducing the number of sequential target decoding steps.
## Model Details
| Property | Spark-X2.6-4B-DSpark |
| :--- | :--- |
| Target model | Spark-X2.6-4B |
| Draft parameters | 624M |
| Draft weight precision | BF16 |
| Draft layers | 5 SWA layers |
| Sliding-window size | 512 |
| Hidden size | 2560 |
| Draft tokens per forward pass | 7 |
| Target hidden-state layers | `[2, 10, 18, 25, 33]` |
## Performance
### Mean Acceptance Length
We evaluate acceptance with a draft length of 7 tokens and a maximum output length of 4096 tokens. Acceptance length includes one additional target token, giving a maximum of 8.
Table notation:
- `T`: target model temperature.
- `High` / `Low` / `No thinking`: target model thinking modes.
- `Prob.` / `Greedy`: probabilistic sampling / argmax draft decoding.
| Dataset | T=1, High, Prob. | T=1, Low, Prob. | T=1, No thinking, Prob. | T=0, High, Greedy |
| :--- | :--- | :--- | :--- | :--- |
| AIME25 | 4.1833 | 4.0749 | 4.8251 | 4.6493 |
| GSM8K | 4.4056 | 4.3266 | 4.9514 | 4.4845 |
| MATH500 | 4.5942 | 4.6721 | 5.1758 | 5.0335 |
| HumanEval | 3.8743 | 4.1370 | 4.5859 | 4.4330 |
| MBPP | 3.8671 | 4.0179 | 4.3935 | 4.5525 |
| Arena-Hard-v2 | 3.2172 | 3.1772 | 3.0499 | 4.1601 |
| **Average** | **4.0236** | **4.0676** | **4.4969** | **4.5522** |
### On-Device Inference
We compare inference throughput with and without DSpark on **NVIDIA RTX 4090 (llama.cpp)** and **Apple M5 Pro (MLX)**, using BF16 and quantized target models (Q4_K_M on RTX 4090 and INT4 on M5 Pro).
| Dataset | BF16 · RTX 4090 | BF16 · M5 Pro | Q4_K_M · RTX 4090 | INT4 · M5 Pro |
| :--- | :--- | :--- | :--- | :--- |
| AIME25 | **2.22×**
95 → 210 tok/s | **2.62×**
31 → 80 tok/s | **2.00×**
226 → 451 tok/s | **1.69×**
71 → 119 tok/s |
| GSM8K | **2.49×**
95 → 236 tok/s | **2.57×**
31 → 79 tok/s | **2.35×**
226 → 532 tok/s | **1.66×**
71 → 118 tok/s |
| MATH500 | **2.51×**
95 → 239 tok/s | **2.94×**
31 → 90 tok/s | **2.33×**
226 → 527 tok/s | **1.89×**
71 → 135 tok/s |
| HumanEval | **2.06×**
95 → 196 tok/s | **2.41×**
31 → 74 tok/s | **1.72×**
226 → 389 tok/s | **1.56×**
71 → 110 tok/s |
| MBPP | **1.88×**
95 → 178 tok/s | **2.49×**
31 → 77 tok/s | **1.85×**
227 → 420 tok/s | **1.61×**
71 → 115 tok/s |
| Arena-Hard-v2 | **1.70×**
94 → 160 tok/s | **2.00×**
31 → 61 tok/s | **1.62×**
225 → 365 tok/s | **1.30×**
71 → 92 tok/s |
| **Average** | **2.14×**
95 → 203 tok/s | **2.50×**
31 → 77 tok/s | **1.98×**
226 → 447 tok/s | **1.62×**
71 → 115 tok/s |
## Usage with llama.cpp
Use [XHToken/llama.cpp](https://github.com/XHToken/llama.cpp) to run Spark DSpark and send requests through an OpenAI-compatible API. The following commands are for Linux, Bash, and NVIDIA GPUs.
Branch: `master` (`22c0ae1871b60123c5afc008a78fe9c6fffa2858`). Integration into upstream llama.cpp is in progress.
### 1. Build
Clone the source and build:
```bash
git clone --branch master https://github.com/XHToken/llama.cpp.git
cd llama.cpp
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --target llama-server -j8
```
### 2. Start the Server
```bash
set -o pipefail
./build-cuda/bin/llama-server \
-m models/Spark-target.gguf \
-md models/Spark-draft.gguf \
--alias spark-dspark \
--spec-type draft-dspark \
--spec-draft-sampling greedy \
--spec-draft-n-max 7 \
--spec-draft-p-min 0 \
-ngl 999 -ngld 999 \
-ctk f16 -ctv f16 \
-ctkd f16 -ctvd f16 \
-fa on \
-c 8192 -np 1 \
-b 2048 -ub 512 -t 8 \
--jinja \
--host 127.0.0.1 --port 8080 \
2>&1 | tee dspark-server.log
```
### 3. Send an Example Request
```bash
curl -fsS http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "spark-dspark",
"messages": [
{
"role": "user",
"content": "你好"
}
],
"temperature": 1.0,
"top_p": 0.95,
"max_tokens": 512,
"stream": false
}'
```
## License
The Spark-X2.6 model series is licensed under the [Apache 2.0 License](LICENSE).
## Citation
If you find our work helpful, feel free to give us a cite.
```bibtex
@misc{sparkx2.6,
title = {Spark-X2.6-4B&1.7B: Advancing the Frontier of On-Device Agents for Real-World Tasks},
author = {SparkLLM Team},
year = {2026}
}
```