---
title: Spark-X2.6-1.7B-DSpark-GGUF
canonical_url: "https://www.modelscope.cn/models/XHToken/Spark-X2.6-1.7B-DSpark-GGUF"
md_url: "https://www.modelscope.cn/models/XHToken/Spark-X2.6-1.7B-DSpark-GGUF.md"
repository: XHToken/Spark-X2.6-1.7B-DSpark-GGUF
last_updated: 2026-11-01
license: apache-2.0
pipeline_tag: text-generation
tasks:
- text-generation
base_model:
- XHToken/Spark-X2.6-1.7B
base_model_relation: quantized
library_name:
- gguf
language:
- en
- zh
downloads: 2
stars: 1
tags:
- gguf
- llama.cpp
- sparkx2_6
- dspark
- speculative-decoding
- draft-model
---
# Spark-X2.6-1.7B-DSpark-GGUF
> Spark-X2.6-1.7B-DSpark-GGUF - XHToken 在 ModelScope 开源的模型。Spark-X2.6-1.7B-DSpark-GGUF
XHToken/Spark-X2.6-1.7B-DSpark-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 XHToken/Spark-X2.6-1.7B 构建。
- **Repository**: XHToken/Spark-X2.6-1.7B-DSpark-GGUF
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: XHToken/Spark-X2.6-1.7B
- **Tags**: gguf, llama.cpp, sparkx2_6, dspark, speculative-decoding, draft-model
- **Downloads**: 2
- **Stars**: 1
- **Last updated**: 2026-11-01
Source: https://www.modelscope.cn/models/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
---
# Spark-X2.6-1.7B-DSpark-GGUF
**Spark-X2.6-1.7B-DSpark-GGUF** provides a BF16 GGUF draft model for speculative decoding with [Spark-X2.6-1.7B-GGUF](https://modelscope.cn/models/XHToken/Spark-X2.6-1.7B-GGUF). The target model can use BF16 or quantized GGUF weights.
DSpark accelerates generation through speculative decoding: the draft proposes up to seven tokens per drafting pass, and the target model verifies the proposed tokens in parallel, reducing the number of sequential target decoding steps.
## Model Details
| Property | Spark-X2.6-1.7B-DSpark |
| :--- | :--- |
| Target model | Spark-X2.6-1.7B |
| Draft parameters | 345M |
| Draft weight precision | BF16 |
| Draft layers | 5 SWA layers |
| Sliding-window size | 512 |
| Hidden size | 2048 |
| Draft tokens per forward pass | 7 |
| Target hidden-state layers | `[2, 8, 14, 19, 25]` |
## Performance
### Mean Acceptance Length
We evaluate acceptance with a draft length of 7 tokens and a maximum output length of 4096 tokens. Acceptance length includes one additional target token, giving a maximum of 8.
Table notation:
- `T`: target model temperature.
- `High` / `Low` / `No thinking`: target model thinking modes.
- `Prob.` / `Greedy`: probabilistic sampling / argmax draft decoding.
| Dataset | T=1, High, Prob. | T=1, Low, Prob. | T=1, No thinking, Prob. | T=0, High, Greedy |
| :--- | :--- | :--- | :--- | :--- |
| AIME25 | 3.5443 | 3.5919 | 4.4806 | 4.1771 |
| GSM8K | 3.7875 | 4.0283 | 4.4305 | 4.4367 |
| MATH500 | 3.9027 | 3.9425 | 4.7694 | 4.5103 |
| HumanEval | 3.5324 | 3.5795 | 4.0170 | 4.4917 |
| MBPP | 3.6004 | 3.7508 | 3.8723 | 4.3375 |
| Arena-Hard-v2 | 3.0905 | 3.0219 | 2.9768 | 3.9412 |
| **Average** | **3.5763** | **3.6525** | **4.0911** | **4.3158** |
### On-Device Inference
We compare inference throughput with and without DSpark on **NVIDIA RTX 4090 (llama.cpp)** and **Apple M5 Pro (MLX)**, using BF16 and quantized target models (Q4_K_M on RTX 4090 and INT4 on M5 Pro).
| Dataset | BF16 · RTX 4090 | BF16 · M5 Pro | Q4_K_M · RTX 4090 | INT4 · M5 Pro |
| :--- | :--- | :--- | :--- | :--- |
| AIME25 | **2.03×**
220 → 447 tok/s | **1.80×**
68 → 123 tok/s | **1.44×**
424 → 612 tok/s | **1.15×**
140 → 161 tok/s |
| GSM8K | **2.43×**
220 → 536 tok/s | **1.97×**
68 → 134 tok/s | **1.81×**
424 → 769 tok/s | **1.25×**
137 → 172 tok/s |
| MATH500 | **2.40×**
220 → 530 tok/s | **2.04×**
69 → 140 tok/s | **1.71×**
424 → 724 tok/s | **1.30×**
135 → 176 tok/s |
| HumanEval | **1.96×**
220 → 430 tok/s | **1.85×**
68 → 126 tok/s | **1.49×**
422 → 628 tok/s | **1.18×**
139 → 164 tok/s |
| MBPP | **1.96×**
221 → 434 tok/s | **1.93×**
68 → 132 tok/s | **1.43×**
426 → 609 tok/s | **1.23×**
140 → 172 tok/s |
| Arena-Hard-v2 | **1.83×**
219 → 401 tok/s | **1.66×**
68 → 114 tok/s | **1.33×**
422 → 560 tok/s | **1.06×**
137 → 146 tok/s |
| **Average** | **2.10×**
220 → 463 tok/s | **1.88×**
68 → 128 tok/s | **1.53×**
424 → 650 tok/s | **1.20×**
138 → 165 tok/s |
## Usage with llama.cpp
Use [XHToken/llama.cpp](https://github.com/XHToken/llama.cpp) to run Spark DSpark and send requests through an OpenAI-compatible API. The following commands are for Linux, Bash, and NVIDIA GPUs.
Branch: `master` (`22c0ae1871b60123c5afc008a78fe9c6fffa2858`). Integration into upstream llama.cpp is in progress.
### 1. Build
Clone the source and build:
```bash
git clone --branch master https://github.com/XHToken/llama.cpp.git
cd llama.cpp
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --target llama-server -j8
```
### 2. Start the Server
```bash
set -o pipefail
./build-cuda/bin/llama-server \
-m models/Spark-target.gguf \
-md models/Spark-draft.gguf \
--alias spark-dspark \
--spec-type draft-dspark \
--spec-draft-sampling greedy \
--spec-draft-n-max 7 \
--spec-draft-p-min 0 \
-ngl 999 -ngld 999 \
-ctk f16 -ctv f16 \
-ctkd f16 -ctvd f16 \
-fa on \
-c 8192 -np 1 \
-b 2048 -ub 512 -t 8 \
--jinja \
--host 127.0.0.1 --port 8080 \
2>&1 | tee dspark-server.log
```
### 3. Send an Example Request
```bash
curl -fsS http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "spark-dspark",
"messages": [
{
"role": "user",
"content": "你好"
}
],
"temperature": 1.0,
"top_p": 0.95,
"max_tokens": 512,
"stream": false
}'
```
## License
The Spark-X2.6 model series is licensed under the [Apache 2.0 License](LICENSE).
## Citation
If you find our work helpful, feel free to give us a cite.
```bibtex
@misc{sparkx2.6,
title = {Spark-X2.6-4B&1.7B: Advancing the Frontier of On-Device Agents for Real-World Tasks},
author = {SparkLLM Team},
year = {2026}
}
```