# Qwen3-8B No-Thinking Drafters: MTP and DFly
## Model Summary
This repository contains speculative-decoding drafter checkpoints for **Qwen3-8B**, including an **MTP drafter** and a **DFly drafter**. The drafters propose candidate tokens for the target model to verify, improving inference throughput while preserving the target model's output distribution.
The models were trained with **AngelSpec** and are intended to be deployed in **vLLM** together with the Qwen3-8B target model.
- **Target model:** Qwen3-8B
- **Inference mode:** No-thinking
- **Drafter types:** MTP and DFly
- **Training framework:** AngelSpec
- **Recommended serving backend:** vLLM speculative decoding
- **MTP speculative tokens:** 3
- **DFly speculative tokens:** 8
## Training Data
For Qwen3-8B, prompts are taken from **Open-PerfectBlend** (Xu et al., 2024). Responses are regenerated with the Qwen3-8B target model in **non-thinking mode**, and the resulting target-generated responses are used to train the drafter.
Regenerating responses with the same target model helps the drafter better match the target distribution, which is critical for speculative decoding acceptance rate and end-to-end acceleration.
## Training Hyperparameters
These Qwen3-8B draft models use the following training configuration:
| Hyperparameter | Value |
|---|---:|
| Batch size | 512 |
| Loss schedule | LK loss cold start for 300 steps, then end-to-end TV loss |
| Learning rate | 6.0e-4 |
| Minimum learning rate | 6.0e-5 |
| LR decay style | Cosine |
| Warmup ratio | 0.04 |
| Weight decay | 0.0 |
| Epochs | 10 |
| Maximum sequence length | 4096 |
## Evaluation
The table reports acceleration-related metrics across math, code, and chat benchmarks.
| Target Model | Drafter | Math500 | GSM8K | HumanEval | MBPP | LiveCodeBench | MT-Bench | Avg. |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| Qwen3-8B | MTP | 3.53 | 3.56 | 3.33 | 3.22 | 3.25 | 2.57 | 3.24 |
| Qwen3-8B | DFlash | 4.97 | 5.54 | 4.77 | 4.50 | 4.46 | 3.16 | 4.57 |
| Qwen3-8B | DSpark | 5.87 | 6.25 | 5.56 | 5.25 | 5.20 | 3.77 | 5.32 |
| Qwen3-8B | DFly | **6.06** | **6.42** | **5.60** | **5.34** | **5.36** | 3.67 | **5.41** |
DFly achieves the highest average result among the listed Qwen3-8B drafters in this evaluation, with an average score of 5.41.
## Deployment
Set `TARGET` to the Qwen3-8B target model and `DRAFT` to the corresponding drafter repository or local checkpoint path. The relevant inference code can be found in PR https://github.com/vllm-project/vllm/pull/50246.
### MTP-3
```bash
export TARGET="Qwen/Qwen3-8B"
export DRAFT=""
vllm serve "$TARGET" \
--tensor-parallel-size 1 \
--speculative-config "{"method":"mtp","model":"$DRAFT","num_speculative_tokens":3}"
```
### DFly-7
```bash
export TARGET="Qwen/Qwen3-8B"
export DRAFT=""
vllm serve "$TARGET" \
--tensor-parallel-size 1 \
--speculative-config "{"method":"dspark","model":"$DRAFT","num_speculative_tokens":7}"
```
## Technical Report
For more details, see the technical report:
> **AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding**
> arXiv: https://arxiv.org/abs/2607.25852
