---
title: gpt-oss-120b-DFlash
canonical_url: "https://www.modelscope.cn/models/z-lab/gpt-oss-120b-DFlash"
md_url: "https://www.modelscope.cn/models/z-lab/gpt-oss-120b-DFlash.md"
repository: z-lab/gpt-oss-120b-DFlash
last_updated: 2026-04-08
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3
architectures:
  - DFlashDraftModel
parameters: 784.8M
tensor_type:
  - BF16
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 82
stars: 0
tags:
  - dflash
  - speculative-decoding
  - diffusion
  - efficiency
  - flash-decoding
  - diffusion-language-model
  - gpt-oss
---

# gpt-oss-120b-DFlash

> gpt-oss-120b-DFlash - z-lab 在 ModelScope 开源的模型。gpt-oss-120b-DFlash Paper | GitHub | Blog

z-lab/gpt-oss-120b-DFlash 是 ModelScope 魔搭社区上的 784.8M 参数text-generation模型，采用 mit 许可。

- **Repository**: z-lab/gpt-oss-120b-DFlash
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 784.8M
- **Tags**: dflash, speculative-decoding, diffusion, efficiency, flash-decoding, diffusion-language-model, gpt-oss
- **Downloads**: 82
- **Stars**: 0
- **Last updated**: 2026-04-08

Source: https://www.modelscope.cn/models/z-lab/gpt-oss-120b-DFlash

---

# gpt-oss-120b-DFlash
[**Paper**](https://arxiv.org/abs/2602.06036) | [**GitHub**](https://github.com/z-lab/dflash) | [**Blog**](https://z-lab.ai/projects/dflash/)

**DFlash** is a novel speculative decoding method that utilizes a lightweight **block diffusion** model for drafting. It enables efficient, high-quality parallel drafting that pushes the limits of inference speed.

This model serves as the **drafter** component and contains **0.8B parameters**. It must be used in conjunction with the target model `openai/gpt-oss-120b`.

<div align="center">
  <img src="assets/dflash_system.png" alt="DFlash Architecture" width="100%">
</div>

## 📊 Training Data

**gpt-oss-120b-DFlash** is trained on **800K samples**, drawn from:
- [`nvidia/Nemotron-Post-Training-Dataset-v2`](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2)  
- [`theblackcat102/evol-codealpaca-v1`](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1)

For all samples, the response portion was regenerated using the target model `openai/gpt-oss-120b`.

## 🚀 Quick Start

### SGLang

#### Installation
```bash
uv pip install "git+https://github.com/sgl-project/sglang.git@refs/pull/20547/head#subdirectory=python"
```

#### Launch Server
```bash
# Optional: enable schedule overlapping (experimental, may not be stable)
# export SGLANG_ENABLE_SPEC_V2=1
# export SGLANG_ENABLE_DFLASH_SPEC_V2=1
# export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1

python -m sglang.launch_server \
    --model-path openai/gpt-oss-120b \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path z-lab/gpt-oss-120b-DFlash \
    --tp-size 1 \
    --dtype bfloat16 \
    --attention-backend fa3 \
    --mem-fraction-static 0.75 \
    --trust-remote-code
```

#### Usage

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Write a quicksort in Python."}],
    max_tokens=2048,
    temperature=0.0,
)
print(response.choices[0].message.content)
```

### vLLM

#### Installation

```bash
uv pip install vllm
uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly
```

#### Launch Server

```bash
vllm serve openai/gpt-oss-120b \
  --speculative-config '{"method": "dflash", "model": "z-lab/gpt-oss-120b-DFlash", "num_speculative_tokens": 9}' \
  --attention-backend flash_attn \
  --max-num-batched-tokens 32768
```

#### Usage

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Write a quicksort in Python."}],
    max_tokens=2048,
    temperature=0.0,
)
print(response.choices[0].message.content)
```

## Evaluation

The draft model is trained with a block size of 10. During evaluation, we use three settings:
- **Block size = 4** (3 draft tokens)
- **Block size = 6** (5 draft tokens)
- **Block size = 10** (9 draft tokens)

All experiments are conducted using **SGLang** on a single **H200 GPU**.

The reported speedups are **end-to-end speedups**, including prefill time. The **pure decoding speedup** is higher.

For all tasks, the reasoning effort is set to **medium**. Using **low reasoning effort** would further increase the acceptance length.

#### Acceptance Length

| Task       | Block Size = 4 | Block Size = 6 | Block Size = 10 |
|------------|----------------|----------------|------------------|
| GSM8K      | 3.3 | 4.3 | 5.3 |
| Math500    | 3.3 | 4.3 | 5.4 |
| HumanEval  | 3.1 | 3.8 | 4.4 |
| MBPP       | 3.1 | 3.9 | 4.6 |
| MT-Bench   | 2.7 | 3.3 | 3.7 |

#### Speedup

##### GSM8K

| Concurrency | Block Size = 4 | Block Size = 10 |
|------------|----------------|------------------|
| 1  | 1.3× | 1.8× |
| 8  | 1.2× | 1.6× |
| 16 | 1.3× | 1.6× |
| 32 | 1.2× | 1.5× |
| 64 | 1.2× | 1.5× |

##### Math500

| Concurrency | Block Size = 4 | Block Size = 10 |
|------------|----------------|------------------|
| 1  | 1.5× | 1.9× |
| 8  | 1.4× | 1.7× |
| 16 | 1.5× | 1.6× |
| 32 | 1.4× | 1.5× |
| 64 | 1.4× | 1.5× |

##### HumanEval

| Concurrency | Block Size = 4 | Block Size = 10 |
|------------|----------------|------------------|
| 1  | 1.3× | 1.7× |
| 8  | 1.4× | 1.7× |
| 16 | 1.4× | 1.8× |
| 32 | 1.5× | 1.7× |
| 64 | 1.4× | 1.5× |

##### MBPP

| Concurrency | Block Size = 4 | Block Size = 10 |
|------------|----------------|------------------|
| 1  | 1.4× | 1.8× |
| 8  | 1.5× | 1.7× |
| 16 | 1.5× | 1.8× |
| 32 | 1.6× | 1.8× |
| 64 | 1.6× | 1.6× |

##### MT-Bench

| Concurrency | Block Size = 4 | Block Size = 10 |
|------------|----------------|------------------|
| 1  | 1.3× | 1.3× |
| 8  | 1.2× | 1.3× |
| 16 | 1.3× | 1.3× |
| 32 | 1.4× | 1.3× |
| 64 | 1.3× | 1.2× |


## **Acknowledgement**
We are grateful to [Yotta Labs](https://www.yottalabs.ai/) for their compute support in training this draft model.

## **Citation**
If you find DFlash useful for your research or applications, please cite our project.

```bibtex
@misc{chen2026dflash,
  title         = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author        = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year          = {2026},
  eprint        = {2602.06036},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2602.06036}
}
```