---
title: qwen3-from-scratch-grpo-checkpoints
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/qwen3-from-scratch-grpo-checkpoints"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/qwen3-from-scratch-grpo-checkpoints.md"
repository: AI-ModelScope/qwen3-from-scratch-grpo-checkpoints
last_updated: 2026-07-18
license: apache-2.0
library_name:
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 53
stars: 0
tags:
  - pytorch
  - qwen3
  - grpo
  - rlvr
  - math
---

# qwen3-from-scratch-grpo-checkpoints

> qwen3-from-scratch-grpo-checkpoints - AI-ModelScope 在 ModelScope 开源的模型。Qwen3 From Scratch GRPO Checkpoints

AI-ModelScope/qwen3-from-scratch-grpo-checkpoints 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/qwen3-from-scratch-grpo-checkpoints
- **License**: apache-2.0
- **Tags**: pytorch, qwen3, grpo, rlvr, math
- **Downloads**: 53
- **Stars**: 0
- **Last updated**: 2026-07-18

Source: https://www.modelscope.cn/models/AI-ModelScope/qwen3-from-scratch-grpo-checkpoints

---

# Qwen3 From Scratch GRPO Checkpoints

This repository contains GRPO training checkpoints for the [rasbt/qwen3-from-scratch](https://huggingface.co/rasbt/qwen3-from-scratch) model from chapters 6 and 7 of [Build a Reasoning Model (From Scratch)](https://github.com/rasbt/reasoning-from-scratch).

These files are raw PyTorch `state_dict` checkpoints intended for use with the `reasoning_from_scratch` package.

<img src="https://sebastianraschka.com/images/reasoning-from-scratch-images/ch06/other/grpo-from-scratch.webp" width="700">

&nbsp;
## Available Checkpoints

- [grpo_original_no_kl](https://huggingface.co/rasbt/qwen3-from-scratch-grpo-checkpoints/tree/main/grpo_original_no_kl): the original GRPO algorithm without a KL term from [chapter 6](https://github.com/rasbt/reasoning-from-scratch/tree/main/ch06/02_rlvr_grpo_scripts_intro)
- [7_3_plus_tracking](https://huggingface.co/rasbt/qwen3-from-scratch-grpo-checkpoints/tree/main/7_3_plus_tracking): chapter 7 tracking variant from [`7_3_plus_tracking.py`](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/03_rlvr_grpo_scripts_advanced/7_3_plus_tracking.py)
- [7_4_plus_clip_ratio](https://huggingface.co/rasbt/qwen3-from-scratch-grpo-checkpoints/tree/main/7_4_plus_clip_ratio): clipped policy-ratio variant from [`7_4_plus_clip_ratio.py`](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/03_rlvr_grpo_scripts_advanced/7_4_plus_clip_ratio.py)
- [7_5_plus_kl](https://huggingface.co/rasbt/qwen3-from-scratch-grpo-checkpoints/tree/main/7_5_plus_kl): KL-regularized variant from [`7_5_plus_kl.py`](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/03_rlvr_grpo_scripts_advanced/7_5_plus_kl.py)
- [7_6_plus_format_reward](https://huggingface.co/rasbt/qwen3-from-scratch-grpo-checkpoints/tree/main/7_6_plus_format_reward): format-reward variant from [`7_6_plus_format_reward.py`](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/03_rlvr_grpo_scripts_advanced/7_6_plus_format_reward.py)

&nbsp;
## Usage Example

The checkpoints in this repository are intended for use with the `reasoning_from_scratch` package.

For the chapter 6 no-KL checkpoint, you can download it via:

```python
from reasoning_from_scratch.qwen3 import download_qwen3_grpo_checkpoints

download_qwen3_grpo_checkpoints(
    grpo_type="no_kl",
    step="00050",
    out_dir="qwen3",
)
```

For a chapter 7 checkpoint, the intended usage is:

```python
from reasoning_from_scratch.qwen3 import download_qwen3_grpo_checkpoints

download_qwen3_grpo_checkpoints(
    grpo_type="clip_ratio",
    step="00050",
    out_dir="qwen3",
)
```

Once downloaded, you can load a checkpoint and stream text as follows:

```python
from pathlib import Path
import torch

from reasoning_from_scratch.ch02 import (
    get_device,
    generate_text_basic_stream_cache,
)
from reasoning_from_scratch.qwen3 import (
    download_qwen3_small,
    Qwen3Model,
    Qwen3Tokenizer,
    QWEN_CONFIG_06_B,
)

device = get_device()
local_dir = Path("qwen3")

download_qwen3_small(kind="base", tokenizer_only=True, out_dir=local_dir)

tokenizer = Qwen3Tokenizer(tokenizer_file_path=local_dir / "tokenizer-base.json")
model = Qwen3Model(QWEN_CONFIG_06_B)
state_dict = torch.load(
    local_dir / "qwen3-0.6B-rlvr-grpo-step00050.pth",
    map_location=device,
)
model.load_state_dict(state_dict)
model.to(device)
model.eval()

prompt = "Solve: If x + 7 = 19, what is x?"
input_ids = torch.tensor(tokenizer.encode(prompt), device=device).unsqueeze(0)

for token in generate_text_basic_stream_cache(
    model=model,
    token_ids=input_ids,
    max_new_tokens=256,
    eos_token_id=tokenizer.eos_token_id,
):
    token_id = token.squeeze(0).item()
    print(tokenizer.decode([token_id]), end="", flush=True)
print()
```

&nbsp;
## Notes

- The chapter 6 checkpoints are based on the original no-KL GRPO setup described in [chapter 6](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch06/01_main-chapter-code/ch06_main.ipynb).
- The chapter 7 checkpoints correspond to the improved GRPO variants described in [chapter 7](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/01_main-chapter-code/ch07_main.ipynb) and the supplementary scripts in [`ch07/03_rlvr_grpo_scripts_advanced`](https://github.com/rasbt/reasoning-from-scratch/tree/main/ch07/03_rlvr_grpo_scripts_advanced).
- The `7_6_plus_format_reward` checkpoints are based on the reasoning model and are meant to be used with the reasoning tokenizer.

&nbsp;
## Download Helper Reference

These are the supported `grpo_type` values for `download_qwen3_grpo_checkpoints(...)`:

- Section 6.2 no-KL:
```python
download_qwen3_grpo_checkpoints(
    grpo_type="no_kl",
    step="00050",
    out_dir="qwen3",
)
```

Available chapter 6 saved steps: `00050`, `00100`, `00500`, `01000`, `01500`, `03000`, `05000`, `09000`.

- Section 7.3 tracking:
```python
download_qwen3_grpo_checkpoints(
    grpo_type="tracking",
    step="00050",
    out_dir="qwen3",
)
```

- Section 7.4 clip-ratio:
```python
download_qwen3_grpo_checkpoints(
    grpo_type="clip_ratio",
    step="00050",
    out_dir="qwen3",
)
```

- Section 7.5 KL:
```python
download_qwen3_grpo_checkpoints(
    grpo_type="kl",
    step="00050",
    out_dir="qwen3",
)
```

- Section 7.6 format-reward:
```python
download_qwen3_grpo_checkpoints(
    grpo_type="format_reward",
    step="00050",
    out_dir="qwen3",
)
```

For the chapter 7 variants, replace `step="00050"` with any other available saved step such as `00100`, `00150`, `00200`, `00250`, `00300`, `00350`, `00400`, `00450`, or `00500`.