---
title: rwkv-7-pile
canonical_url: "https://www.modelscope.cn/models/Blink_DL/rwkv-7-pile"
md_url: "https://www.modelscope.cn/models/Blink_DL/rwkv-7-pile.md"
repository: Blink_DL/rwkv-7-pile
last_updated: 2026-07-05
license: apache-2.0
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 1271
stars: 0
---

# rwkv-7-pile

> rwkv-7-pile - Blink_DL 在 ModelScope 开源的模型。RWKV-7 trained on the Pile w/ "20b tokenizer" (332115325534 tokens)

Blink_DL/rwkv-7-pile 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可。

- **Repository**: Blink_DL/rwkv-7-pile
- **License**: apache-2.0
- **Downloads**: 1271
- **Stars**: 0
- **Last updated**: 2026-07-05

Source: https://www.modelscope.cn/models/Blink_DL/rwkv-7-pile

---

RWKV-7 trained on the Pile w/ "20b tokenizer" (332115325534 tokens)

0.1B = L12-D768, lr 8e-4 to 3e-5 cosine decay, wd 0.1, bsz 8x30x4096

0.4B = L24-D1024, lr 6e-4 to 2e-5 cosine decay, wd 0.1, bsz 8x30x4096

1.5B = L24-D2048, lr 5e-4 to 1.5e-5 cosine decay, wd 0.1, bsz 8x45x4096

Check https://github.com/BlinkDL/RWKV-LM for details.

How to run it:

https://pypi.org/project/rwkv/

or

https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7