---
title: OpenSeek-Pretrain-100B
canonical_url: "https://www.modelscope.cn/datasets/BAAI/OpenSeek-Pretrain-100B"
md_url: "https://www.modelscope.cn/datasets/BAAI/OpenSeek-Pretrain-100B.md"
repository: BAAI/OpenSeek-Pretrain-100B
last_updated: 2025-06-05
license: "Apache License 2.0"
storage_size: "385 GB"
downloads: 3020
stars: 4
---

# OpenSeek-Pretrain-100B

> OpenSeek-Pretrain-100B - BAAI 在 ModelScope 开源的数据集。OpenSeek Pretraining Dataset v1.0

BAAI/OpenSeek-Pretrain-100B 是 ModelScope 魔搭社区上的数据集，存储大小 385 GB，采用 Apache License 2.0 许可。

- **Repository**: BAAI/OpenSeek-Pretrain-100B
- **License**: Apache License 2.0
- **Storage size**: 385 GB
- **Downloads**: 3020
- **Stars**: 4
- **Last updated**: 2025-06-05

Source: https://www.modelscope.cn/datasets/BAAI/OpenSeek-Pretrain-100B

---

# OpenSeek Pretraining Dataset v1.0

We have released a portion of the sampled 100B tokens data from the **CCI4.0-M2 v1**, including **Chinese and English Web datasets**, domain-specific datasets and Chain-of-Thought reasoning datasets.

The dataset directory is consistent with the overall dataset. For a detailed description of the dataset, please refer to [CCI4.0-M2 v1 README](https://huggingface.co/datasets/BAAI/CCI4.0-M2-Base-v1/blob/main/README.md).

# Data Composition
The Tokens for each subdirectory are as follows:
| Name                                              | Tokens       | Tokens(B) |
|---------------------------------------------------|--------------|----------|
| Nemotron-CC-high-actual-actual-high               | 1140543860   | 1.14     |
| Nemotron-CC-high-actual-actual-low                | 368646238    | 0.37     |
| Nemotron-CC-high-actual-actual-mid                | 801213010    | 0.80     |
| Nemotron-CC-high-synthetic-distill-high           | 294569308    | 0.29     |
| Nemotron-CC-high-synthetic-distill-low            | 172342068    | 0.17     |
| Nemotron-CC-high-synthetic-distill-mid            | 240998642    | 0.24     |
| Nemotron-CC-high-synthetic-diverse_qa_pairs-high  | 556137649    | 0.56     |
| Nemotron-CC-high-synthetic-diverse_qa_pairs-low   | 418742390    | 0.42     |
| Nemotron-CC-high-synthetic-diverse_qa_pairs-mid   | 515733187    | 0.52     |
| Nemotron-CC-high-synthetic-extract_knowledge-high | 475714119    | 0.48     |
| Nemotron-CC-high-synthetic-extract_knowledge-low  | 68996838     | 0.07     |
| Nemotron-CC-high-synthetic-extract_knowledge-mid  | 353316407    | 0.35     |
| Nemotron-CC-high-synthetic-knowledge_list-high    | 268953064    | 0.27     |
| Nemotron-CC-high-synthetic-knowledge_list-low     | 187973360    | 0.19     |
| Nemotron-CC-high-synthetic-knowledge_list-mid     | 238373108    | 0.24     |
| Nemotron-CC-high-synthetic-wrap_medium-high       | 848837296    | 0.85     |
| Nemotron-CC-high-synthetic-wrap_medium-low        | 295324405    | 0.30     |
| Nemotron-CC-high-synthetic-wrap_medium-mid        | 687328353    | 0.69     |
| Nemotron-CC-low-synthetic-wrap_medium-high        | 479896420    | 0.48     |
| Nemotron-CC-low-synthetic-wrap_medium-low         | 206574167    | 0.21     |
| Nemotron-CC-low-synthetic-wrap_medium-mid         | 444865784    | 0.44     |
| Nemotron-CC-medium-actual-actual-high            | 1174405205   | 1.17     |
| Nemotron-CC-medium-actual-actual-low              | 698884310    | 0.70     |
| Nemotron-CC-medium-actual-actual-mid              | 945401567    | 0.95     |
| arxiv                                             | 660912931    | 0.66     |
| books                                             | 483917796    | 0.48     |
| code-high                                         | 1040945650   | 1.04     |
| code-low                                          | 1175000655   | 1.18     |
| code-mid                                          | 996826302    | 1.00     |
| cot_synthesis2_CC-high                            | 386941302    | 0.39     |
| cot_synthesis2_CC-low                             | 51390680     | 0.05     |
| cot_synthesis2_CC-mid                             | 1885475230   | 1.89     |
| cot_synthesis2_OpenSource-high                    | 265167656    | 0.27     |
| cot_synthesis2_OpenSource-low                     | 168830028    | 0.17     |
| cot_synthesis2_OpenSource-mid                     | 334976884    | 0.33     |
| cot_synthesis2_arxiv-high                         | 12894983685  | 12.89    |
| cot_synthesis2_arxiv-low                          | 9177670132   | 9.18     |
| cot_synthesis2_arxiv-mid                          | 10446468216  | 10.45    |
| cot_synthesis2_code-high                          | 473767419    | 0.47     |
| cot_synthesis2_code-low                           | 706636812    | 0.71     |
| cot_synthesis2_code-mid                           | 926436168    | 0.93     |
| cot_synthesis2_math-high                          | 1353517224   | 1.35     |
| cot_synthesis2_math-low                           | 1703361358   | 1.70     |
| cot_synthesis2_math-mid                           | 364330324    | 0.36     |
| cot_synthesis2_wiki-high                          | 650684154    | 0.65     |
| cot_synthesis2_wiki-low                           | 615978070    | 0.62     |
| cot_synthesis2_wiki-mid                           | 814947142    | 0.81     |
| cot_synthesis_CC-high                             | 229324269    | 0.23     |
| cot_synthesis_CC-low                              | 185148748    | 0.19     |
| cot_synthesis_CC-mid                              | 210471356    | 0.21     |
| cot_synthesis_OpenSource-high                     | 420505110    | 0.42     |
| cot_synthesis_OpenSource-low                      | 170987708    | 0.17     |
| cot_synthesis_OpenSource-mid                      | 1321855051   | 1.32     |
| cot_synthesis_arxiv-high                          | 5853027309   | 5.85     |
| cot_synthesis_arxiv-low                           | 7718911399   | 7.72     |
| cot_synthesis_arxiv-mid                           | 9208148090   | 9.21     |
| cot_synthesis_code-high                           | 789672525    | 0.79     |
| cot_synthesis_code-low                            | 417526994    | 0.42     |
| cot_synthesis_code-mid                            | 197436971    | 0.20     |
| cot_synthesis_math-high                           | 522900778    | 0.52     |
| cot_synthesis_math-low                            | 663320643    | 0.66     |
| cot_synthesis_math-mid                            | 660137084    | 0.66     |
| cot_synthesis_wiki-high                           | 412152225    | 0.41     |
| cot_synthesis_wiki-low                            | 367306600    | 0.37     |
| cot_synthesis_wiki-mid                            | 594421619    | 0.59     |
| math-high                                         | 1871864190   | 1.87     |
| math-low                                          | 1745580082   | 1.75     |
| math-mid                                          | 1680811027   | 1.68     |
| pes2o                                             | 6386997158   | 6.39     |
| pes2o-full-train                                  | 1469110938   | 1.47     |
| pes2o-full-val                                    | 14693152     | 0.01     |
| stack                                             | 435813429    | 0.44     |
| wiki                                              | 433002447    | 0.43     |
| zh_cc-high-loss0                                  | 1872431176   | 1.87     |
| zh_cc-high-loss1                                  | 1007405788   | 1.01     |
| zh_cc-high-loss2                                  | 383830893    | 0.38     |
| zh_cc-medidum-loss0                               | 978118384    | 0.98     |
| zh_cc-medidum-loss1                               | 951741139    | 0.95     |
| zh_cc-medidum-loss2                               | 1096769115   | 1.10     |
