--- title: OpenSeek-Small-v1-Baseline canonical_url: "https://www.modelscope.cn/models/BAAI/OpenSeek-Small-v1-Baseline" md_url: "https://www.modelscope.cn/models/BAAI/OpenSeek-Small-v1-Baseline.md" repository: BAAI/OpenSeek-Small-v1-Baseline last_updated: 2025-09-08 license: open-mdw model_type: - deepseek_v3 architectures: - DeepseekV3ForCausalLM parameters: 1.6B tensor_type: - BF16 library_name: - transformer - safetensors - pytorch frameworks: - pytorch inference_backends: - "deploy_task text/emb" - "sglang 0.5.2" - "vllm 0.9.2" downloads: 419 stars: 0 --- # OpenSeek-Small-v1-Baseline > OpenSeek-Small-v1-Baseline - BAAI 在 ModelScope 开源的模型。OpenSeek-Small-v1-Baseline Model Documentation BAAI/OpenSeek-Small-v1-Baseline 是 ModelScope 魔搭社区上的 1.6B 参数机器学习模型，采用 open-mdw 许可，可用 deploy_task text/emb、sglang 0.5.2、vllm 0.9.2 部署。 - **Repository**: BAAI/OpenSeek-Small-v1-Baseline - **License**: open-mdw - **Parameters**: 1.6B - **Inference backends**: deploy_task text/emb, sglang 0.5.2, vllm 0.9.2 - **Downloads**: 419 - **Stars**: 0 - **Last updated**: 2025-09-08 Source: https://www.modelscope.cn/models/BAAI/OpenSeek-Small-v1-Baseline --- # OpenSeek-Small-v1-Baseline Model Documentation ## Overview We sampled 100 billion tokens from the CCI4.0 dataset and trained a 1.4B-parameter MoE model with 0.4B active parameters. This model, along with the dataset, is open-sourced as a baseline for future experiments in areas such as dataset construction, algorithmic strategies, and parallel training frameworks. The model arch is same as OpenSeek-Small-v1 model. ## Training Data The ratio for each domain is as follows: | Name | Ratio | |-------------------------------------------|---------| | Nemotron-CC-high-actual-actual-high | 1.1068 | | Nemotron-CC-high-actual-actual-low | 0.3577 | | Nemotron-CC-high-actual-actual-mid | 0.7775 | | Nemotron-CC-high-synthetic-distill-high | 0.2859 | | Nemotron-CC-high-synthetic-distill-low | 0.1672 | | Nemotron-CC-high-synthetic-distill-mid | 0.2339 | | Nemotron-CC-high-synthetic-diverse_qa_pairs-high | 0.5397 | | Nemotron-CC-high-synthetic-diverse_qa_pairs-low | 0.4064 | | Nemotron-CC-high-synthetic-diverse_qa_pairs-mid | 0.5005 | | Nemotron-CC-high-synthetic-extract_knowledge-high | 0.4616 | | Nemotron-CC-high-synthetic-extract_knowledge-low | 0.0670 | | Nemotron-CC-high-synthetic-extract_knowledge-mid | 0.3429 | | Nemotron-CC-high-synthetic-knowledge_list-high | 0.2610 | | Nemotron-CC-high-synthetic-knowledge_list-low | 0.1824 | | Nemotron-CC-high-synthetic-knowledge_list-mid | 0.2313 | | Nemotron-CC-high-synthetic-wrap_medium-high | 0.8237 | | Nemotron-CC-high-synthetic-wrap_medium-low | 0.2866 | | Nemotron-CC-high-synthetic-wrap_medium-mid | 0.6670 | | Nemotron-CC-low-synthetic-wrap_medium-high | 0.4657 | | Nemotron-CC-low-synthetic-wrap_medium-low | 0.2005 | | Nemotron-CC-low-synthetic-wrap_medium-mid | 0.4317 | | Nemotron-CC-medium-actual-actual-high | 1.1397 | | Nemotron-CC-medium-actual-actual-low | 0.6782 | | Nemotron-CC-medium-actual-actual-mid | 0.9175 | | arxiv | 0.6414 | | books | 0.4696 | | code-high | 1.0102 | | code-low | 1.1403 | | code-mid | 0.9674 | | cot_synthesis2_CC-high | 0.3755 | | cot_synthesis2_CC-low | 0.0499 | | cot_synthesis2_CC-mid | 1.8299 | | cot_synthesis2_OpenSource-high | 0.2573 | | cot_synthesis2_OpenSource-low | 0.1638 | | cot_synthesis2_OpenSource-mid | 0.3251 | | cot_synthesis2_arxiv-high | 6.0237 | | cot_synthesis2_arxiv-low | 8.9063 | | cot_synthesis2_arxiv-mid | 10.1376 | | cot_synthesis2_code-high | 0.4598 | | cot_synthesis2_code-low | 0.6857 | | cot_synthesis2_code-mid | 0.8990 | | cot_synthesis2_math-high | 1.3135 | | cot_synthesis2_math-low | 1.6530 | | cot_synthesis2_math-mid | 0.3536 | | cot_synthesis2_wiki-high | 0.6314 | | cot_synthesis2_wiki-low | 0.5978 | | cot_synthesis2_wiki-mid | 0.7909 | | cot_synthesis_CC-high | 0.2225 | | cot_synthesis_CC-low | 0.1797 | | cot_synthesis_CC-mid | 0.2042 | | cot_synthesis_OpenSource-high | 0.4081 | | cot_synthesis_OpenSource-low | 0.1659 | | cot_synthesis_OpenSource-mid | 1.2828 | | cot_synthesis_arxiv-high | 5.68 | | cot_synthesis_arxiv-low | 7.4907 | | cot_synthesis_arxiv-mid | 8.9359 | | cot_synthesis_code-high | 0.7663 | | cot_synthesis_code-low | 0.4052 | | cot_synthesis_code-mid | 0.1916 | | cot_synthesis_math-high | 0.5074 | | cot_synthesis_math-low | 0.6437 | | cot_synthesis_math-mid | 0.6406 | | cot_synthesis_wiki-high | 0.4000 | | cot_synthesis_wiki-low | 0.3564 | | cot_synthesis_wiki-mid | 0.5768 | | math-high | 1.8165 | | math-low | 1.6940 | | math-mid | 1.6311 | | pes2o | 6.1982 | | pes2o-full-train | 1.4257 | | pes2o-full-val | 0.0143 | | stack | 0.4229 | | wiki | 0.4202 | | zh_cc-high-loss0 | 1.8171 | | zh_cc-high-loss1 | 0.9776 | | zh_cc-high-loss2 | 0.3725 | | zh_cc-medium-loss0 | 0.9492 | | zh_cc-medium-loss1 | 0.9236 | | zh_cc-medium-loss2 | 1.0643 | ## Wandb Our training curves have been recorded in Weights & Biases [wandb](https://wandb.ai/openseek-team/OpenSeek-Small-v1-Baseline). ## Evalation We used the LightEval library for model evaluation, following the same setup as in FineWeb and CCI3-HQ. All evaluations were conducted in a zero-shot setting. To directly compare the performance across different datasets, we use Average, which refers to the overall average score across all Chinese and English benchmarks. | Metrics | Score | |-------------------|---------| | HellaSwag | 42.09 | | ARC (Average) | 40.11 | | PIQA | 67.14 | | MMLU (cloze) | 31.29 | | CommonsenseQA | 28.17 | | TriviaQA | 6.51 | | Winograde | 51.38 | | OpenBookQA | 33.00 | | GSM8K (5-shot) | 6.67 | | SIQA | 41.86 | | CEval | 30.19 | | CMMLU | 30.25 | | **Average-English** | **34.82** | | **Average-Chinese** | **30.22** | | **Overall Average** | **32.52** | ## License [OpenMDW 1.0](https://github.com/OpenMDW/OpenMDW/blob/main/1.0/LICENSE.openmdw) ## Usage Instructions ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("BAAI/OpenSeek-Small-v1-Baseline") tokenizer = AutoTokenizer.from_pretrained("BAAI/OpenSeek-Small-v1-Baseline") inputs = tokenizer("The future of AI is", return_tensors="pt") outputs = model.generate(**inputs, max_length=50) print(tokenizer.decode(outputs[0])) ```