---
title: plamo-3-nict-31b-base
canonical_url: "https://www.modelscope.cn/models/pfnet/plamo-3-nict-31b-base"
md_url: "https://www.modelscope.cn/models/pfnet/plamo-3-nict-31b-base.md"
repository: pfnet/plamo-3-nict-31b-base
last_updated: 2026-08-26
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - plamo3
architectures:
  - Plamo3ForCausalLM
parameters: 31.8B
tensor_type:
  - BF16
library_name:
  - transformer
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - ja
downloads: 80
stars: 0
---

# plamo-3-nict-31b-base

> plamo-3-nict-31b-base - pfnet 在 ModelScope 开源的模型。PLaMo 3 NICT 31B Base

pfnet/plamo-3-nict-31b-base 是 ModelScope 魔搭社区上的 31.8B 参数text-generation模型，采用 other 许可。

- **Repository**: pfnet/plamo-3-nict-31b-base
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 31.8B
- **Downloads**: 80
- **Stars**: 0
- **Last updated**: 2026-08-26

Source: https://www.modelscope.cn/models/pfnet/plamo-3-nict-31b-base

---

# PLaMo 3 NICT 31B Base


## Model Description

PLaMo 3 NICT 31B Base is a 31B model pre-trained on English and Japanese datasets, developed by Preferred Networks, Inc. collaborative with National Institute of Information and Communications Technology, NICT.

PLaMo 3 NICT models adapt a hybrid architecture with Sliding Window Attention (SWA) and Traditional Attetntion layers.

PLaMo 3 NICT 31B Base is released under PLaMo community license. Please check the following license and agree to this before downloading.

- (EN) https://plamo.preferredai.jp/info/plamo-community-license-en
- (JA) https://plamo.preferredai.jp/info/plamo-community-license-ja

**NOTE**: This model has **NOT** been instruction-tuned for chat dialog or other downstream tasks.

### For commercial users

Please check the PLaMo community license and contact us via the following form to use commercial purpose.

- (EN/JA) https://forms.gle/mTL8tBLrMYXKNZD56


## Usage

### Requirements

```
python>=3.10.0
numpy>=1.26.4
numba>=0.60.0
torch>=2.6.0,<=2.9.0
transformers>=4.56.0,<=4.57.1
```

### Use vLLM

We provide the implementation via [vLLM plugin](https://docs.vllm.ai/en/v0.11.0/design/plugin_system.html).

See more details to setup the plugin for PLaMo 3 models: https://github.com/pfnet-research/vllm-plamo3-plugin

```python
import vllm
model = vllm.LLM("pfnet/plamo-3-nict-31b-base", trust_remote_code=True)
response = model.generate(
    prompts=["The future of artificial intelligence technology is "],
    sampling_params=vllm.SamplingParams(
        n=1,
        max_tokens=32,
        top_k=50,
        top_p=0.95,
        temperature=1.0,
    ),
)[0]
generated_text = response.outputs[0].text
print(generated_text)
```

### Load model directly

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-3-nict-31b-base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pfnet/plamo-3-nict-31b-base", trust_remote_code=True)
text = "これからの人工知能技術は"
input_ids = tokenizer(text, return_tensors="pt").input_ids
generated_tokens = model.generate(
    inputs=input_ids,
    max_new_tokens=32,
    do_sample=True,
    top_k=50,
    top_p=0.95,
    temperature=1.0,
)[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text)
```


## Model Details

- Model size: 31B
- Trained tokens: 3T
- Developed by:
    - Preferred Networks, Inc.
    - collaborative with National Institute of Information and Communications Technology, NICT
- Model type: Causal decoder-only
- Language(s): English, Japanese
- License: PLaMo community license

### Training Dataset

We trained PLaMo 3 NICT 31B Base in two phases, phase 1 with 2.625T tokens and phase 2 with 0.375T tokens.
The percentage of datasets in each phase is shown in the following table.

||2.625T (phase 1)|0.375T (phase 2)|Tokens|
|---|:---:|:---:|---:|
|English|45%|35%|1.3125T|
|Japanese|30%|40%|0.9375T|
|Code|15%|15%|0.4500T|
|Other|10%|10%|0.3000T|

The training dataset also includes the following publicly available community resources:

- [HuggingFaceFW/fineweb-2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2)
- [HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus)
- [bigcode/the-stack-v2-train-smol-ids](https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids)
- [nvidia/Nemotron-Pretraining-Dataset-sample](https://huggingface.co/datasets/nvidia/Nemotron-Pretraining-Dataset-sample)
- [EssentialAI/eai-taxonomy-stem-w-dclm](https://huggingface.co/datasets/EssentialAI/eai-taxonomy-stem-w-dclm)

### Tokenizer

PLaMo 3 NICT 31B Base tokenizer is optimized by numba, which is JIT compiler for numerical functions.
The tokenizer is trained on a subst of the datasets for model pre-training.


## Bias, Risks, and Limitations

PLaMo 3 NICT 31B Base is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo 3 NICT 31B Base's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo 3 NICT 31B Base, developers should perform safety testing and tuning tailored to their specific applications of the model.


## AI policies for Preferred Networks, Inc. group

- (EN) https://www.preferred.jp/en/company/aipolicy/
- (JA) https://www.preferred.jp/ja/company/aipolicy/