---
title: npm-single
canonical_url: "https://www.modelscope.cn/models/facebook/npm-single"
md_url: "https://www.modelscope.cn/models/facebook/npm-single.md"
repository: facebook/npm-single
last_updated: 2025-05-20
license: cc-by-nc-4.0
pipeline_tag: fill-mask
tasks:
  - fill-mask
model_type:
  - roberta
architectures:
  - RobertaForMaskedLM
library_name:
  - pytorch
frameworks:
  - pytorch
inference_backends:
  - "deploy_task emb"
  - "vllm 0.9.2"
downloads: 329
stars: 0
---

# npm-single

> npm-single - facebook 在 ModelScope 开源的模型。NPM-single is a nonparametric masked language model, pretrained on English text data. It was introduced by ["Nonparametric Masked Language Modeling"][paper] and first released in [facebookresearch/NPM][repo].

facebook/npm-single 是 ModelScope 魔搭社区上的fill-mask模型，采用 cc-by-nc-4.0 许可，可用 deploy_task emb、vllm 0.9.2 部署。

- **Repository**: facebook/npm-single
- **License**: cc-by-nc-4.0
- **Tasks**: fill-mask
- **Inference backends**: deploy_task emb, vllm 0.9.2
- **Downloads**: 329
- **Stars**: 0
- **Last updated**: 2025-05-20

Source: https://www.modelscope.cn/models/facebook/npm-single

---

# NPM-single

NPM-single is a nonparametric masked language model, pretrained on English text data.
It was introduced by ["Nonparametric Masked Language Modeling"][paper]
and first released in [facebookresearch/NPM][repo].

### Model description

NPM consists of an encoder and a reference corpus, and models a nonparametric distribution over a reference corpus.
The key idea is to map all the phrases in the corpus into a dense vector space using the
encoder and, when given a query with a MASK at inference, use the encoder to locate the nearest
phrase from the corpus and fill in the MASK.

NPM-single is a variant of NPM that retrieves a token from the corpus, instead of a phrase.

### Intended uses & limitations
While this repo includes the encoder weights, NPM-single has to be used together with a datstore.
For more details on how to use NPM-single, please refer to the [original repo][repo].

Note that this model is primarily for filling in a MASK token. Future work can investigate how to use NPM-single for text generation.

### Training procedure

NPM-single was trained on English Wikipedia (August 2019) and an English portion of CC-News (Mackenzie et al. (2020), February 2019), which contains 13B tokens in total.
NPM-single used the model architecture and initial weights of RoBERTa large (Liu et al., 2019), consisting of 354M parameters.
Training is done for 100,000 steps, using thirty-two 32GB GPUs. 

More details about training can be found in the [paper][paper].
Code for training NPM-single can be found in the [original repo][repo].

### Evaluation results
NPM-single is evaluated on nine closed-set tasks (tasks with a small set of options given).
NPM-single consistently outperforms significantly larger models such as GPT-3 and T5.
Detailed results can be found from the [paper][paper].

### BibTeX entry and citation info
```
@article{ min2022nonparametric,
    title={ Nonparametric Masked Language Modeling },
    author={ Min, Sewon and Shi, Weijia and Lewis, Mike and Chen, Xilun and Yih, Wen-tau and Hajishirzi, Hannaneh and Zettlemoyer, Luke },
    year={ 2022 }
}
```

[paper]: https://arxiv.org/abs/2212.01349
[repo]: https://github.com/facebookresearch/NPM