---
title: nomic-embed-unsupervised-data
canonical_url: "https://www.modelscope.cn/datasets/nomic-ai/nomic-embed-unsupervised-data"
md_url: "https://www.modelscope.cn/datasets/nomic-ai/nomic-embed-unsupervised-data.md"
repository: nomic-ai/nomic-embed-unsupervised-data
last_updated: 2025-01-24
license: "Apache License 2.0"
storage_size: "243 GB"
downloads: 4536
stars: 0
---

# nomic-embed-unsupervised-data

> nomic-embed-unsupervised-data - nomic-ai 在 ModelScope 开源的数据集。Weakly Supervised Contrastive Training data for Text Embedding models used in Nomic Embed models

nomic-ai/nomic-embed-unsupervised-data 是 ModelScope 魔搭社区上的数据集，存储大小 243 GB，采用 Apache License 2.0 许可。

- **Repository**: nomic-ai/nomic-embed-unsupervised-data
- **License**: Apache License 2.0
- **Storage size**: 243 GB
- **Downloads**: 4536
- **Stars**: 0
- **Last updated**: 2025-01-24

Source: https://www.modelscope.cn/datasets/nomic-ai/nomic-embed-unsupervised-data

---

Weakly Supervised Contrastive Training data for Text Embedding models used in [Nomic Embed](https://huggingface.co/collections/nomic-ai/nomic-embed-65c0426827a5fdca81a87b89) models


## Training
Click the Nomic Atlas map below to visualize a 5M sample of our contrastive pretraining data!

[![image/webp](https://cdn-uploads.huggingface.co/production/uploads/607997c83a565c15675055b3/pjhJhuNyRfPagRd_c_iUz.webp)](https://atlas.nomic.ai/map/nomic-text-embed-v1-5m-sample)

We train our embedder using a multi-stage training pipeline. Starting from a long-context [BERT model](https://huggingface.co/nomic-ai/nomic-bert-2048),
the first unsupervised contrastive stage trains on a dataset generated from weakly related text pairs, such as question-answer pairs from forums like StackExchange and Quora, title-body pairs from Amazon reviews, and summarizations from news articles.

In the second finetuning stage, higher quality labeled datasets such as search queries and answers from web searches are leveraged. Data curation and hard-example mining is crucial in this stage.

For more details, see the Nomic Embed [Technical Report](https://static.nomic.ai/reports/2024_Nomic_Embed_Text_Technical_Report.pdf) and corresponding [blog post](https://blog.nomic.ai/posts/nomic-embed-text-v1).

Training data to train the models is released in its entirety. For more details, see the `contrastors` [repository](https://github.com/nomic-ai/contrastors)


# Join the Nomic Community

- Nomic: [https://nomic.ai](https://nomic.ai)
- Discord: [https://discord.gg/myY5YDR8z8](https://discord.gg/myY5YDR8z8)
- Twitter: [https://twitter.com/nomic_ai](https://twitter.com/nomic_ai)


# Citation

If you find the model, dataset, or training code useful, please cite our work

```bibtex
@misc{nussbaum2024nomic,
      title={Nomic Embed: Training a Reproducible Long Context Text Embedder}, 
      author={Zach Nussbaum and John X. Morris and Brandon Duderstadt and Andriy Mulyar},
      year={2024},
      eprint={2402.01613},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```