---
title: jina-embeddings-v2-small-en
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/jina-embeddings-v2-small-en"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/jina-embeddings-v2-small-en.md"
repository: AI-ModelScope/jina-embeddings-v2-small-en
last_updated: 2023-10-30
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - bert
architectures:
  - JinaBertForMaskedLM
parameters: 32.7M
tensor_type:
  - F16
library_name:
  - transformer
  - sentence-transformers
  - onnx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 41
stars: 0
tags:
  - finetuner
  - mteb
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
---

# jina-embeddings-v2-small-en

> jina-embeddings-v2-small-en - AI-ModelScope 在 ModelScope 开源的模型。The text embedding set trained by Jina AI , Finetuner team.

AI-ModelScope/jina-embeddings-v2-small-en 是 ModelScope 魔搭社区上的 32.7M 参数text-generation模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/jina-embeddings-v2-small-en
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 32.7M
- **Tags**: finetuner, mteb, sentence-transformers, feature-extraction, sentence-similarity
- **Downloads**: 41
- **Stars**: 0
- **Last updated**: 2023-10-30

Source: https://www.modelscope.cn/models/AI-ModelScope/jina-embeddings-v2-small-en

---

<!-- TODO: add evaluation results here -->
<br><br>

<p align="center">
<img src="https://github.com/jina-ai/finetuner/blob/main/docs/_static/finetuner-logo-ani.svg?raw=true" alt="Finetuner logo: Finetuner helps you to create experiments in order to improve embeddings on search tasks. It accompanies you to deliver the last mile of performance-tuning for neural search applications." width="150px">
</p>


<p align="center">
<b>The text embedding set trained by <a href="https://jina.ai/"><b>Jina AI</b></a>, <a href="https://github.com/jina-ai/finetuner"><b>Finetuner</b></a> team.</b>
</p>


## Intended Usage & Model Info

`jina-embeddings-v2-small-en` is an English, monolingual **embedding model** supporting **8192 sequence length**.
It is based on a Bert architecture (JinaBert) that supports the symmetric bidirectional variant of [ALiBi](https://arxiv.org/abs/2108.12409) to allow longer sequence length.
The backbone `jina-bert-v2-small-en` is pretrained on the C4 dataset.
The model is further trained on Jina AI's collection of more than 400 millions of sentence pairs and hard negatives.
These pairs were obtained from various domains and were carefully selected through a thorough cleaning process.

The embedding model was trained using 512 sequence length, but extrapolates to 8k sequence length (or even longer) thanks to ALiBi.
This makes our model useful for a range of use cases, especially when processing long documents is needed, including long document retrieval, semantic textual similarity, text reranking, recommendation, RAG and LLM-based generative search, etc.

This model has 33 million parameters, which enables lightning-fast and memory efficient inference, while still delivering impressive performance.
Additionally, we provide the following embedding models:

**V1 (Based on T5, 512 Seq)**

- [`jina-embeddings-v1-small-en`](https://huggingface.co/jinaai/jina-embedding-s-en-v1): 35 million parameters.
- [`jina-embeddings-v1-base-en`](https://huggingface.co/jinaai/jina-embedding-b-en-v1): 110 million parameters.
- [`jina-embeddings-v1-large-en`](https://huggingface.co/jinaai/jina-embedding-l-en-v1): 330 million parameters.

**V2 (Based on JinaBert, 8k Seq)**

- [`jina-embeddings-v2-small-en`](https://huggingface.co/jinaai/jina-embeddings-v2-small-en): 33 million parameters **(you are here)**.
- [`jina-embeddings-v2-base-en`](https://huggingface.co/jinaai/jina-embeddings-v2-base-en): 137 million parameters.
- [`jina-embeddings-v2-large-en`](): 435 million parameters (releasing soon).

## Data & Parameters

Jina Embeddings V2 technical report coming soon.

Jina Embeddings V1 [technical report](https://arxiv.org/abs/2307.11224).

## Usage

You can use Jina Embedding models directly from transformers package:
```python
!pip install transformers
from transformers import AutoModel
from numpy.linalg import norm

cos_sim = lambda a,b: (a @ b.T) / (norm(a)*norm(b))
model = AutoModel.from_pretrained('jinaai/jina-embeddings-v2-small-en', trust_remote_code=True) # trust_remote_code is needed to use the encode method
embeddings = model.encode(['How is the weather today?', 'What is the current weather like today?'])
print(cos_sim(embeddings[0], embeddings[1]))
```

If you only want to handle shorter sequence, such as 2k, pass the `max_length` parameter to the `encode` function:

```python
embeddings = model.encode(
    ['Very long ... document'],
    max_length=2048
)
```

*Alternatively, you can use Jina AI's Embeddings platform for fully-managed access to Jina Embeddings models (Coming soon!)*. 

## Fine-tuning

Please consider [Finetuner](https://github.com/jina-ai/finetuner).

## Plans

The development of new bilingual models is currently underway. We will be targeting mainly the German and Spanish languages.
The upcoming models will be called `jina-embeddings-v2-small-de/es`.

## Contact

Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.

## Citation

If you find Jina Embeddings useful in your research, please cite the following paper:

Coming soon.

<!-- TODO: update the paper ID once it is published on arxiv -->
<!---

``` latex
@misc{günther2023jina,
      title={Beyond the 512-Token Barrier: Training General-Purpose Text
Embeddings for Large Documents}, 
      author={Michael Günther and Jackmin Ong and Isabelle Mohr and Alaeddine Abdessalem and Tanguy Abel and Mohammad Kalim Akram and Susana Guzman and Georgios Mastrapas and Saba Sturua and Bo Wang},
      year={2023},
      eprint={2307.11224},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{günther2023jina,
      title={Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models}, 
      author={Michael Günther and Louis Milliken and Jonathan Geuter and Georgios Mastrapas and Bo Wang and Han Xiao},
      year={2023},
      eprint={2307.11224},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
-->