---
title: "Gemini Embedding: Generalizable Embeddings from Gemini"
canonical_url: "https://www.modelscope.cn/papers/125362"
md_url: "https://www.modelscope.cn/papers/125362.md"
arxiv_id: 2503.07891
published: 2025-03-10
last_updated: 2025-03-10
authors:
  - "Jinhyuk Lee"
  - "Feiyang Chen"
  - "Sahil Dua"
  - "Daniel Cer"
  - "Madhuri Shanbhogue"
  - "Iftekhar Naim"
  - "Gustavo Hernández Ábrego"
  - "Zhe Li"
  - "Kaifeng Chen"
  - "Henrique Schechter Vera"
  - "Xiaoqi Ren"
  - "Shanfeng Zhang"
  - "Daniel Salz"
  - "Michael Boratko"
  - "Jay Han"
  - "Blair Chen"
  - "Shuo Huang"
  - "Vikram Rao"
  - "Paul Suganthan"
  - "Feng Han"
  - "Andreas Doumanoglou"
  - "Nithi Gupta"
  - "Fedor Moiseev"
  - "Cathy Yip"
  - "Aashi Jain"
  - "Simon Baumgartner"
  - "Shahrokh Shahi"
  - "Frank Palma Gomez"
  - "Sandeep Mariserla"
  - "Min Choi"
  - "Parashar Shah"
  - "Sonam Goenka"
  - "Ke Chen"
  - "Ye Xia"
  - "Koert Chen"
  - "Sai Meher Karthik Duddu"
  - "Yichang Chen"
  - "Trevor Walker"
  - "Wenlei Zhou"
  - "Rakesh Ghiya"
  - "Zach Gleicher"
  - "Karan Gill"
  - "Zhe Dong"
  - "Mojtaba Seyedhosseini"
  - "Yunhsuan Sung"
  - "Raphael Hoffmann"
  - "Tom Duerig"
model_name: "Gemini Embedding"
model_developer: Google
domain:
  - "自然语言处理"
  - "文本嵌入"
  - "多语言表示学习"
  - "信息检索"
  - "代码检索"
type:
  - "自然语言处理"
  - "文本嵌入"
  - "多语言表示学习"
  - "信息检索"
  - "代码检索"
  - "Computation and Language"
  - "Artificial Intelligence"
arxiv_url: "https://arxiv.org/abs/2503.07891"
pdf_url: "https://arxiv.org/pdf/2503.07891"
---

# Gemini Embedding: Generalizable Embeddings from Gemini

> In this report, we introduce Gemini Embedding, a state-of-the-art embedding model leveraging the power of Gemini, Google's most capable large language model. Capitalizing on Gemini's inherent multilingual and code understanding capabilities, Gemini Embedding…

「Gemini Embedding: Generalizable Embeddings from Gemini」是 ModelScope 魔搭社区收录的论文，arXiv 2503.07891，作者为 Jinhyuk Lee, Feiyang Chen, Sahil Dua et al.，发表于 2025-03-10，属于 自然语言处理、文本嵌入、多语言表示学习 领域。

- **ArXiv**: 2503.07891
- **Published**: 2025-03-10
- **Authors**: Jinhyuk Lee, Feiyang Chen, Sahil Dua, Daniel Cer, Madhuri Shanbhogue, Iftekhar Naim, Gustavo Hernández Ábrego, Zhe Li, Kaifeng Chen, Henrique Schechter Vera, Xiaoqi Ren, Shanfeng Zhang, Daniel Salz, Michael Boratko, Jay Han, Blair Chen, Shuo Huang, Vikram Rao, Paul Suganthan, Feng Han, Andreas Doumanoglou, Nithi Gupta, Fedor Moiseev, Cathy Yip, Aashi Jain, Simon Baumgartner, Shahrokh Shahi, Frank Palma Gomez, Sandeep Mariserla, Min Choi, Parashar Shah, Sonam Goenka, Ke Chen, Ye Xia, Koert Chen, Sai Meher Karthik Duddu, Yichang Chen, Trevor Walker, Wenlei Zhou, Rakesh Ghiya, Zach Gleicher, Karan Gill, Zhe Dong, Mojtaba Seyedhosseini, Yunhsuan Sung, Raphael Hoffmann, Tom Duerig
- **Model**: Gemini Embedding
- **Developer**: Google
- **Domain**: 自然语言处理, 文本嵌入, 多语言表示学习, 信息检索, 代码检索
- **ArXiv URL**: https://arxiv.org/abs/2503.07891
- **PDF**: https://arxiv.org/pdf/2503.07891

Source: https://www.modelscope.cn/papers/125362

---

> Gemini Embedding：跨越250种语言的通用嵌入模型新标杆

## 摘要

本文介绍了Gemini Embedding，一种基于Google最强大的大型语言模型Gemini的最新嵌入模型。Gemini Embedding利用Gemini的多语言和代码理解能力，生成适用于多种文本模式的高度通用嵌入表示。这些嵌入可以预先计算并应用于分类、相似性、聚类、排序和检索等多种下游任务。在包含超过100个任务和250多种语言的Massive Multilingual Text Embedding Benchmark（MMTEB）上，Gemini Embedding显著优于先前的最先进模型，展示了嵌入质量的重大改进。尤其在多语言、英语和代码基准测试中表现出色，超越了特定领域的专用模型。

研究背景方面，嵌入模型通过将输入转换为密集向量表示，在捕捉各种领域和模态的语义信息方面至关重要。近年来，研究人员致力于开发能够在多样化下游任务中表现优异的通用嵌入模型，而大型语言模型（LLMs）因其丰富的预训练知识成为构建此类模型的有前途的方向。Gemini Embedding通过从Gemini初始化参数，并结合对比学习目标进行训练，以提高其鲁棒性和泛化能力。

提出的方法包括使用Gemini进行数据清洗、生成高质量合成数据集以及硬负样本挖掘，从而构建一个高质量的异构训练数据集。此外，Gemini Embedding引入了任务提示和预微调阶段，并采用Model Soup技术平均多个微调检查点，以获得最终的嵌入模型。

取得的结果显示，Gemini Embedding在MMTEB等多个基准测试中取得了最先进的性能，不仅在高资源语言如英语中表现出色，也在低资源语言如马其顿语中展现了卓越的表现。

## Abstract

In this report, we introduce Gemini Embedding, a state-of-the-art embedding model leveraging the power of Gemini, Google's most capable large language model. Capitalizing on Gemini's inherent multilingual and code understanding capabilities, Gemini Embedding produces highly generalizable embeddings for text spanning numerous languages and textual modalities. The representations generated by Gemini Embedding can be precomputed and applied to a variety of downstream tasks including classification, similarity, clustering, ranking, and retrieval. Evaluated on the Massive Multilingual Text Embedding Benchmark (MMTEB), which includes over one hundred tasks across 250+ languages, Gemini Embedding substantially outperforms prior state-of-the-art models, demonstrating considerable improvements in embedding quality. Achieving state-of-the-art performance across MMTEB's multilingual, English, and code benchmarks, our unified model demonstrates strong capabilities across a broad selection of tasks and surpasses specialized domain-specific models.
