---
title: bge-m3-mlx-6bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/bge-m3-mlx-6bit"
md_url: "https://www.modelscope.cn/models/mlx-community/bge-m3-mlx-6bit.md"
repository: mlx-community/bge-m3-mlx-6bit
last_updated: 2026-03-16
license: mit
pipeline_tag: feature-extraction
tasks:
  - feature-extraction
model_type:
  - xlm-roberta
architectures:
  - XLMRobertaModel
base_model:
  - BAAI/bge-m3
base_model_relation: finetune
parameters: 124.4M
tensor_type:
  - F16
  - U32
library_name:
  - mlx
  - sentence-transformers
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - multilingual
  - en
  - zh
  - de
  - es
  - fr
  - it
  - ja
  - ko
  - pt
  - ru
inference_backends:
  - "deploy_task emb"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 81
stars: 0
tags:
  - mlx
  - embedding
  - text-embeddings-inference
  - retrieval
  - sentence-transformers
  - 6bit
  - quantized
---

# bge-m3-mlx-6bit

> bge-m3-mlx-6bit - mlx-community 在 ModelScope 开源的模型。BGE-M3 MLX (6-bit Quantized)

mlx-community/bge-m3-mlx-6bit 是 ModelScope 魔搭社区上的 124.4M 参数feature-extraction模型，采用 mit 许可，基于 BAAI/bge-m3 构建，可用 deploy_task emb、sglang 0.5.2、vllm 0.9.2 部署。

- **Repository**: mlx-community/bge-m3-mlx-6bit
- **License**: mit
- **Tasks**: feature-extraction
- **Parameters**: 124.4M
- **Base model**: BAAI/bge-m3
- **Inference backends**: deploy_task emb, sglang 0.5.2, vllm 0.9.2
- **Tags**: mlx, embedding, text-embeddings-inference, retrieval, sentence-transformers, 6bit, quantized
- **Downloads**: 81
- **Stars**: 0
- **Last updated**: 2026-03-16

Source: https://www.modelscope.cn/models/mlx-community/bge-m3-mlx-6bit

---

# BGE-M3 MLX (6-bit Quantized)

This is the [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) model converted to MLX format with 6-bit quantization for Apple Silicon.

## Model Description

BGE-M3 is a versatile embedding model capable of:
- Dense retrieval
- Sparse retrieval
- Multi-vector (ColBERT) retrieval

This 6-bit quantized version offers a good balance between size and quality.

## Model Details

| Property | Value |
|----------|-------|
| Architecture | XLM-RoBERTa |
| Precision | 6-bit (affine quantization) |
| Embedding Dimension | 1024 |
| Max Sequence Length | 8192 |
| Model Size | ~457 MB |
| Quantization Group Size | 64 |
| Languages | 100+ languages |

## Size Comparison

| Version | Size | Compression |
|---------|------|-------------|
| FP16 | 1.1 GB | - |
| 8-bit | 592 MB | 46% |
| **6-bit** | **457 MB** | **58%** |
| 4-bit | 321 MB | 71% |

## Usage

### With MLX

```python
from mlx_embeddings.utils import load_model, load_tokenizer
import mlx.core as mx

model_path = "mlx-community/bge-m3-mlx-6bit"

# Load model and tokenizer
model = load_model(model_path)
tokenizer = load_tokenizer(model_path)

# Generate embeddings
text = "Hello, world!"
tokens = tokenizer.encode(text)
input_ids = mx.array([tokens])
output = model(input_ids)
embedding = output.last_hidden_state.mean(axis=1)  # Mean pooling

print(f"Embedding shape: {embedding.shape}")  # (1, 1024)
```

### With oMLX

```bash
curl http://127.0.0.1:8000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model": "bge-m3-mlx-6bit", "input": "Your text here"}'
```

## Quantization Details

- **Method**: Affine quantization
- **Bits per weight**: 6
- **Group size**: 64
- **Source**: Converted from MLX FP16 version

## Recommended Use Cases

- Balanced performance and memory usage
- Production deployments with moderate memory constraints
- Applications requiring better quality than 4-bit

## License

MIT license (inherited from BAAI/bge-m3)

## Citation

```bibtex
@article{bge_m3,
  title={BGE M3-Embedding: Accurate, Efficient and Versatile Text Embedding},
  author={Chen, Jianlv and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Zhang, Zheng},
  journal={arXiv preprint arXiv:2402.03216},
  year={2024}
}
```

## Disclaimer

This is an unofficial MLX conversion of the BAAI/bge-m3 model. For the original model, see [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3).