---
title: LFM2.5-ColBERT-350M-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/LFM2.5-ColBERT-350M-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/LFM2.5-ColBERT-350M-4bit.md"
repository: mlx-community/LFM2.5-ColBERT-350M-4bit
last_updated: 2026-06-23
license: other
pipeline_tag: sentence-similarity
tasks:
  - sentence-similarity
model_type:
  - lfm2
architectures:
  - Lfm2BidirectionalModel
base_model:
  - LiquidAI/LFM2.5-ColBERT-350M
base_model_relation: finetune
parameters: 55.3M
tensor_type:
  - U32
  - BF16
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - es
  - de
  - fr
  - it
  - pt
  - ar
  - sv
  - no
  - ja
  - ko
downloads: 35
stars: 0
tags:
  - mlx
  - lfm2
  - lfm2.5
  - ColBERT
  - late-interaction
  - sentence-similarity
  - feature-extraction
  - retrieval
  - 4-bit
  - quantized
---

# LFM2.5-ColBERT-350M-4bit

> LFM2.5-ColBERT-350M-4bit - mlx-community 在 ModelScope 开源的模型。LFM2.5-ColBERT-350M — MLX (4-bit)

mlx-community/LFM2.5-ColBERT-350M-4bit 是 ModelScope 魔搭社区上的 55.3M 参数sentence-similarity模型，采用 other 许可，基于 LiquidAI/LFM2.5-ColBERT-350M 构建。

- **Repository**: mlx-community/LFM2.5-ColBERT-350M-4bit
- **License**: other
- **Tasks**: sentence-similarity
- **Parameters**: 55.3M
- **Base model**: LiquidAI/LFM2.5-ColBERT-350M
- **Tags**: mlx, lfm2, lfm2.5, ColBERT, late-interaction, sentence-similarity, feature-extraction, retrieval, 4-bit, quantized
- **Downloads**: 35
- **Stars**: 0
- **Last updated**: 2026-06-23

Source: https://www.modelscope.cn/models/mlx-community/LFM2.5-ColBERT-350M-4bit

---

# LFM2.5-ColBERT-350M — MLX (4-bit)

MLX build of [**LiquidAI/LFM2.5-ColBERT-350M**](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M), a multilingual late-interaction retriever (128-dim vector **per token**, scored with MaxSim), for local inference on Apple Silicon with [MLX](https://github.com/ml-explore/mlx).

All weights, architecture, and behavior are LiquidAI's. This repository changes the file format (PyTorch/safetensors → MLX) and post-training **quantized to 4-bit** (affine, group size 64) from the bf16 MLX conversion. Every Linear and embedding layer (including the 1024→128 `Dense` projection head) is quantized; the non-quantized layers (conv, norms) stay bf16. See the [original model card](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M) for training details and intended use.

## Quantization details

- Quantized with `mlx.nn.quantize(mode='affine', bits=4, group_size=64)` — the exact configuration benchmarked below.
- **Verified bit-exact:** the reloaded checkpoint's encodings are identical (max abs diff `0`) to the in-memory-quantized model used for the benchmark (`verify_export.py`) — the shipped artifact *is* the model measured below. Downstream ColBERT NDCG can wobble ≤0.002 across processes from GPU MaxSim reduction order — a scoring artifact, not the weights.
- Reload applies the quant from `config.json["quantization"]` before loading weights (see `retrieval.load_model`).

## Evaluation

Retrieval quality of this checkpoint (and its sibling precisions), measured as **NDCG@10 / Recall@10** on judged pools. *Retention* = metric ÷ bf16 metric, averaged per-dataset.

**Setup.** English = the four **NanoBEIR** sets (full small corpora, ~2–5k passages, 50 queries each). Multilingual = **MIRACL** dev (the real queries and relevance judgments) for Spanish, German, Japanese, Arabic, each scored over a reduced pool of ~6k passages (judged positives + hard-mined negatives + sampled distractors, from `mteb/MIRACLRetrievalHardNegatives`), 100 queries each. Reduced pools make *absolute* scores easier than full-corpus MIRACL and not leaderboard-comparable — but every precision searches the identical pool, so the **retention** numbers (the point of this table) are sound. ColBERT uses brute-force MaxSim with no query augmentation, so its absolute scores sit a touch below a full PLAID setup.

### Summary (mean over 8 datasets)

| precision | NDCG@10 | NDCG retention | Recall@10 | Recall retention | size |
|---|---|---|---|---|---|
| bf16 | 0.740 | 100.0% | 0.780 | 100.0% | 707 MB |
| 8-bit | 0.741 | 100.0% | 0.779 | 99.4% | 376 MB |
| **4-bit** ◄ | 0.731 | 98.7% | 0.780 | 99.7% | 199 MB |
| mxfp4 | 0.730 | 98.5% | 0.773 | 98.8% | — |

### NDCG@10 by dataset

| dataset | bf16 | 8-bit | **4-bit** ◄ | mxfp4 |
|---|---|---|---|---|
| NanoNQ · en | 0.757 | 0.751 | 0.716 | 0.742 |
| NanoFiQA2018 · en | 0.528 | 0.512 | 0.524 | 0.520 |
| NanoSciFact · en | 0.693 | 0.712 | 0.702 | 0.682 |
| NanoNFCorpus · en | 0.345 | 0.342 | 0.335 | 0.334 |
| MIRACL · es | 0.900 | 0.901 | 0.899 | 0.900 |
| MIRACL · de | 0.823 | 0.837 | 0.826 | 0.811 |
| MIRACL · ja | 0.934 | 0.933 | 0.923 | 0.926 |
| MIRACL · ar | 0.938 | 0.941 | 0.924 | 0.926 |

## License & attribution

Redistributed under the **LFM Open License v1.0** ([`LICENSE`](LICENSE)) — the same license as the original model. Per Section 4, this notice records that the files were **modified (format conversion to MLX + 4-bit quantization)**. The original work is by **Liquid AI**; this repository is an independent conversion, **not affiliated with or endorsed by Liquid AI**. The license includes a **commercial-use threshold (Section 5)** — review it for your use case.

**Base model:** [LiquidAI/LFM2.5-ColBERT-350M](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M)