---
title: Qwen3-Reranker-0.6B-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Qwen3-Reranker-0.6B-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Qwen3-Reranker-0.6B-4bit.md"
repository: mlx-community/Qwen3-Reranker-0.6B-4bit
last_updated: 2026-06-06
license: apache-2.0
pipeline_tag: text-ranking
tasks:
  - text-ranking
model_type:
  - qwen3
architectures:
  - Qwen3ForCausalLM
base_model:
  - Qwen/Qwen3-Reranker-0.6B
base_model_relation: quantized
parameters: 93.1M
tensor_type:
  - BF16
  - U32
library_name:
  - sentence-transformers
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 127
stars: 0
tags:
  - sentence-transformers
  - text-ranking
  - reranker
  - mlx
---

# Qwen3-Reranker-0.6B-4bit

> Qwen3-Reranker-0.6B-4bit - mlx-community 在 ModelScope 开源的模型。mlx-community/Qwen3-Reranker-0.6B-4bit

mlx-community/Qwen3-Reranker-0.6B-4bit 是 ModelScope 魔搭社区上的 93.1M 参数text-ranking模型，采用 apache-2.0 许可，基于 Qwen/Qwen3-Reranker-0.6B 构建。

- **Repository**: mlx-community/Qwen3-Reranker-0.6B-4bit
- **License**: apache-2.0
- **Tasks**: text-ranking
- **Parameters**: 93.1M
- **Base model**: Qwen/Qwen3-Reranker-0.6B
- **Tags**: sentence-transformers, text-ranking, reranker, mlx
- **Downloads**: 127
- **Stars**: 0
- **Last updated**: 2026-06-06

Source: https://www.modelscope.cn/models/mlx-community/Qwen3-Reranker-0.6B-4bit

---

# mlx-community/Qwen3-Reranker-0.6B-4bit

This model was converted to MLX format from
[Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B)
using **mlx-lm 0.31.3**.

- Quantization: **affine 4-bit, group_size=64** (~4.5 bits/weight)
- On-disk size: ~331 MB
- Task: text reranking (cross-encoder, yes/no relevance scoring)

## Scoring recipe

Qwen3-Reranker is a causal LM used as a reranker: the relevance score of a
`(query, document)` pair is `softmax([logit("no"), logit("yes")])[1]` at the
last position of the prompt below.

```python
import mlx.core as mx
from mlx_lm import load

model, tok = load("mlx-community/Qwen3-Reranker-0.6B-4bit")
hf = getattr(tok, "_tokenizer", tok)

INSTRUCT = "Given a web search query, retrieve relevant passages that answer the query"
PREFIX = ('<|im_start|>system\nJudge whether the Document meets the requirements '
          'based on the Query and the Instruct provided. Note that the answer can '
          'only be "yes" or "no".<|im_end|>\n<|im_start|>user\n')
SUFFIX = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
true_id, false_id = hf.convert_tokens_to_ids("yes"), hf.convert_tokens_to_ids("no")
pre, suf = hf.encode(PREFIX, add_special_tokens=False), hf.encode(SUFFIX, add_special_tokens=False)

def rerank_score(query, doc):
    content = f"<Instruct>: {INSTRUCT}\n<Query>: {query}\n<Document>: {doc}"
    ids = pre + hf.encode(content, add_special_tokens=False) + suf
    logits = model(mx.array([ids]))[:, -1, :]
    pair = mx.stack([logits[0, false_id], logits[0, true_id]])
    return float(mx.exp((pair - mx.logsumexp(pair))[1]))

print(rerank_score("What is the capital of China?", "The capital of China is Beijing."))
```