---
title: VibeThinker-3B-OptiQ-4bit
canonical_url: "https://www.modelscope.cn/models/latentnode/VibeThinker-3B-OptiQ-4bit"
md_url: "https://www.modelscope.cn/models/latentnode/VibeThinker-3B-OptiQ-4bit.md"
repository: latentnode/VibeThinker-3B-OptiQ-4bit
chinese_name: VibeThinker-3B-OptiQ-4bit
last_updated: 2026-07-27
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
base_model:
  - mlx-community/VibeThinker-3B
base_model_relation: quantized
parameters: 618.6M
tensor_type:
  - U32
  - BF16
library_name:
  - mlx
  - safetensors
downloads: 9
stars: 0
tags:
  - mlx
  - quantized
  - mixed-precision
  - 4bit
  - 8bit
  - optiq
  - apple-silicon
  - text-generation
  - reasoning
---

# VibeThinker-3B-OptiQ-4bit

> VibeThinker-3B-OptiQ-4bit - latentnode 在 ModelScope 开源的模型。mlx-community/VibeThinker-3B-OptiQ-4bit

latentnode/VibeThinker-3B-OptiQ-4bit 是 ModelScope 魔搭社区上的 618.6M 参数text-generation模型，采用 mit 许可，基于 mlx-community/VibeThinker-3B 构建。

- **Repository**: latentnode/VibeThinker-3B-OptiQ-4bit
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 618.6M
- **Base model**: mlx-community/VibeThinker-3B
- **Tags**: mlx, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon, text-generation, reasoning
- **Downloads**: 9
- **Stars**: 0
- **Last updated**: 2026-07-27

Source: https://www.modelscope.cn/models/latentnode/VibeThinker-3B-OptiQ-4bit

---

# mlx-community/VibeThinker-3B-OptiQ-4bit

A 4-bit mixed-precision MLX quant produced by [mlx-optiq](https://mlx-optiq.com/) — the sensitivity-aware quantization toolkit for Apple Silicon. It edges stock uniform 4-bit on the six-metric Capability Score and stays markedly closer to the original bf16 weights (KL 0.58 vs 1.44).

A 4-bit mixed-precision MLX quant of [mlx-community/VibeThinker-3B](https://huggingface.co/mlx-community/VibeThinker-3B) (a reasoning model fine-tuned from Qwen2.5-Coder-3B). Per-layer bit-widths come from a KL-divergence sensitivity pass on a [six-domain calibration mix](https://mlx-optiq.com/blog/calibration-mix) (prose · reasoning · code · agent · tool-call · constraint-bearing instructions). Sensitive layers go to 8-bit; robust ones stay at 4-bit.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 4-bit |
| Layers at 8-bit (sensitive) | 141 |
| Layers at 4-bit (robust) | 111 |
| Total quantized layers | 252 |
| Achieved bits per weight | 5.12 |
| Group size | 64 |
| Calibration mix | [six-domain mix](https://mlx-optiq.com/blog/calibration-mix) (40 samples × 6 domains) |
| Reference for sensitivity | bf16 |

We follow the same naming convention `llama.cpp` uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average. The few sensitive layers held at 8-bit make this build +0.5 GB larger than a stock uniform-4-bit quant (2.1 GB vs 1.6 GB) while recovering quality the uniform quant loses.

## Usage

Load it with `mlx-lm` and use it as usual:

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/VibeThinker-3B-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=512,
)
```

This is a reasoning model — it emits a `<think>…</think>` block before its final answer. Give it room with `max_tokens`.

For more (mixed-precision KV-cache serving, sensitivity-aware LoRA fine-tuning, OpenAI + Anthropic-compatible inference server with structured/JSON output, hot-swap mounted adapters), install [`mlx-optiq`](https://mlx-optiq.com/):

```bash
pip install mlx-optiq
```

## Links

- **Project website:** [mlx-optiq.com](https://mlx-optiq.com/)
- **Docs:** [mlx-optiq.com/docs](https://mlx-optiq.com/docs/)
- **PyPI:** [pypi.org/project/mlx-optiq](https://pypi.org/project/mlx-optiq/)
- **Calibration mix:** [mlx-optiq.com/blog/calibration-mix](https://mlx-optiq.com/blog/calibration-mix)
- **Eval framework:** [mlx-optiq.com/blog/eval-framework](https://mlx-optiq.com/blog/eval-framework)
- **Base model:** [mlx-community/VibeThinker-3B](https://huggingface.co/mlx-community/VibeThinker-3B)

## License

MIT (inherits from the base model).