---
title: MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit.md"
repository: mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit
last_updated: 2026-09-15
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
base_model:
  - GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
base_model_relation: quantized
parameters: 243.6M
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 44
stars: 0
tags:
  - mlx
  - quantized
  - mixed-precision
  - 4bit
  - 8bit
  - optiq
  - apple-silicon
  - minicpm
  - minicpm5
  - llama
  - text-generation
  - thinking
  - fable5
  - tool-calling
  - function-calling
  - coding
  - instruction-following
  - conversational
---

# MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit

> MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit - mlx-community 在 ModelScope 开源的模型。mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit

mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit 是 ModelScope 魔搭社区上的 243.6M 参数text-generation模型，采用 apache-2.0 许可，基于 GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking 构建。

- **Repository**: mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 243.6M
- **Base model**: GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
- **Tags**: mlx, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon, minicpm, minicpm5, llama, text-generation, thinking, fable5, tool-calling, function-calling, coding, instruction-following, conversational
- **Downloads**: 44
- **Stars**: 0
- **Last updated**: 2026-09-15

Source: https://www.modelscope.cn/models/mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit

---

# mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Try the Lab](https://mlx-optiq.com/docs/lab/) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

A 4-bit mixed-precision MLX quant of [GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking](https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking), a reasoning fine-tune of MiniCPM5-1B. Sensitive layers are kept at 8-bit and robust ones at 4-bit.

2.0 GB of bf16 weights become **874 MB**, small enough to run comfortably on any Apple Silicon Mac.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 4-bit |
| Layers at 8-bit (sensitive) | 67 |
| Layers at 4-bit (robust) | 102 |
| Total quantized layers | 169 |
| Achieved bits per weight | 5.805 |
| Group size | 64 |
| Size on disk | 874 MB, from a 2.0 GB bf16 base |

We follow the same naming convention `llama.cpp` uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is the predominant precision, not the weighted average. At 1B parameters there is less redundancy to spend, so the optimizer keeps more layers at 8-bit and the weighted average lands higher than on a larger model.

### How the bit-widths were chosen

The per-layer allocation is transferred from [mlx-community/MiniCPM5-1B-OptiQ-4bit](https://huggingface.co/mlx-community/MiniCPM5-1B-OptiQ-4bit), where it was derived by a KL-divergence sensitivity sweep against the bf16 reference on a [six-domain calibration mix](https://mlx-optiq.com/blog/calibration-mix).

This model is a fine-tune of `openbmb/MiniCPM5-1B` with an unchanged architecture, so all 169 quantizable layers map across exactly and the allocation lands at the same 5.805 bits per weight when recomputed against this model's own tensors.

These are **measured** bit-widths, not a static rule-of-thumb recipe. But they were measured on the base model, not on this fine-tune. Fine-tuning shifts weights, so this model's own per-layer sensitivities could differ somewhat from the base's. Which layers are fragile is mostly a property of the architecture, so the transfer is sound, but it is a transfer and you should know that.

## Usage

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is 17 + 25?"}],
    add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```

This is a reasoning model: it thinks inside `<think>...</think>` before answering, so give it enough `max_tokens` to finish.

For an OpenAI-compatible server, mixed-precision KV-cache serving, and sensitivity-aware LoRA fine-tuning, install [`mlx-optiq`](https://mlx-optiq.com/):

```bash
pip install mlx-optiq
optiq serve --model mlx-community/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-OptiQ-4bit
```

## Verification

Generation was exercised on the finished artifact before release: raw completion, chat-template reasoning (it correctly reasons to `42` for `17 + 25`), and free prose.

The quantization was also checked numerically: dequantizing individual layers out of the artifact and comparing them against the bf16 checkpoint gives 0.7% mean relative error on the 8-bit layers and 9.8% on the 4-bit layers, which is what each bit-width should cost.

No task benchmarks were run on this quant; for measured quality numbers on the base architecture, see the [MiniCPM5-1B OptiQ card](https://huggingface.co/mlx-community/MiniCPM5-1B-OptiQ-4bit).

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as [the original](https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking).