---
title: MiniCPM5-2B-chat-mlx-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/MiniCPM5-2B-chat-mlx-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/MiniCPM5-2B-chat-mlx-4bit.md"
repository: mlx-community/MiniCPM5-2B-chat-mlx-4bit
last_updated: 2026-10-14
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
base_model:
  - openbmb/MiniCPM5-2B
base_model_relation: quantized
parameters: 393.4M
tensor_type:
  - U32
  - BF16
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 13
stars: 0
tags:
  - mlx
  - safetensors
  - apple-silicon
  - 4-bit
  - conversational
  - transformers
  - llama
  - text-generation
  - minicpm
  - minicpm5
  - long-context
  - tool-calling
  - on-device
  - edge-ai
  - en
  - zh
  - "dataset:openbmb/Ultra-FineWeb"
  - "dataset:openbmb/UltraX-Preview"
  - "dataset:openbmb/Ultra-FineWeb-L3"
  - "dataset:openbmb/UltraData-Math"
  - "dataset:openbmb/UltraData-Code"
  - "dataset:openbmb/UltraData-SFT-2605"
  - "dataset:openbmb/UltraData-SFT-Agent-2609"
  - "dataset:openbmb/UltraData-RL-2609"
  - "arxiv:2506.07900"
  - "arxiv:2602.09003"
  - "license:apache-2.0"
  - text-generation-inference
  - endpoints_compatible
---

# MiniCPM5-2B-chat-mlx-4bit

> MiniCPM5-2B-chat-mlx-4bit - mlx-community 在 ModelScope 开源的模型。MiniCPM5-2B-mlx-4bit

mlx-community/MiniCPM5-2B-chat-mlx-4bit 是 ModelScope 魔搭社区上的 393.4M 参数text-generation模型，采用 apache-2.0 许可，基于 openbmb/MiniCPM5-2B 构建。

- **Repository**: mlx-community/MiniCPM5-2B-chat-mlx-4bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 393.4M
- **Base model**: openbmb/MiniCPM5-2B
- **Tags**: mlx, safetensors, apple-silicon, 4-bit, conversational, transformers, llama, text-generation, minicpm, minicpm5, long-context, tool-calling, on-device, edge-ai, en, zh, dataset:openbmb/Ultra-FineWeb, dataset:openbmb/UltraX-Preview, dataset:openbmb/Ultra-FineWeb-L3, dataset:openbmb/UltraData-Math, dataset:openbmb/UltraData-Code, dataset:openbmb/UltraData-SFT-2605, dataset:openbmb/UltraData-SFT-Agent-2609, dataset:openbmb/UltraData-RL-2609, arxiv:2506.07900, arxiv:2602.09003, license:apache-2.0, text-generation-inference, endpoints_compatible
- **Downloads**: 13
- **Stars**: 0
- **Last updated**: 2026-10-14

Source: https://www.modelscope.cn/models/mlx-community/MiniCPM5-2B-chat-mlx-4bit

---

# MiniCPM5-2B-mlx-4bit

> 4-bit MLX conversion of [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) optimized for Apple Silicon native GPU inference.

**Converted by**: [SirSahOl](https://huggingface.co/SirSahOl)  
**Source Model**: [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)  
**Framework**: [MLX](https://github.com/ml-explore/mlx) by Apple  
**Quantization**: 4-bit (Average 4.50 bits per weight)  
**Format**: safetensors  
**License**: apache-2.0  

---

## Model Details

*   **Architecture**: LlamaForCausalLM
*   **Parameters**: 2B
*   **Context Length**: 131,072 tokens
*   **Format**: MLX (Apple Silicon native GPU format)
*   **Quantization**: 4-bit (Average 4.50 bits per weight)
*   **Active VRAM Footprint**: ~1.5 GB (Minimum recommended: 8 GB Unified Memory)

---

## Quick Start

### Installation

```bash
pip install mlx-lm
```

### Usage

#### CLI

```bash
# Chat interactively
mlx_lm.chat --model SirSahOl/MiniCPM5-2B-chat-mlx-4bit

# Generate text
mlx_lm.generate --model SirSahOl/MiniCPM5-2B-chat-mlx-4bit --prompt "Write a short poem about artificial intelligence."
```

#### Python API (with Chat Template)

```python
from mlx_lm import load, generate

model, tokenizer = load("SirSahOl/MiniCPM5-2B-chat-mlx-4bit")

messages = [
    {"role": "user", "content": "Explain quantum superposition in simple terms."}
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)
```

---

## Performance Benchmarks

### Measured Benchmarks (Apple M1)

| Metric | 4-bit | 8-bit | 16-bit |
| :--- | :--- | :--- | :--- |
| **Tokens/sec** | **35.25** | **20.35** | **6.53** |
| **TTFT** | 28.38 ms | 49.23 ms | 179.1 ms |
| **Peak Memory** | 443.7 MB | 47.8 MB | 29.5 MB |

> Benchmarked on **Apple M1** with **8GB** unified memory.
> Average over 5 runs with 256 max tokens.

---

## Multi-Quantization Comparison

Evaluate your hardware budget and choose the optimal precision:

| Variant | Disk Size | VRAM Footprint | Target Apple Silicon Hardware | Key Advantage |
| :--- | :--- | :--- | :--- | :--- |
| **4-bit MLX** (This Repository) | ~1.5 GB | ~1.5 GB | M1 / M2 / M3 / M4 (8GB+) | Maximum generation speed and lowest RAM overhead. |
| **[8-bit MLX](https://huggingface.co/SirSahOl/MiniCPM5-2B-chat-mlx-8bit)** | ~2.6 GB | ~2.6 GB | M1 / M2 / M3 / M4 Pro/Max (16GB+) | Balanced accuracy and generation speed; near-lossless reasoning. |
| **[16-bit MLX](https://huggingface.co/SirSahOl/MiniCPM5-2B-chat-mlx-16bit)** | ~4.8 GB | ~4.8 GB | M2 / M3 / M4 Max/Ultra (32GB+) | Full unquantized precision; reference evaluation quality. |

---

## Who Should Use This?

| Your Hardware | Recommended Quantization |
|---------------|--------------------------|
| M1/M2/M3/M4 (8GB – 16GB) | **4-bit** — Best balance of speed, low memory, and multitasking capability |
| M1/M2/M3/M4 Pro/Max (18GB – 36GB) | **8-bit** — Higher quality reasoning with comfortable memory headroom |
| M1/M2/M3/M4 Max/Ultra (36GB – 192GB) | **16-bit** — Unquantized full precision, zero quality degradation |

**General guidance:**
- Use **4-bit** if you want to run this model alongside IDEs, browsers, and background development tools.
- Use **8-bit** if you have 16GB+ unified memory and require superior reasoning and code accuracy.
- Use **16-bit** for research, benchmarking, evaluation, or high-end workstation deployments.

---

## Other Quantization Variants

| Variant | Link |
|---------|------|
| 4-bit | [SirSahOl/MiniCPM5-2B-chat-mlx-4bit](https://huggingface.co/SirSahOl/MiniCPM5-2B-chat-mlx-4bit) |
| 8-bit | [SirSahOl/MiniCPM5-2B-chat-mlx-8bit](https://huggingface.co/SirSahOl/MiniCPM5-2B-chat-mlx-8bit) |
| 16-bit | [SirSahOl/MiniCPM5-2B-chat-mlx-16bit](https://huggingface.co/SirSahOl/MiniCPM5-2B-chat-mlx-16bit) |

---

## LM Studio & Local Inference Setup Guide

To prevent runaway loops and ensure correct conversational turn-taking, configure custom stop tokens in your local inference runtime:

1. `<|im_start|>`
2. `<|im_end|>`
3. `<|endoftext|>`

### Prompt Template Formatting

*   **System Prefix**: `<|im_start|>system\n`
*   **System Suffix**: `<|im_end|>\n`
*   **User Prefix**: `<|im_start|>user\n`
*   **Assistant Suffix**: `<|im_end|>\n<|im_start|>assistant\n`

### Ollama Quickstart

```dockerfile
FROM SirSahOl/MiniCPM5-2B-chat-mlx-4bit
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.7
```

```bash
ollama create minicpm5-2b-chat-mlx-4bit -f Modelfile
ollama run minicpm5-2b-chat-mlx-4bit
```

---

## Conversion Details

| Property | Value |
|----------|-------|
| **Source Model** | [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) |
| **Quantization** | 4-bit |
| **mlx-lm Version** | 0.31.3 |
| **Conversion Time** | 9.91s |
| **Output Size** | 1.3 GB |
| **Date** | 2026-09-22T07:05:06.791533+00:00 |

### Reproduction

To reproduce this conversion:

```bash
pip install mlx-lm==0.31.3
python3 -m mlx_lm convert --hf-path /Users/z4/.cache/huggingface/hub/models--openbmb--MiniCPM5-2B/snapshots/12a3808a956f869c767195e9266b59c4d21d92e2 --mlx-path output/MiniCPM5-2B-mlx-4bit -q --q-bits 4
```

---

## Limitations & Known Issues

- 4-bit group-wise quantization introduces minor precision loss compared to unquantized weights; for deep mathematical derivations or precision-critical reasoning, test the 8-bit or 16-bit variants.
- High context sequences (>32K tokens) require sufficient unified memory headroom; ensure unified memory is not overcommitted.
- This is a weight-only MLX conversion designed specifically for Apple Silicon GPUs (M1/M2/M3/M4 series).

---

## License

This model conversion inherits the license of the source model: **apache-2.0**.

See the [original model card](https://huggingface.co/openbmb/MiniCPM5-2B) for full license details.

---

## Changelog

| Version | Date | Changes |
|---------|------|---------|
| v1.0 | 2026-09-22 | Initial conversion |

---

*Converted with [MLX Foundry](https://github.com/SirSahOl/mlx-foundry) — a professional pipeline for converting models to Apple MLX format.*