---
title: mini-coder-4b-OptiQ-4bit
canonical_url: "https://www.modelscope.cn/models/latentnode/mini-coder-4b-OptiQ-4bit"
md_url: "https://www.modelscope.cn/models/latentnode/mini-coder-4b-OptiQ-4bit.md"
repository: latentnode/mini-coder-4b-OptiQ-4bit
chinese_name: mini-coder-4b-OptiQ-4bit
last_updated: 2026-07-27
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3
architectures:
  - Qwen3ForCausalLM
base_model:
  - ricdomolm/mini-coder-4b
base_model_relation: quantized
parameters: 808.9M
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
downloads: 8
stars: 0
tags:
  - mlx
  - quantized
  - mixed-precision
  - 4bit
  - 8bit
  - optiq
  - apple-silicon
  - text-generation
  - code
  - agent
---

# mini-coder-4b-OptiQ-4bit

> mini-coder-4b-OptiQ-4bit - latentnode 在 ModelScope 开源的模型。mlx-community/mini-coder-4b-OptiQ-4bit

latentnode/mini-coder-4b-OptiQ-4bit 是 ModelScope 魔搭社区上的 808.9M 参数text-generation模型，采用 mit 许可，基于 ricdomolm/mini-coder-4b 构建。

- **Repository**: latentnode/mini-coder-4b-OptiQ-4bit
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 808.9M
- **Base model**: ricdomolm/mini-coder-4b
- **Tags**: mlx, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon, text-generation, code, agent
- **Downloads**: 8
- **Stars**: 0
- **Last updated**: 2026-07-27

Source: https://www.modelscope.cn/models/latentnode/mini-coder-4b-OptiQ-4bit

---

# mlx-community/mini-coder-4b-OptiQ-4bit

A 4-bit mixed-precision MLX quant produced by [mlx-optiq](https://mlx-optiq.com/) — the sensitivity-aware quantization toolkit for Apple Silicon. It beats stock uniform 4-bit on the six-metric Capability Score (45.83 vs 42.74, +3.09) and is ~2.2x more faithful to the original bf16 weights (KL 0.06 vs 0.13).

A 4-bit mixed-precision MLX quant of [ricdomolm/mini-coder-4b](https://huggingface.co/ricdomolm/mini-coder-4b) (a code/agent model fine-tuned from Qwen3-4B-Instruct-2507). Per-layer bit-widths come from a KL-divergence sensitivity pass on a [six-domain calibration mix](https://mlx-optiq.com/blog/calibration-mix) (prose · reasoning · code · agent · tool-call · constraint-bearing instructions). Sensitive layers go to 8-bit; robust ones stay at 4-bit.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 4-bit |
| Layers at 8-bit (sensitive) | 123 |
| Layers at 4-bit (robust) | 129 |
| Total quantized layers | 252 |
| Achieved bits per weight | 5.16 |
| Group size | 64 |
| Calibration mix | [six-domain mix](https://mlx-optiq.com/blog/calibration-mix) (40 samples × 6 domains) |
| Reference for sensitivity | bf16 |

We follow the same naming convention `llama.cpp` uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average. The few sensitive layers held at 8-bit make this build +0.7 GB larger than a stock uniform-4-bit quant (2.8 GB vs 2.1 GB) while recovering quality the uniform quant loses.

## Usage

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/mini-coder-4b-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Write a Python function to check if a string is a palindrome.",
    max_tokens=512,
)
```

It's a code/agent model — best paired with a lightweight agent harness / loop. For more (mixed-precision KV-cache serving, sensitivity-aware LoRA fine-tuning, OpenAI + Anthropic-compatible inference server with structured/JSON output and server-side tool-call healing, hot-swap mounted adapters), install [`mlx-optiq`](https://mlx-optiq.com/):

```bash
pip install mlx-optiq
```

## Benchmarks

Six-metric Capability Score (mean of MMLU + GSM8K + IFEval + BFCL + HumanEval + HashHop). Apples-to-apples comparison against stock uniform 4-bit:

| Metric | OptiQ | Uniform 4-bit | Δ |
|---|---:|---:|---:|
| MMLU (5-shot, 1000 samples) | **69.9%** | 68.1% | +1.8 |
| GSM8K (1000 samples, CoT) | **59.6%** | 48.1% | +11.5 |
| IFEval (full set, strict) | **36.6%** | 38.4% | -1.8 |
| BFCL-V3 simple (200 calls) | **56.5%** | 47.5% | +9.0 |
| HumanEval (164 problems, pass@1) | **52.4%** | 54.3% | -1.8 |
| HashHop (long-context retrieval) | **0.0%** | 0.0% | +0.0 |
| **Capability Score** (mean of 6) | **45.83** | 42.74 | **+3.09** |
| KL vs bf16 (mean / p95) | **0.0571** / 0.2336 | 0.1277 / 0.4872 | — |
| On-disk size | 2.8 GB | 2.1 GB | +0.7 |

OptiQ's clearest wins are **GSM8K (+11.5)** and **tool-calling (BFCL +9.0)**, plus markedly better fidelity to the original weights. IFEval and HumanEval are within noise (~±7 pp at these sample sizes). HashHop is 0% for **both** quants: a genuine model-capability gap — the model has no long-context multi-hop retrieval ability (verified by inspecting outputs, it reasons in prose instead of walking the hash chain) — not a quantization artifact, which is why OptiQ and uniform-4 match there.

Every metric gets one equal vote; disk size is an honest second axis, not folded into the score. See the [eval-framework writeup](https://mlx-optiq.com/blog/eval-framework).

## Links

- **Project website:** [mlx-optiq.com](https://mlx-optiq.com/)
- **Docs:** [mlx-optiq.com/docs](https://mlx-optiq.com/docs/)
- **PyPI:** [pypi.org/project/mlx-optiq](https://pypi.org/project/mlx-optiq/)
- **Eval framework:** [mlx-optiq.com/blog/eval-framework](https://mlx-optiq.com/blog/eval-framework)
- **Base model:** [ricdomolm/mini-coder-4b](https://huggingface.co/ricdomolm/mini-coder-4b)

## License

MIT (inherits from the base model).