---
title: Nanbeige4.2-3B-OptiQ-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Nanbeige4.2-3B-OptiQ-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Nanbeige4.2-3B-OptiQ-4bit.md"
repository: mlx-community/Nanbeige4.2-3B-OptiQ-4bit
last_updated: 2026-09-15
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - nanbeige
architectures:
  - NanbeigeForCausalLM
base_model:
  - Nanbeige/Nanbeige4.2-3B
base_model_relation: quantized
parameters: 891.1M
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 49
stars: 0
tags:
  - mlx
  - optiq
  - nanbeige
  - looped-transformer
  - quantized
---

# Nanbeige4.2-3B-OptiQ-4bit

> Nanbeige4.2-3B-OptiQ-4bit - mlx-community 在 ModelScope 开源的模型。Nanbeige4.2-3B-OptiQ-4bit

mlx-community/Nanbeige4.2-3B-OptiQ-4bit 是 ModelScope 魔搭社区上的 891.1M 参数text-generation模型，采用 apache-2.0 许可，基于 Nanbeige/Nanbeige4.2-3B 构建。

- **Repository**: mlx-community/Nanbeige4.2-3B-OptiQ-4bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 891.1M
- **Base model**: Nanbeige/Nanbeige4.2-3B
- **Tags**: mlx, optiq, nanbeige, looped-transformer, quantized
- **Downloads**: 49
- **Stars**: 0
- **Last updated**: 2026-09-15

Source: https://www.modelscope.cn/models/mlx-community/Nanbeige4.2-3B-OptiQ-4bit

---

# Nanbeige4.2-3B-OptiQ-4bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Try the Lab](https://mlx-optiq.com/docs/lab/) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

An [OptiQ](https://mlx-optiq.com) mixed-precision **4-bit** quant of [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B) for Apple Silicon.

Nanbeige4.2 is a **looped transformer**. It has a 22-layer decoder stack, but the stack is run twice with the same shared weights (`num_loops: 2`), so a 3B-parameter model does the compute of a much deeper one. Each loop keeps its own KV cache. Stock `mlx-lm` has no `nanbeige` class, so OptiQ (0.4.6+) ships a vendored, mlx-native port of the architecture that registers itself with `mlx-lm` on `import optiq`. It is a reasoning model and opens its answers with a short chain of thought before the final response.

## Install

```bash
pip install "mlx-optiq>=0.4.6"
```

The vendored `nanbeige` architecture ships in the wheel, so this repo loads under stock `mlx-lm` once `optiq` is imported. Older OptiQ releases cannot load it.

## Usage

```python
import optiq  # registers the nanbeige architecture with mlx-lm
from mlx_lm import load, generate

model, tok = load("mlx-community/Nanbeige4.2-3B-OptiQ-4bit")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt, max_tokens=512))
```

`optiq serve --model mlx-community/Nanbeige4.2-3B-OptiQ-4bit` serves an OpenAI/Anthropic-compatible API. LoRA fine-tuning uses `optiq lora train`.

## The quant

OptiQ measures each layer's sensitivity to quantization and assigns per-layer bit-widths, rather than putting every layer at the same width. Here that gives a mix of 4-bit and 8-bit layers:

| | |
|---|---|
| Base model | Nanbeige/Nanbeige4.2-3B (bf16) |
| Quantized layers | 155 (93 at 4-bit, 62 at 8-bit) |
| Average precision | 5.5 bits/weight |
| Size on disk | 3.15 GB |
| Group size | 64 |

The full per-layer bit assignment is in `optiq_metadata.json`.

## Links

- [mlx-optiq on PyPI](https://pypi.org/project/mlx-optiq/) · [Docs](https://mlx-optiq.com/docs/) · [All OptiQ quants](https://mlx-optiq.com/models)
- Base model: [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B)