---
title: Ternary-Bonsai-2-4B-2bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Ternary-Bonsai-2-4B-2bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Ternary-Bonsai-2-4B-2bit.md"
repository: mlx-community/Ternary-Bonsai-2-4B-2bit
last_updated: 2026-10-16
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - prism_hadamard_qwen35
base_model:
  - Qwen/Qwen3.5-4B
base_model_relation: finetune
parameters: 667.4M
tensor_type:
  - F16
  - U32
  - F32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 37
stars: 0
tags:
  - ternary
  - 2-bit
  - mlx
  - metal
  - on-device
  - hybrid-attention
  - vision
  - prismml
  - bonsai
---

# Ternary-Bonsai-2-4B-2bit

> Ternary-Bonsai-2-4B-2bit - mlx-community 在 ModelScope 开源的模型。Prism ML Website  |  Demo & Examples  |  Discord

mlx-community/Ternary-Bonsai-2-4B-2bit 是 ModelScope 魔搭社区上的 667.4M 参数image-text-to-text模型，采用 apache-2.0 许可，基于 Qwen/Qwen3.5-4B 构建。

- **Repository**: mlx-community/Ternary-Bonsai-2-4B-2bit
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Parameters**: 667.4M
- **Base model**: Qwen/Qwen3.5-4B
- **Tags**: ternary, 2-bit, mlx, metal, on-device, hybrid-attention, vision, prismml, bonsai
- **Downloads**: 37
- **Stars**: 0
- **Last updated**: 2026-10-16

Source: https://www.modelscope.cn/models/mlx-community/Ternary-Bonsai-2-4B-2bit

---

<p align="center">
  <a href="https://prismml.com"><b>Prism ML Website</b></a> &nbsp;|&nbsp;
  <a href="https://github.com/PrismML-Eng/Bonsai-demo"><b>Demo &amp; Examples</b></a> &nbsp;|&nbsp;
  <a href="https://discord.gg/prismml"><b>Discord</b></a>
</p>

# Bonsai 2 Ternary 4B: MLX

A 4B-class vision-language model with ternary transformer weights, packed for MLX on Apple Silicon.

> **1.87 GB** on disk, language model + vision tower (FP16 language model: ~8.4 GB)

## Highlights

- **Ternary language weights** stored in MLX's 2-bit grouped format (group 128). MLX stores a scale and a bias per group, so the container costs a little more than the GGUF packings; the weights decode to exactly the same ternary values.
- **Vision input** through the official vision tower, carried unrotated in FP16.
- **262K-token context**, inherited from the Qwen3.5-4B hybrid-attention backbone (24 linear-attention and 8 full-attention layers).

## Model Overview

| Item | Specification |
| :--- | :--- |
| Base model | Derived from Qwen3.5-4B (architecture unchanged) |
| Parameters | ~4.21B language model (32 blocks, hidden 2560) + ~0.33B vision tower |
| Weight format | Ternary {−1, 0, +1} in MLX affine 2-bit, group 128 |
| Weight basis | Blockwise Hadamard rotation folded into the stored weights; the bundled runtime applies the matching transform to activations |
| Backends | MLX on Apple Silicon (Metal): mlx-vlm 0.7.8+ natively, or the loader in `runtime/` |
| License | Apache 2.0 |

## Files

| File | Contents | Size |
| :--- | :--- | ---: |
| `model.safetensors` | language model (2-bit, group 128) + vision tower (FP16) | 1.87 GB |
| `mtp/` | multi-token-prediction draft head, 4-bit (optional, see below) | 0.07 GB |
| `runtime/` | a bundled loader for older mlx-vlm versions, plus pinned `requirements.txt` | |
| `quickstart.py` | one-command test with the recommended sampler | |
| `PACK-RUNTIME.md` | the bundled loader's contract and limits | |

## Quickstart

> **[PrismML-Eng/Bonsai-demo](https://github.com/PrismML-Eng/Bonsai-demo) is the source of truth for running these models.** Where anything here disagrees with it, it is right.

```bash
hf download prism-ml/Ternary-Bonsai-2-4B-mlx-2bit --local-dir bonsai2-4b-mlx
pip install "mlx-vlm>=0.7.8"
```

mlx-vlm 0.7.8 and newer load this pack natively (`model_type: prism_hadamard_qwen35`):

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("bonsai2-4b-mlx")
prompt = apply_chat_template(processor, model.config, "What is in this image?", num_images=1)
print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=4096,
               temperature=1.0, top_p=0.95, top_k=20))
```

For an OpenAI-compatible server, run `python -m mlx_vlm.server --model bonsai2-4b-mlx` and send the sampling settings in each request.

The released `mlx-lm` (0.32.0) cannot load this pack yet; its support is merged and arrives with the next mlx-lm release. On older mlx-vlm versions, use the loader bundled in `runtime/` (`pip install -r bonsai2-4b-mlx/runtime/requirements.txt`, then `python bonsai2-4b-mlx/quickstart.py "your prompt"`); `PACK-RUNTIME.md` documents it. `quickstart.py` is a smoke test: it answers with thinking off and stops at 256 tokens.

Pass no images for text-only use.

For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: **[Ternary-Bonsai-2-4B-gguf](https://huggingface.co/prism-ml/Ternary-Bonsai-2-4B-gguf)**.

## Best Practices

### Generation Parameters

We recommend these sampling settings (the same as the GGUF packs):

> - Thinking mode (default): `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.0`, `repetition_penalty=1.0`
> - Non-thinking mode: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`

**Pass the sampler explicitly.** mlx-vlm's Python `generate()` decodes greedily unless you pass these arguments. `mlx_vlm.server` takes temperature, top_p and top_k from `generation_config.json` when a request omits them, but not presence penalty or min_p, so send the full set per request. Apply penalties over the last 64 tokens where your client supports a penalty window. Avoid greedy decoding with thinking on; it can get stuck reasoning on some prompts. To turn thinking off, render the chat template with `enable_thinking=False`.

### Output Length

Give thinking mode room to finish: even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.

### System Prompt

You can use a simple system prompt such as:

```
You are a helpful assistant
```

## Speculative Decoding (optional)

`mtp/` holds this model's trained multi-token-prediction head as an mlx-vlm drafter. The main model verifies every drafted token, so the output matches plain decoding (up to float16 near-ties). It helps most on copy-heavy work such as code edits and rewrites; on general chat it is about as fast as plain decoding or slightly slower, so plain decoding stays the default. `MTP.md` has the code. It was validated with the bundled runtime on mlx-vlm 0.6.3 and has not yet been re-checked on 0.7.8.

## License

Apache 2.0, following the base model.