---
title: NeoHorse-1-4B-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/NeoHorse-1-4B-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/NeoHorse-1-4B-4bit.md"
repository: mlx-community/NeoHorse-1-4B-4bit
last_updated: 2026-09-14
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3_5
architectures:
  - Qwen3_5ForCausalLM
base_model:
  - TokenRhythm/NeoHorse-1-4B
base_model_relation: quantized
parameters: 658.0M
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 9
stars: 1
tags:
  - mlx
  - 4-bit
  - apple-silicon
  - agentic
  - tool-use
  - coding
  - reasoning
  - instruction-following
---

# NeoHorse-1-4B-4bit

> NeoHorse-1-4B-4bit - mlx-community 在 ModelScope 开源的模型。NeoHorse-1-4B — MLX 4-bit

mlx-community/NeoHorse-1-4B-4bit 是 ModelScope 魔搭社区上的 658.0M 参数text-generation模型，采用 apache-2.0 许可，基于 TokenRhythm/NeoHorse-1-4B 构建。

- **Repository**: mlx-community/NeoHorse-1-4B-4bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 658.0M
- **Base model**: TokenRhythm/NeoHorse-1-4B
- **Tags**: mlx, 4-bit, apple-silicon, agentic, tool-use, coding, reasoning, instruction-following
- **Downloads**: 9
- **Stars**: 1
- **Last updated**: 2026-09-14

Source: https://www.modelscope.cn/models/mlx-community/NeoHorse-1-4B-4bit

---

# NeoHorse-1-4B — MLX 4-bit

A 4-bit MLX port of [NeoHorse-1-4B](https://huggingface.co/TokenRhythm/NeoHorse-1-4B) by TokenRhythm, quantized for Apple Silicon.

This is the same fine-tuned model — just repacked as a uniform 4-bit safetensor so it runs on MLX. Vision weights are not included; this is text-only, matching the upstream release.

## What changed from the upstream model

| Property | Original (BF16) | This release |
|---|---|---|
| Precision | bfloat16 | 4-bit GPTQ (uniform) |
| Format | HuggingFace transformers | MLX safetensors |
| Size | ~8 GB | ~2.2 GB |
| Context | 262,144 | 262,144 |

The tensor values are unchanged. The only difference is the quantization and the format swap.

## Architecture

NeoHorse-1-4B is a hybrid-attention model post-trained from Qwen3.5-4B:

- **32 layers**, mostly linear attention with full-attention layers every 4th position (4 total: layers 3, 7, 11, 15, 19, 23, 27, 31)
- **GQA**: 16 query heads, 4 KV heads
- **Partial RoPE** (0.25 rotary factor) — only a quarter of dimensions get positional encoding
- **262K context** natively

The hybrid attention means the KV cache grows slowly — only the full-attention layers add per-token state, so long contexts stay cheap.

## Validation

Ran a quick sanity check on an M3 Max 64GB:

```
Prompt: "Explain the theory of relativity in two sentences."
Generation: 128 tokens, 108.6 tokens/sec
Peak memory: 2.48 GB
```

The output was coherent chain-of-thought about special and general relativity. Quantization held up.

## Quick start

Requires `mlx-lm`:

```bash
pip install mlx-lm
```

Generate from Python:

```python
from mlx_lm import load, generate

model, tokenizer = load("hermitdave/NeoHorse-1-4B-4bit")

response = generate(
    model,
    tokenizer,
    prompt="Write a Python function that returns the first n Fibonacci numbers.",
    max_tokens=512,
)
print(response)
```

You'll also need to patch `mlx-lm`'s model remapping if it doesn't already include `qwen3_5_text`. In `mlx_lm/utils.py`, add:

```python
MODEL_REMAPPING = {
    ...
    "qwen3_5_text": "qwen3_5",
}
```

(If you're on `mlx-lm >= 0.31.3` with the patch already applied, this works out of the box.)

## Performance notes

| Chip | Tokens/sec | Peak memory (4B-4bit) |
|---|---|---|
| M3 Max | 108 | ~2.5 GB |
| M2/M3 | ~90 | ~2.5 GB |

The model fits comfortably in 8 GB RAM, but 16 GB+ is recommended for headroom with longer contexts.

## Benchmarks

These are from the [upstream model](https://huggingface.co/TokenRhythm/NeoHorse-1-4B) — the 4-bit quant preserves the same weights, so accuracy should match.

**Ten-benchmark average:** 64.87 vs 58.94 for Qwen3.5-4B (**+5.93**)

| Benchmark | NeoHorse-1-4B | Qwen3.5-4B | Δ |
|---|---|---|---|
| QwenClawBench | **44.68** | 38.47 | +6.21 |
| WorkBuddy Bench | **34.41** | 24.62 | +9.79 |
| PinchBench | **77.33** | 71.19 | +6.14 |
| VitaBench | 32.00 | 21.50 | +10.50 |
| BFCL v4 | 61.79 | 61.02 | +0.77 |
| tau2-Bench | **88.46** | 84.29 | +4.17 |
| HumanEval | 96.95 | 87.20 | +9.75 |
| LiveCodeBench v6 | 59.43 | 53.71 | +5.72 |
| IFBench | 65.33 | 60.33 | +5.00 |
| IFEval | 88.35 | 87.06 | +1.29 |

*Reported protocol: SGLang, temperature=1.0, top_p=0.95, thinking mode enabled.*

## Credits

- **TokenRhythm** — for NeoHorse-1-4B and the routing-harness post-training
- **Qwen / Alibaba Cloud** — for the Qwen3.5-4B base model
- **Hermes Agent** (by Nous Research) — for the MLX conversion pipeline and validation workflow

## License

Apache 2.0. The upstream model copyright (© 2026 Alibaba Cloud) is retained in the original release. Tensor values are unchanged by this repackaging.

## Citation

```bibtex
@misc{neohorse2026,
  title  = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
  author = {NeoHorse Team},
  year   = {2026},
  eprint = {2609.08183},
  url    = {https://arxiv.org/abs/2609.08183}
}
```