---
title: Ornith-1.0-35B-OptiQ-6bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Ornith-1.0-35B-OptiQ-6bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Ornith-1.0-35B-OptiQ-6bit.md"
repository: mlx-community/Ornith-1.0-35B-OptiQ-6bit
last_updated: 2026-09-15
license: mit
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - qwen3_5_moe
architectures:
  - Qwen3_5MoeForConditionalGeneration
base_model:
  - deepreinforce-ai/Ornith-1.0-35B
base_model_relation: quantized
parameters: 8.1B
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 59
stars: 1
tags:
  - mlx
  - quantized
  - mixed-precision
  - 6bit
  - 8bit
  - optiq
  - moe
  - apple-silicon
  - image-text-to-text
  - vision-language
  - qwen3.5
---

# Ornith-1.0-35B-OptiQ-6bit

> Ornith-1.0-35B-OptiQ-6bit - mlx-community 在 ModelScope 开源的模型。mlx-community/Ornith-1.0-35B-OptiQ-6bit

mlx-community/Ornith-1.0-35B-OptiQ-6bit 是 ModelScope 魔搭社区上的 8.1B 参数image-text-to-text模型，采用 mit 许可，基于 deepreinforce-ai/Ornith-1.0-35B 构建。

- **Repository**: mlx-community/Ornith-1.0-35B-OptiQ-6bit
- **License**: mit
- **Tasks**: image-text-to-text
- **Parameters**: 8.1B
- **Base model**: deepreinforce-ai/Ornith-1.0-35B
- **Tags**: mlx, quantized, mixed-precision, 6bit, 8bit, optiq, moe, apple-silicon, image-text-to-text, vision-language, qwen3.5
- **Downloads**: 59
- **Stars**: 1
- **Last updated**: 2026-09-15

Source: https://www.modelscope.cn/models/mlx-community/Ornith-1.0-35B-OptiQ-6bit

---

# mlx-community/Ornith-1.0-35B-OptiQ-6bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Try the Lab](https://mlx-optiq.com/docs/lab/) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

A 6-bit mixed-precision MLX quant of [deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B), a 35B sparse MoE built on the Qwen3.5-35B-A3B architecture. Sensitive layers are kept at 8-bit and robust ones at 4-bit.

70.2 GB of bf16 weights become **27 GB**.

**Image input works.** The vision tower is kept at bf16 in a sidecar, so this quant takes images as well as text.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 6-bit |
| Layers at 8-bit (sensitive) | 448 |
| Layers at 4-bit (robust) | 63 |
| Total quantized layers | 511 |
| Group size | 64 |
| Experts | 256 routed, 40 layers |
| Vision tower | bf16, 333 tensors, in `optiq/optiq_vision.safetensors` |
| Size on disk | 27 GB, from a 70.2 GB bf16 base |

We follow the same naming convention `llama.cpp` uses for Q6_K and similar mixed-precision quants: the "6-bit" label is the predominant precision, not the weighted average.

The base model ships no MTP head, so this quant has no speculative-decoding sidecar.

## Usage

Everything OptiQ-specific lives in an `optiq/` subfolder, so a stock `*.safetensors` glob ignores it and `mlx-lm` sees a clean language model.

### Serving

At 27 GB this is comfortable on a 36 GB Mac and fits smaller machines with SSD expert streaming, which keeps attention, the router and the embeddings resident and reads the routed experts from disk as the router picks them. `optiq serve` turns it on by itself when the model would not fit in RAM; `--stream-experts` forces it.

```bash
pip install mlx-optiq
optiq serve --model mlx-community/Ornith-1.0-35B-OptiQ-6bit --stream-experts
```

That gives you an OpenAI-compatible endpoint that accepts image content parts, with mixed-precision KV cache, tool-call healing and prompt caching.

### Text

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Ornith-1.0-35B-OptiQ-6bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain the difference between TCP and UDP."}],
    add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```

This is a reasoning model: it thinks before answering, so give it enough `max_tokens` to finish.

### Images

Image input needs [`mlx-optiq`](https://mlx-optiq.com/), which loads the bf16 vision sidecar and feeds the merged embeddings to the quantized language tower. On a large MoE, serve it and send image content parts:

```python
import base64, json, urllib.request

b64 = base64.b64encode(open("photo.jpg", "rb").read()).decode()
body = {"model": "x", "max_tokens": 512, "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + b64}},
    {"type": "text", "text": "What is in this image?"}]}]}
req = urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"])
```

## Verification

Text, arithmetic reasoning, and image understanding were all exercised on the finished artifact before release.

No task benchmarks were run on this quant; for measured quality numbers on the base architecture, see the [Qwen3.5-35B-A3B OptiQ card](https://huggingface.co/mlx-community/Qwen3.5-35B-A3B-OptiQ-4bit).

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as [the original](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B).