---
title: Hy3-oQ2
canonical_url: "https://www.modelscope.cn/models/mlx-community/Hy3-oQ2"
md_url: "https://www.modelscope.cn/models/mlx-community/Hy3-oQ2.md"
repository: mlx-community/Hy3-oQ2
last_updated: 2026-07-16
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - hy_v3
architectures:
  - HYV3ForCausalLM
base_model:
  - tencent/Hy3
base_model_relation: quantized
parameters: 29.3B
tensor_type:
  - BF16
  - U32
  - F32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 36
stars: 1
tags:
  - mlx
  - oq
  - quantized
  - moe
  - hunyuan
---

# Hy3-oQ2

> Hy3-oQ2 - mlx-community 在 ModelScope 开源的模型。Calibrated 2-bit MLX quantization of tencent/Hy3 (Hunyuan 3.0, 295B-A21B MoE), produced with omlx oQ at level 2 — 2.7 bits/weight effective, 92 GB on disk. Data-driven mixed precision: bits are allocated per-tensor…

mlx-community/Hy3-oQ2 是 ModelScope 魔搭社区上的 29.3B 参数text-generation模型，采用 apache-2.0 许可，基于 tencent/Hy3 构建。

- **Repository**: mlx-community/Hy3-oQ2
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 29.3B
- **Base model**: tencent/Hy3
- **Tags**: mlx, oq, quantized, moe, hunyuan
- **Downloads**: 36
- **Stars**: 1
- **Last updated**: 2026-07-16

Source: https://www.modelscope.cn/models/mlx-community/Hy3-oQ2

---

# Hy3 (MLX, oQ2)

Calibrated 2-bit MLX quantization of [tencent/Hy3](https://huggingface.co/tencent/Hy3) (Hunyuan 3.0, 295B-A21B MoE), produced with **omlx** oQ at level 2 — **2.7 bits/weight effective**, 92 GB on disk. Data-driven mixed precision: bits are allocated per-tensor from a measured sensitivity map, not a fixed rule. For Apple Silicon.

## Requirements

mlx-lm doesn't support the `hy_v3` architecture yet — there's an open PR: [mlx-lm#1211](https://github.com/ml-explore/mlx-lm/pull/1211). Until it lands, install mlx-lm from the PR branch, otherwise the model won't load:

```bash
uv pip install "mlx-lm @ git+https://github.com/kernelpool/mlx-lm.git@add-hy3-preview"
```

Once #1211 lands in a released mlx-lm, this should load as-is. If something breaks after that, open a discussion here asking for a re-upload.

## How it was quantized

Hy3 is 556 GB in BF16 — larger than RAM on a 128 GB machine — so oQ can't hold the full model to measure layer sensitivity, and omlx's automatic 4-bit sensitivity proxy trips the Metal command-buffer watchdog (`GPU Timeout`) on this hardware: the 4-bit proxy (150 GB) overflows the default Metal working-set cap.

The path that worked, in two steps:

1. **Intermediate `mixed_2_6`** — I first made a uniform heuristic mixed quant via `mlx_lm.convert` (2-bit base, 6-bit on a fixed structural set of layers). It's coherent but heavy (3.15 bpw, 108 GB, peaks 116 GB — needs a raised Metal cap) and uneven on harder prompts.
2. **oQ2 calibrated** — then I quantized to oQ2, passing the `mixed_2_6` as `sensitivity_model_path`, so oQ skips the failing auto-proxy, measures sensitivity on it (fits in memory, runs as inference — no watchdog), then streams the final quant tensor-by-tensor. It's **more resilient than `mixed_2_6` at less memory**.

## Memory

Peak **99 GB**, which fits under the **default** Metal working-set cap (107.5 GB on a 128 GB machine) — no `sysctl iogpu.wired_limit_mb` bump needed, unlike `mixed_2_6` (116 GB peak). Headroom for the KV cache is tight at long context, so enabling **TurboQuant KV is highly recommended** if possible — I got good performance with it at 3-bit.

## Conversion check

Smoke-tested after conversion (`mlx_lm.generate`): coherent — solved `17 * 24 = 408` with correct step-by-step reasoning, no repetition loop (plain 2-bit and 2-bit+8-bit-router uniform quants both collapsed here; the calibrated oQ2 does not). On a **Macbook Pro M5 Max 128GB 40 GPU**: 33.6 tok/s generation, 5.05 tok/s prompt, peak 99 GB (short prompt).

## Benchmarks (all Hy3 MLX variants)

oMLX intelligence suite, 300 seeded samples per benchmark, identical questions
across models. I ran seeded samples — this is **not** a complete benchmark run,
so read the differences as noise and test the versions against your own workload
before picking one.

| Benchmark (300) | **oQ2 · 2.68 (this model)** | oQ2e · 2.43 | oQ2e-2.37bpw | oQ2e-2.33bpw | oQ2e-2.31bpw |
|---|---|---|---|---|---|
| mathqa     | 0.63 | 0.65 | 0.64 | 0.62 | 0.60 |
| mmlu_pro   | 0.65 | 0.61 | 0.60 | 0.59 | 0.55 |
| winogrande | 0.74 | 0.68 | 0.68 | 0.65 | 0.65 |

Variants: [oQ2](https://huggingface.co/mlx-community/Hy3-oQ2) ·
[oQ2e](https://huggingface.co/mlx-community/Hy3-oQ2e) ·
[oQ2e-2.37bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.37bpw) ·
[oQ2e-2.33bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.33bpw) ·
[oQ2e-2.31bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.31bpw)

## Usage

```bash
python -m mlx_lm generate --model mlx-community/Hy3-oQ2 --prompt "Explain Bayes' theorem in two sentences." --max-tokens 300
```

```python
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Hy3-oQ2")
```

## License

[Apache-2.0](https://huggingface.co/tencent/Hy3/blob/main/LICENSE), inherited from the base model. Refer to the original model card for architecture, benchmarks, and intended use.