---
title: Hy3-OptiQ-2bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Hy3-OptiQ-2bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Hy3-OptiQ-2bit.md"
repository: mlx-community/Hy3-OptiQ-2bit
last_updated: 2026-09-16
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - hy_v3
architectures:
  - HYV3ForCausalLM
base_model:
  - tencent/Hy3
base_model_relation: quantized
parameters: 24.4B
tensor_type:
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 84
stars: 0
tags:
  - mlx
  - optiq
  - quantized
  - 2bit
  - mixed-precision
  - moe
  - ssd-streaming
  - apple-silicon
  - text-generation
---

# Hy3-OptiQ-2bit

> Hy3-OptiQ-2bit - mlx-community 在 ModelScope 开源的模型。mlx-community/Hy3-OptiQ-2bit

mlx-community/Hy3-OptiQ-2bit 是 ModelScope 魔搭社区上的 24.4B 参数text-generation模型，采用 apache-2.0 许可，基于 tencent/Hy3 构建。

- **Repository**: mlx-community/Hy3-OptiQ-2bit
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 24.4B
- **Base model**: tencent/Hy3
- **Tags**: mlx, optiq, quantized, 2bit, mixed-precision, moe, ssd-streaming, apple-silicon, text-generation
- **Downloads**: 84
- **Stars**: 0
- **Last updated**: 2026-09-16

Source: https://www.modelscope.cn/models/mlx-community/Hy3-OptiQ-2bit

---

# mlx-community/Hy3-OptiQ-2bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

**A 295-billion-parameter model that runs in 7.6 GB of RAM on a Mac.** This is a 2-bit mixed-precision MLX quant of [tencent/Hy3](https://huggingface.co/tencent/Hy3), produced by [mlx-optiq](https://mlx-optiq.com/). It is 88 GB on disk, and while the model generates only about 7.6 GB sits in RAM: attention, the router, the shared expert and the embeddings stay resident, and the 192 routed experts are read off the SSD as the router picks them.

## What it is

| Property | Value |
|---|---|
| Base | [tencent/Hy3](https://huggingface.co/tencent/Hy3) (sparse MoE, 192 experts, 8 active per token, 80 layers) |
| Parameters | 295 B |
| Bit-widths | 2-bit routed experts; 6-bit attention; 8-bit shared expert, embeddings and LM head |
| Achieved bits-per-weight | 2.39 |
| On disk | 88 GB |
| Resident while running | ~7.6 GB (routed experts streamed) |

No Capability Score is published for this quant. Running the six-benchmark suite against a model that decodes off SSD would take days, and at 2 bits on the routed experts the point of the artifact is different: that a 295 B MoE runs at all on consumer Apple Silicon.

## Run it

Hy3 is not an architecture stock mlx-lm knows, so `import optiq` once to register it:

```bash
pip install "mlx-optiq>=0.4.15"
```

The routed experts are far too large to sit resident, so serve it with SSD expert streaming. `optiq serve` turns this on by itself for a MoE quant that would not fit in RAM (`--stream-experts` forces it):

```bash
optiq serve --model mlx-community/Hy3-OptiQ-2bit
```

That gives you an OpenAI and Anthropic compatible endpoint with mixed-precision KV cache, tool-call healing and prompt caching. Only the routed experts stream per token, so the footprint stays near 7.6 GB regardless of how large the model on disk is.

A fast SSD matters more than RAM here. Every token reads 8 experts per layer from disk, so decode speed tracks read throughput.

## Notes

This is an **extreme quant.** Two bits on the routed experts is lossy, and anything where accuracy matters should use the bf16 weights or a higher-bit quant. What this one demonstrates is a model of this size running on a Mac, at a resident footprint that fits a 16 GB machine.

## Links

- **Project website:** [mlx-optiq.com](https://mlx-optiq.com/)
- **All OptiQ quants:** [mlx-optiq.com/models](https://mlx-optiq.com/models)
- **PyPI:** [pypi.org/project/mlx-optiq](https://pypi.org/project/mlx-optiq/)
- **Base model:** [tencent/Hy3](https://huggingface.co/tencent/Hy3)