---
title: Agnes-3.0-Flash-oQ4e
canonical_url: "https://www.modelscope.cn/models/mlx-community/Agnes-3.0-Flash-oQ4e"
md_url: "https://www.modelscope.cn/models/mlx-community/Agnes-3.0-Flash-oQ4e.md"
repository: mlx-community/Agnes-3.0-Flash-oQ4e
last_updated: 2026-09-23
model_type:
  - qwen3_5
architectures:
  - Qwen3_5ForConditionalGeneration
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 0
stars: 0
---

# Agnes-3.0-Flash-oQ4e

> Agnes-3.0-Flash-oQ4e - mlx-community 在 ModelScope 开源的模型。Agnes-3.0-Flash-oQ4e (Mixed-Precision Quantized)

- **Repository**: mlx-community/Agnes-3.0-Flash-oQ4e
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-09-23

Source: https://www.modelscope.cn/models/mlx-community/Agnes-3.0-Flash-oQ4e

---

# Agnes-3.0-Flash-oQ4e (Mixed-Precision Quantized)

Agnes-3.0-Flash converted to `qwen3_5` format and quantized with oMLX's enhanced oQ4e streaming mode.

## Model Details

| Property | Value |
|----------|-------|
| **Base model** | [Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash) |
| **Conversion** | mlx_lm with custom Agnes→qwen3_5 adapter |
| **Quantization** | oQ4e (mixed-precision affine, group_size=64) |
| **Total size** | ~19 GB |
| **Architecture** | 72 layers, 5120 hidden, 19456 intermediate |

## Bit Distribution

| Bit Width | Layers |
|-----------|--------|
| **4-bit** (base) | 428 |
| **5-bit** | 128 |
| **6-bit** | 1 |
| **8-bit** | 1 (embed_tokens) |

## Usage

```python
from mlx_lm import load, generate

model, tokenizer = load("hermitdave/Agnes-3.0-Flash-oQ4e")

response = generate(
    model,
    tokenizer,
    prompt="The capital of France is",
    max_tokens=128
)
print(response)
```

Or with oMLX:
```bash
# Place in ~/.omlx/models/hermitdave/Agnes-3.0-Flash-oQ4e/
# Restart oMLX → model appears in dropdown
```

## Sensitivity Analysis

The `oq_imatrix_report.json` in this repo contains 558 per-layer sensitivity entries collected with `oqe_code_multilingual` (128 samples, seq_length=512). Sensitivity was measured on a uniform 4-bit proxy to determine per-layer bit assignments.

## Speculative Decoding

This quant does not include MTP weights (oQ normalizes `mtp_num_hidden_layers` to 0).

### ⚠️ The shipped MTP drafter does not work with oMLX

The standalone [Agnes MTP drafter](https://huggingface.co/hermitdave/Agnes-3.0-Flash-MTP-drafter) **cannot be used as an oMLX DFlash drafter**. Its `config.json` nests the model fields under `text_config` (qwen3_5_mtp convention), so oMLX's `DFlashDraftModelArgs` construction fails:

```
DFlash start failed: DFlashDraftModelArgs.__init__() missing 11 required
positional arguments: 'hidden_size', 'num_hidden_layers', ...
```

oMLX then **silently falls back to plain batched decoding** — no error is surfaced in the UI, and you get no speedup. (Verified on oMLX, M3 Max 64 GB, Sep 2026.)

### ✅ Use Qwen3.8-27B-DFlash2 as the DFlash drafter instead

[z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2) is a drop-in DFlash drafter for this quant: same tokenizer (vocab 248,320), same hidden size (5,120), same qwen3_5-family architecture. Download it via the oMLX model browser (lands in `~/.omlx/models/z-lab/Qwen3.8-27B-DFlash2`), then enable DFlash in the model's settings, or via the admin API:

```bash
curl -X PUT http://127.0.0.1:8000/admin/api/models/<model_id>/settings \
    -H "Content-Type: application/json" \
    -H "X-Api-Key: <your-api-key>" \
    -d '{
        "dflash_enabled": true,
        "dflash_draft_model": "~/.omlx/models/z-lab/Qwen3.8-27B-DFlash2"
    }'
```

TurboQuant KV (8-bit) can stay enabled alongside DFlash.

**Measured** (M3 Max 64 GB, oMLX, temp 0, 500-token code generations):

| Config | tok/s |
|---|---|
| Plain batched (what the broken drafter silently gives you) | 16.5–17.2 |
| DFlash + Qwen3.8-27B-DFlash2 | **24.3–24.6 (+45%)** |

To verify DFlash actually engaged, check `~/.omlx/logs/server.log` for `DFlashEngine loaded` — if you see `DFlash start failed ... fallback from DFlash` instead, the drafter pairing is broken and you are running plain batched.

Note: oMLX's DFlash engine takes a slower memory-safe prefill path on long prompts (≈16K+ tokens). For long-context workloads, plain batched decoding or a model with native MTP may be preferable.

## Converion Notes

- **One-centered RMSNorm**: `mlx_weight = hf_weight + 1.0`
- **Parallel FFN folding**: `intermediate_size: 19456` (main + parallel concatenated)
- **Attention renaming**: `delta_attn` → `linear_attn`, `global_attn` → `self_attn`
- **Vision**: Not included (Agnes vision tower dimensions incompatible with Qwen3-VL)
- **MTP**: Extracted separately for drafter use

## Attribution

- Original model: [Agnes-AI](https://huggingface.co/Agnes-AI)
- Converted with [Hermes Agent](https://hermes-agent.nousresearch.com) + [mlx_lm](https://github.com/ml-explore/mlx-lm)
- Quantized with [oMLX](https://omlx.ai)