---
title: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B
canonical_url: "https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B"
md_url: "https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B.md"
repository: mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B
last_updated: 2026-08-29
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - nemotron_h
architectures:
  - NemotronHForCausalLM
base_model:
  - mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit
base_model_relation: quantized
parameters: 3.3B
tensor_type:
  - F32
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 29
stars: 0
tags:
  - mlx
  - quantized
  - expert-pruning
  - reap
  - moe
  - optiq
  - apple-silicon
  - text-generation
---

# NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B

> NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B - mlx-community 在 ModelScope 开源的模型。mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B

mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B 是 ModelScope 魔搭社区上的 3.3B 参数text-generation模型，采用 apache-2.0 许可，基于 mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit 构建。

- **Repository**: mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 3.3B
- **Base model**: mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit
- **Tags**: mlx, quantized, expert-pruning, reap, moe, optiq, apple-silicon, text-generation
- **Downloads**: 29
- **Stars**: 0
- **Last updated**: 2026-08-29

Source: https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B

---

# mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, prune, fine-tune, and serve LLMs locally on Apple Silicon. [All OptiQ models](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

**12.1 GB instead of 21.2 GB. 13.7 GB of memory to run.**

| | Parent | This model | |
|---|---|---|---|
| On disk | 21.2 GB | **12.1 GB** | −43% |
| Parameters | 31.1B | **16.9B** | −46% |



50% of the routed experts are removed from [mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit](https://huggingface.co/mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit); active parameters per token are unchanged, since top-6 routing is preserved and only the stored expert bank shrinks. That is why it gets smaller without getting slower.

Retained experts are copied bit-for-bit from the parent quant. Nothing is dequantized, re-quantized, merged, or retrained.

**A Capability Score for this model is not published yet.** It shares its architecture, expert count and routing with [NVIDIA-Nemotron-3-Nano-30B-A3B-OptiQ-4bit-REAP-17B](https://huggingface.co/mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-OptiQ-4bit-REAP-17B), which is being benchmarked now; this card will be updated when that lands. A number borrowed from one of the Qwen or Gemma variants would not transfer — Nemotron-H is a hybrid Mamba/attention stack whose experts are a SwitchMLP rather than a SwitchGLU.

What has been checked here: the ranking rule was chosen by scoring both candidates against this checkpoint's own unpruned output (KL 0.262 for the frequency-weighted rule against 0.297 for the conditional mean), and generation was compared with the unpruned parent on factual, code and arithmetic prompts.

## Details

| Property | Value |
|---|---|
| Experts retained | 64 of 128 per layer |
| Active experts per token | 6 (unchanged) |
| Allocation | uniform (64 of 128 in every MoE layer) |
| Size | 12.1 GB (parent 21.2 GB) |
| Parameters | 16.9B (parent 31.1B) |
| Selection | REAP — router weight × expert output norm, weighted by selection frequency (this router renormalizes its weights, so frequency is what carries preference) |
| Calibration | optiq six-domain mix, 8 samples |
| MTP sidecar | preserved |

## Use it

```bash
pip install mlx-optiq
optiq serve --model mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B
```

```python
from mlx_lm import load, generate
model, tok = load("mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit-REAP-17B")
print(generate(model, tok, prompt="Hello", max_tokens=64))
```

## Method

Expert pruning follows **REAP** (Cerebras Research, ICLR 2026 — [*REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression*](https://arxiv.org/abs/2510.13999)). Experts are ranked by the conditional mean of router weight × expert output norm over calibration data; the lowest-ranked are removed and the router is sliced to match.

OptiQ applies it **in the quantized domain** — directly on a quantized checkpoint, with no BF16 parent and no dequantization of survivors — via `optiq prune-experts`. See the [pruning docs](https://mlx-optiq.com/docs/prune).