---
title: NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit"
md_url: "https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit.md"
repository: mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit
last_updated: 2026-09-16
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - nemotron_h
architectures:
  - NemotronHForCausalLM
base_model:
  - nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
base_model_relation: quantized
parameters: 13.4B
tensor_type:
  - F32
  - BF16
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - fr
  - es
  - it
  - de
  - ja
  - zh
downloads: 58
stars: 0
tags:
  - nvidia
  - nemotron-3
  - mlx
  - quantized
  - 2bit
  - mixed-precision
  - optiq
  - static
  - moe
  - ssd-streaming
  - apple-silicon
  - text-generation
---

# NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit

> NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit - mlx-community 在 ModelScope 开源的模型。mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit

mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit 是 ModelScope 魔搭社区上的 13.4B 参数text-generation模型，采用 other 许可，基于 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 构建。

- **Repository**: mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 13.4B
- **Base model**: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
- **Tags**: nvidia, nemotron-3, mlx, quantized, 2bit, mixed-precision, optiq, static, moe, ssd-streaming, apple-silicon, text-generation
- **Downloads**: 58
- **Stars**: 0
- **Last updated**: 2026-09-16

Source: https://www.modelscope.cn/models/mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit

---

# mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Read the write-up](https://mlx-optiq.com/blog/stream-122b-on-a-mac) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

**A 120-billion-parameter model that runs on a 36 GB Mac.** This is a 2-bit mixed-precision MLX quant of NVIDIA's Nemotron-3-Super-120B-A12B (247 GB at bf16), produced by [mlx-optiq](https://mlx-optiq.com/). It is 47.5 GB on disk. While it generates, only ~14 GB sits in RAM: the Mamba blocks, attention, router and shared experts stay resident, and the 34 GB of routed mixture-of-experts weights stream off the SSD as the router selects them, through `optiq serve --stream-experts`.

Nemotron-3-Super is a hybrid: Mamba2 state-space blocks interleaved with attention and a 512-expert sparse MoE (22 active per token). Asked to write Flappy Bird in a single HTML file, the 2-bit model produced a complete, working game. Here it is playing it:

![Nemotron-3-Super-120B-A12B 2-bit playing the Flappy Bird it wrote, on a 36 GB Mac](flappy.gif)

## What it is

| Property | Value |
|---|---|
| Base | NVIDIA-Nemotron-3-Super-120B-A12B (hybrid Mamba2 + attention + 512-expert MoE, 22 active) |
| Method | OptiQ `static` — structural per-layer bit allocation, no calibration |
| Bit-widths | 4-bit on Mamba / attention / router / shared experts / edges, 2-bit on the routed experts |
| Achieved bits-per-weight | 2.52 |
| On disk | 47.5 GB |
| Resident while running | ~14 GB (routed experts streamed) |
| Decode speed | ~3 tok/s on an M3 Max (36 GB) |

For a model this large, exact calibration-driven sensitivity is impractical (it would run for days and needs the full model resident as a reference), so OptiQ's `static` method assigns bits from architecture alone. See the [methods comparison](https://mlx-optiq.com/docs/sensitivity).

## Run it

This is a Nemotron hybrid (`model_type: nemotron_h`), so it needs **mlx-lm from main** and **`import optiq`** (install from git, not a version pin):

```bash
pip install -U mlx-optiq "mlx-lm @ git+https://github.com/ml-explore/mlx-lm.git"
```

Serve it with SSD expert streaming (auto-enabled for a MoE too big to fit resident; `--stream-experts` forces it):

```bash
optiq serve --model mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit --stream-experts
```

Then open the Lab, ask for a game, and watch it render in the Canvas pane. Only the routed experts stream per token; the Mamba state, attention and shared experts stay resident, so the footprint stays ~14 GB no matter how large the model on disk is.

## Notes

This is an **extreme quant**. 2-bit on the routed experts is lossy, and the point of this artifact is that a 120 B hybrid MoE runs at all on consumer Apple Silicon, with coherent output. For reference quality on this base, use the bf16 weights or a higher-bit quant. The full story is in the [blog post](https://mlx-optiq.com/blog/stream-122b-on-a-mac).