---
title: Nemotron-3-Super-120B-A12B-MLX-6bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit"
md_url: "https://www.modelscope.cn/models/mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit.md"
repository: mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit
last_updated: 2026-03-31
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - nemotron_h
architectures:
  - NemotronHForCausalLM
base_model:
  - nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
base_model_relation: quantized
parameters: 26.5B
tensor_type:
  - U32
  - BF16
  - F32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
inference_backends:
  - "deploy_task text/emb"
  - "vllm 0.9.2"
downloads: 17
stars: 0
tags:
  - mlx
  - safetensors
  - nemotron_h
  - nemotron
  - mamba
  - mamba2
  - mixture-of-experts
  - 6bit
  - quantized
  - apple-silicon
  - text-generation
  - conversational
  - reasoning
  - lm-studio
  - custom_code
---

# Nemotron-3-Super-120B-A12B-MLX-6bit

> Nemotron-3-Super-120B-A12B-MLX-6bit - mlx-community 在 ModelScope 开源的模型。Nemotron-3-Super-120B-A12B — MLX 6-bit

mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit 是 ModelScope 魔搭社区上的 26.5B 参数text-generation模型，采用 other 许可，基于 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 构建，可用 deploy_task text/emb、vllm 0.9.2 部署。

- **Repository**: mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 26.5B
- **Base model**: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
- **Inference backends**: deploy_task text/emb, vllm 0.9.2
- **Tags**: mlx, safetensors, nemotron_h, nemotron, mamba, mamba2, mixture-of-experts, 6bit, quantized, apple-silicon, text-generation, conversational, reasoning, lm-studio, custom_code
- **Downloads**: 17
- **Stars**: 0
- **Last updated**: 2026-03-31

Source: https://www.modelscope.cn/models/mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit

---

# Nemotron-3-Super-120B-A12B — MLX 6-bit

MLX quantization of [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) for Apple Silicon.

## Key Specs

| Detail | Value |
|---|---|
| Architecture | Hybrid Mamba-2 + Transformer Attention + Latent MoE |
| Total Parameters | 120B |
| Active Parameters | 12B per token |
| Context Length | 1M tokens (262,144 default) |
| Experts | 512 routed, 22 active per token, 1 shared |
| Quantization | 6-bit affine (6.507 BPW), group size 64 |
| Disk Size | ~92 GB |
| Peak Memory | ~98.4 GB |

## Requirements

- Apple Silicon Mac with **128GB+ unified memory**
- `mlx-lm >= 0.31.2` (install from git main for Latent MoE support)

```bash
pip install git+https://github.com/ml-explore/mlx-lm.git
```

## Usage

### CLI

```bash
mlx_lm.generate \
  --model FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit \
  --prompt "Hello!" \
  --max-tokens 256
```

### Python

```python
from mlx_lm import load, generate

model, tokenizer = load("FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
```

### LM Studio

This model is compatible with [LM Studio](https://lmstudio.ai) on Apple Silicon. Search for `FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit` in the model browser and download directly.

## Performance

Tested on M5 Pro Max (128GB):

| Metric | Value |
|---|---|
| Generation Speed | ~43.6 tok/s |
| Peak Memory | 98.4 GB |

## About the Architecture

Nemotron-H is a hybrid architecture combining three components:

- **Mamba-2 layers** — efficient state-space model for long-context processing
- **Transformer attention layers** — standard multi-head attention (GQA, 32 heads, 2 KV heads)
- **Latent MoE** — 512 experts with latent routing, 22 active per token, plus 1 shared expert

The layer pattern alternates between Mamba (M) and attention with MoE (E) blocks across 88 layers. This hybrid design achieves strong performance with only 12B active parameters per token despite having 120B total.

## Reasoning Model

This is a reasoning model that outputs chain-of-thought before the final answer. The model uses `<think>` and `</think>` tags to delineate reasoning.

## License

[NVIDIA Open Model License](https://developer.nvidia.com/open-model-license)

## Credits

- Base model by [NVIDIA](https://huggingface.co/nvidia)
- MLX quantization by [FF-01](https://huggingface.co/FF-01)