---
title: gemma-4-31B-it-qat-assistant-mxfp4
canonical_url: "https://www.modelscope.cn/models/mlx-community/gemma-4-31B-it-qat-assistant-mxfp4"
md_url: "https://www.modelscope.cn/models/mlx-community/gemma-4-31B-it-qat-assistant-mxfp4.md"
repository: mlx-community/gemma-4-31B-it-qat-assistant-mxfp4
last_updated: 2026-06-06
license: gemma
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - gemma4_assistant
architectures:
  - Gemma4AssistantForCausalLM
base_model:
  - google/gemma-4-31B-it-qat-q4_0-unquantized-assistant
base_model_relation: quantized
parameters: 88.0M
tensor_type:
  - U32
  - BF16
  - U8
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 9
stars: 0
tags:
  - mlx
  - mlx-vlm
  - gemma4_assistant
  - gemma-4
  - gemma-4-31b
  - gemma
  - mtp
  - speculative-decoding
  - draft-model
  - mxfp4
---

# gemma-4-31B-it-qat-assistant-mxfp4

> gemma-4-31B-it-qat-assistant-mxfp4 - mlx-community 在 ModelScope 开源的模型。gemma-4-31B-it-qat-assistant-mxfp4

mlx-community/gemma-4-31B-it-qat-assistant-mxfp4 是 ModelScope 魔搭社区上的 88.0M 参数text-generation模型，采用 gemma 许可，基于 google/gemma-4-31B-it-qat-q4_0-unquantized-assistant 构建。

- **Repository**: mlx-community/gemma-4-31B-it-qat-assistant-mxfp4
- **License**: gemma
- **Tasks**: text-generation
- **Parameters**: 88.0M
- **Base model**: google/gemma-4-31B-it-qat-q4_0-unquantized-assistant
- **Tags**: mlx, mlx-vlm, gemma4_assistant, gemma-4, gemma-4-31b, gemma, mtp, speculative-decoding, draft-model, mxfp4
- **Downloads**: 9
- **Stars**: 0
- **Last updated**: 2026-06-06

Source: https://www.modelscope.cn/models/mlx-community/gemma-4-31B-it-qat-assistant-mxfp4

---

# gemma-4-31B-it-qat-assistant-mxfp4

This repository contains Multi-Token Prediction (MTP) drafter weights split from `google/gemma-4-31B-it-qat-q4_0-unquantized-assistant` for use with `mlx-vlm` speculative decoding.

This is not a standalone chat or text-generation model. Load it as the draft model alongside a compatible Gemma 4 31B target checkpoint.

## Use with mlx-vlm

```bash
uv run mlx_vlm.generate \
  --model google/gemma-4-31B-it \
  --draft-model mlx-community/gemma-4-31B-it-qat-assistant-mxfp4 \
  --draft-kind mtp \
  --prompt "Describe this image." \
  --max-tokens 256
```

For local weights:

```bash
uv run mlx_vlm.generate \
  --model /path/to/target-model \
  --draft-model /path/to/gemma-4-31B-mtp \
  --draft-kind mtp \
  --prompt "Describe this image." \
  --max-tokens 256
```

## Model Details

- Model type: `gemma4_assistant`
- Target architecture: Gemma 4 31B
- Precision: mxfp4
- Runtime: MLX / `mlx-vlm`
- Format: Safetensors with MLX-compatible config and tokenizer files

The stored tensors are mxfp4 MLX-compatible drafter weights.

## Intended Use

Use this repo only as a speculative decoding drafter for compatible Gemma 4 31B checkpoints. The target model verifies drafted tokens, while this MTP model proposes candidate tokens per decoding step.

## Limitations

This checkpoint requires runtime support for Gemma 4 MTP draft models in `mlx-vlm`. Standard standalone generation through generic Transformers APIs is not expected to work with this repository by itself.

Please refer to the upstream `google/gemma-4-31B-it` model card and license terms for model usage constraints.