---
title: gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test.md"
repository: AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test
last_updated: 2026-06-09
license: apache-2.0
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 22
stars: 0
tags:
  - gguf
---

# gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test

> gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test - AI-ModelScope 在 ModelScope 开源的模型。[!NOTE] AI Disclosure This file was generated by an AI Agent, but tested and verified by a human.

AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test
- **License**: apache-2.0
- **Tags**: gguf
- **Downloads**: 22
- **Stars**: 0
- **Last updated**: 2026-06-09

Source: https://www.modelscope.cn/models/AI-ModelScope/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test

---

> [!NOTE] **AI Disclosure**  
> This file was generated by an AI Agent, but tested and verified by a human.

Drafter/Assistant: https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-unquantized-assistant/tree/main

llama.cpp support: https://github.com/ggml-org/llama.cpp/pull/24282

Asked Hermes to convert the Gemma 4 E2B Assistant safetensors to GGUF.

![image](https://cdn-uploads.huggingface.co/production/uploads/683b74909c3fe5951bce2e37/T9uDcH9nPK46caZ57GQ12.png)

![image](https://cdn-uploads.huggingface.co/production/uploads/683b74909c3fe5951bce2e37/9K0EntkN7RfLO6vfdBOa7.png)

Tested with llama.cpp
```ini
[HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive]
alias = HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
model = /root/.cache/llama.cpp/HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive-Q4_K_P.gguf
mmproj = /root/.cache/llama.cpp/HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive/mmproj-Gemma-4-E2B-Uncensored-HauhauCS-Aggressive-f16.gguf
spec-draft-model = /root/.cache/llama.cpp/mtp/gemma-4-E2B-it-qat-assistant-q4_0.gguf
temperature = 1.0
top-k = 64
top-p = 0.95
spec-type = draft-mtp,ngram-mod,ngram-map-k4v
spec-draft-n-max = 3
spec-draft-p-min = 0.50
spec-draft-prio = 2
spec-draft-prio-batch = 2
spec-ngram-mod-n-match = 24
spec-ngram-mod-n-min = 48
spec-ngram-mod-n-max = 64
spec-ngram-map-k4v-size-n = 8
spec-ngram-map-k4v-size-m = 24
spec-ngram-map-k4v-min-hits = 2

```

![image](https://cdn-uploads.huggingface.co/production/uploads/683b74909c3fe5951bce2e37/OjEJHMqnwJHpznZsqfF_X.png)

Now getting peak 76 tokens/second with Gemma 4 E2B + MTP (baseline was around ~40 t/s)