---
title: Qwen3.6-27B-MTP-ONLY-GGUF
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF.md"
repository: AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF
last_updated: 2026-07-18
license: apache-2.0
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 121
stars: 0
tags:
  - gguf
---

# Qwen3.6-27B-MTP-ONLY-GGUF

> Qwen3.6-27B-MTP-ONLY-GGUF - AI-ModelScope 在 ModelScope 开源的模型。Separate MTP GGUF [!NOTE] This repo contains Multi-Token Prediction (MTP) GGUF for LLaMA.cpp extracted from the base model (Qwen/Qwen3.6-27B). It can be paired with a target model using the…

AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF
- **License**: apache-2.0
- **Tags**: gguf
- **Downloads**: 121
- **Stars**: 0
- **Last updated**: 2026-07-18

Source: https://www.modelscope.cn/models/AI-ModelScope/Qwen3.6-27B-MTP-ONLY-GGUF

---

# Separate MTP GGUF
> [!NOTE]  
> This repo contains **Multi-Token Prediction (MTP) GGUF for LLaMA.cpp** extracted from the base model ([Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)).  
> It can be paired with a target model using the `--spec-draft-model` flag.  
> See PR: https://github.com/ggml-org/llama.cpp/pull/22673  
>
> If you’re looking for an MTP GGUF for transplanting/"grafting" onto your model, check out:  
> - https://huggingface.co/havenoammo/Qwen3.6-27B-MTP-GGUF
> - https://huggingface.co/IHaveNoClueAndIMustPost/Qwen3.6-27B-MTP-TENSORS-ONLY

[am17an](https://github.com/am17an):
>yes it can be loaded separately using `--spec-draft-model`.
>The `convert_hf_to_gguf.py` changes have an option of `--mtp` which just outputs the MTP gguf.
>
>Using the "grafted" on MTP is more VRAM efficient though.
>
>Another thing is that `-hf` option will try to look for the MTP gguf like it does for `mmproj` in case `spec-draft-type draft-mtp` is mentioned.

Discussion: https://github.com/ggml-org/llama.cpp/pull/22673#issuecomment-4456979078  

# Findings

Original MTP tensors: 
- https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model-00013-of-00015.safetensors
- https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model-00015-of-00015.safetensors

Shared embeddings/output weights: 
- https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model-00001-of-00015.safetensors
- https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model-00008-of-00015.safetensors

Ref: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model.safetensors.index.json

MTP tensors to GGUF conversion: 
`python convert_hf_to_gguf.py ../Qwen3.6-27B --outtype bf16 --outfile ../Qwen3.6-27B-MTP/Qwen3.6-27B-MTP-bf16.gguf --mtp`


Conversion log: [conversion.log](https://huggingface.co/lym00/Qwen3.6-27B-MTP-ONLY-GGUF/blob/main/conversion.log)
```log
INFO:hf-to-gguf:Loading model: Qwen3.6-27B
INFO:hf-to-gguf:Model architecture: Qwen3_5ForConditionalGeneration
INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00015.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00008-of-00015.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00013-of-00015.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00015-of-00015.safetensors'
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only

INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:token_embd.weight,                    torch.bfloat16 --> BF16, shape = {5120, 248320}
INFO:hf-to-gguf:output.weight,                        torch.bfloat16 --> BF16, shape = {5120, 248320}
INFO:hf-to-gguf:blk.64.nextn.eh_proj.weight,          torch.bfloat16 --> BF16, shape = {10240, 5120}
INFO:hf-to-gguf:blk.64.ffn_down.weight,               torch.bfloat16 --> BF16, shape = {17408, 5120}
INFO:hf-to-gguf:blk.64.ffn_gate.weight,               torch.bfloat16 --> BF16, shape = {5120, 17408}
INFO:hf-to-gguf:blk.64.ffn_up.weight,                 torch.bfloat16 --> BF16, shape = {5120, 17408}
INFO:hf-to-gguf:blk.64.attn_k.weight,                 torch.bfloat16 --> BF16, shape = {5120, 1024}
INFO:hf-to-gguf:blk.64.attn_q.weight,                 torch.bfloat16 --> BF16, shape = {5120, 12288}
INFO:hf-to-gguf:blk.64.attn_v.weight,                 torch.bfloat16 --> BF16, shape = {5120, 1024}
INFO:hf-to-gguf:output_norm.weight,                   torch.bfloat16 --> F32, shape = {5120}
INFO:hf-to-gguf:blk.64.attn_norm.weight,              torch.bfloat16 --> F32, shape = {5120}
INFO:hf-to-gguf:blk.64.post_attention_norm.weight,    torch.bfloat16 --> F32, shape = {5120}
INFO:hf-to-gguf:blk.64.attn_k_norm.weight,            torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.64.attn_output.weight,            torch.bfloat16 --> BF16, shape = {6144, 5120}
INFO:hf-to-gguf:blk.64.attn_q_norm.weight,            torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.64.nextn.shared_head_norm.weight, torch.bfloat16 --> F32, shape = {5120}
INFO:hf-to-gguf:blk.64.nextn.enorm.weight,            torch.bfloat16 --> F32, shape = {5120}
INFO:hf-to-gguf:blk.64.nextn.hnorm.weight,            torch.bfloat16 --> F32, shape = {5120}

INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:..\Qwen3.6-27B-MTP\Qwen3.6-27B-MTP-bf16.gguf: n_tensors = 18, total_size = 5.9G
Writing: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 5.94G/5.94G [00:46<00:00, 127Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to ..\Qwen3.6-27B-MTP\Qwen3.6-27B-MTP-bf16.gguf
```

---

# Test Results
WebUI (llama.cpp)

![image](https://cdn-uploads.huggingface.co/production/uploads/683b74909c3fe5951bce2e37/NYCOo4LOU3EGuHz0VHSDm.png)

Harness (OpenCode)

![image](https://cdn-uploads.huggingface.co/production/uploads/683b74909c3fe5951bce2e37/f7wupaEOFui7yt_5HmEur.png)

Config (models.ini)
```ini
version = 1

[*]
flash-attn = on
mlock = off
mmap = on
fit = on
warmup = on
batch-size = 256
ubatch-size = 256
cache-type-k = q4_0
cache-type-v = q4_0
kv-unified = true
swa-full = true
jinja = true
direct-io = off
cache-prompt = true
cache-ram = 28672
n-gpu-layers = 99
reasoning = off
reasoning-budget = 0
min-p = 0
presence-penalty = 1.5
top-k = 40
chat-template-kwargs = {"preserve_thinking": true}
spec-default = true
ctx-checkpoints = 64
parallel = 1
threads-http = 1
ctx-size = 131072

# --- MODELS ---
[TeichAI/Qwen3.6-27B-Fable-5-Experimental-GGUF]
alias = TeichAI/Qwen3.6-27B-Fable-5-Experimental-GGUF
model = /root/.cache/llama.cpp/TeichAI/Qwen3.6-27B-Fable-5-Experimental-GGUF/Qwen3.6-27B-Fable-5-Distill.iq4_nl.gguf
mmproj = /root/.cache/llama.cpp/mmproj/mmproj-Qwen3.6-27B-BF16.gguf
spec-draft-model = /root/.cache/llama.cpp/mtp/Qwen3.6-27B-MTP-q4_0.gguf
temperature = 0.7
top-k = 20
top-p = 0.8
presence-penalty = 1.5
repeat-penalty = 1.0
seed = 42
spec-type = draft-mtp,ngram-mod,ngram-map-k4v
spec-draft-n-max = 3
spec-draft-p-min = 0.50
spec-draft-prio = 2
spec-draft-prio-batch = 2
spec-ngram-mod-n-match = 24
spec-ngram-mod-n-min = 48
spec-ngram-mod-n-max = 64
spec-ngram-map-k4v-size-n = 8
spec-ngram-map-k4v-size-m = 24
spec-ngram-map-k4v-min-hits = 2
```

Hardware tested (low-budget mini PC)
- Model: Machenike GTR Mini PC (~$600)
- CPU: AMD R7-H255 (780M iGPU)
- RAM: 32G DDR5 (Shared/Unified memory)
- Backend: llama.cpp (Vulkan)