---
title: ComfyUI-Qwen3-VL-GGUF
canonical_url: "https://www.modelscope.cn/models/Abiray/ComfyUI-Qwen3-VL-GGUF"
md_url: "https://www.modelscope.cn/models/Abiray/ComfyUI-Qwen3-VL-GGUF.md"
repository: Abiray/ComfyUI-Qwen3-VL-GGUF
last_updated: 2026-10-28
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
base_model:
  - Comfy-Org/Qwen3-VL
  - Qwen/Qwen3-VL-8B-Instruct
  - Qwen/Qwen3-VL-4B-Instruct
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 69
stars: 1
tags:
  - comfyui
  - comfyui-gguf-loader
  - vision
  - multimodal
  - gguf
  - llama.cpp
  - qwen3-vl
---

# ComfyUI-Qwen3-VL-GGUF

> ComfyUI-Qwen3-VL-GGUF - Abiray 在 ModelScope 开源的模型。ComfyUI Qwen3-VL GGUF (8B & 4B) — Unified Hybrid & Split mmproj Editions

Abiray/ComfyUI-Qwen3-VL-GGUF 是 ModelScope 魔搭社区上的image-text-to-text模型，采用 apache-2.0 许可，基于 Comfy-Org/Qwen3-VL、Qwen/Qwen3-VL-8B-Instruct、Qwen/Qwen3-VL-4B-Instruct 构建。

- **Repository**: Abiray/ComfyUI-Qwen3-VL-GGUF
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Base model**: Comfy-Org/Qwen3-VL, Qwen/Qwen3-VL-8B-Instruct, Qwen/Qwen3-VL-4B-Instruct
- **Tags**: comfyui, comfyui-gguf-loader, vision, multimodal, gguf, llama.cpp, qwen3-vl
- **Downloads**: 69
- **Stars**: 1
- **Last updated**: 2026-10-28

Source: https://www.modelscope.cn/models/Abiray/ComfyUI-Qwen3-VL-GGUF

---

# ComfyUI Qwen3-VL GGUF (8B & 4B) — Unified Hybrid & Split `mmproj` Editions

GGUF quantizations of **Qwen3-VL** extracted from [Comfy-Org's Qwen3-VL repackaged weights](https://huggingface.co/Comfy-Org/Qwen3-VL).

This repository provides **two distinct formats** depending on your runtime:
1. **🔥 NEW: Unified Single-File Hybrid GGUFs (`*_vision_q8_0.gguf`)** — Combines the K-quantized LLM backbone (`Q4_K_M` / `Q5_K_M`) and the `Q8_0` Vision Tower + DeepStack Mergers + 5D `Conv3d` shape metadata (`comfy.gguf.orig_shape`) inside a **single file**. 
2. **Split LLM + Standalone `mmproj` GGUFs** — Standard two-file format for **llama.cpp**, **llama-server**, **Ollama**, and split VLM loaders.

---

## Model Overview

- **Original Weights Source:** [Comfy-Org/Qwen3-VL](https://huggingface.co/Comfy-Org/Qwen3-VL)
- **Underlying Architecture:** Alibaba Cloud Qwen3-VL (`Qwen3VLForConditionalGeneration` / `qwen3vl`)
- **Hybrid Format Vision Precision:** `Q8_0` Vision Tower & DeepStack Mergers + `F32` LayerNorms/Biases + preserved 5D `[1152, 3, 2, 16, 16]` patch embedding metadata
- **Split Format Projector Precision:** Standalone `F16` GGUF (`mmproj`) preserving full vision patch fidelity
- **Quantization Engine:** `llama.cpp` k-quants + custom ComfyUI-GGUF-Loader hybrid tensor packer

---

## Available Files & Specifications

### 1. Unified Single-File Hybrid Series (Recommended for ComfyUI)
*Contains both the Language Backbone (`blk.*`) and Vision Tower (`model.visual.*`) in a single GGUF file.*

| File Name | LLM Precision | Vision Precision | File Size | Description |
|---|---|---|---|---|
| `qwen3vl_8b_q4_k_m_vision_q8_0.gguf` | Q4_K_M | Q8_0 / F32 | 5.62 GB | **All-in-one hybrid GGUF.** Best speed/VRAM balance for `ComfyUI-GGUF-Loader` (no external `mmproj` needed). |
| `qwen3vl_8b_q5_k_m_vision_q8_0.gguf` | Q5_K_M | Q8_0 / F32 | ~6.39 GB | **All-in-one hybrid GGUF.** Higher-precision text backbone paired with `Q8_0` vision & DeepStack mergers. |

### 2. Split 8B Series (LLM + Standalone `mmproj`)
*For `llama.cpp`, `llama-server`, or split-loader workflows.*

| File Name | Precision | File Size | Description |
|---|---|---|---|
| `mmproj-qwen3vl-8b-f16.gguf` | F16 | 1.08 GB | Standalone multimodal vision projector (pair with split 8B files below) |
| `qwen3vl-8b-Q4_K_M.gguf` | Q4_K_M | 4.68 GB | Optimal balance of generation speed, memory, and accuracy |
| `qwen3vl-8b-Q5_K_M.gguf` | Q5_K_M | 5.45 GB | Balanced higher-precision text backbone |
| `qwen3vl-8b-Q6_K.gguf` | Q6_K | 6.26 GB | Near-lossless language reasoning and instruction following |

### 3. Split 4B Series (LLM + Standalone `mmproj`)

| File Name | Precision | File Size | Description |
|---|---|---|---|
| `mmproj-qwen3vl-4b-f16.gguf` | F16 | 0.78 GB | Standalone 4B Multimodal vision projector |
| `qwen3vl-4b-Q4_K_M.gguf` | Q4_K_M | 2.33 GB | Ultra-lightweight edge quant for low-memory environments |
| `qwen3vl-4b-Q5_K_M.gguf` | Q5_K_M | 2.69 GB | Higher-precision 4B variant |

---

## Quickstart & Usage

### Option A: Unified Single-File Setup 

1. Download either `qwen3vl_8b_q4_k_m_vision_q8_0.gguf` or `qwen3vl_8b_q5_k_m_vision_q8_0.gguf` and place it into:
   ```text
   ComfyUI/models/text_encoders/
   # or
   ComfyUI/models/clip/
   ```
2. Load it directly inside ComfyUI using the **`CLIPLoader (GGUF)`** node 

### Option B: Split Two-File Setup (`llama.cpp` / Split ComfyUI Loaders)

1. Place your desired language GGUF (e.g., `qwen3vl-8b-Q4_K_M.gguf`) into:
   ```text
   ComfyUI/models/unet/
   # or
   ComfyUI/models/text_encoders/
   ```
2. Place the matching projector (e.g., `mmproj-qwen3vl-8b-f16.gguf`) into:
   ```text
   ComfyUI/models/clip_vision/
   ```
3. Load the model using your split GGUF loader or via `llama.cpp` CLI/Server:
   ```bash
   llama-server -m qwen3vl-8b-Q4_K_M.gguf --mmproj mmproj-qwen3vl-8b-f16.gguf
   ```

---

## Technical Verification (Hybrid Series)

Both `qwen3vl_8b_q4_k_m_vision_q8_0.gguf` and `qwen3vl_8b_q5_k_m_vision_q8_0.gguf` have been validated end-to-end:
- **750 Total Tensors:** 399 LLM tensors (`Q4_K_M` or `Q5_K_M`) + 351 Vision tensors (`Q8_0` weights + `F32` norms/biases).
- **5D Patch Embedding Metadata:** Includes `comfy.gguf.orig_shape.model.visual.patch_embed.proj.weight = [1152, 3, 2, 16, 16]` for 3D temporal-spatial patch projection.
- **DeepStack Mergers:** All 3 DeepStack spatial mergers (`[4096, 4608]`) and final vision-to-LLM merger verified with zero `NaN`/`Inf` values during multimodal forward passes.

---

## Acknowledgements

- [Alibaba Cloud Qwen Team](https://github.com/QwenLM) for the foundational Qwen3-VL models.
- [Comfy-Org](https://huggingface.co/Comfy-Org) for packaging the unified BF16 safetensors weights.
- [Georgi Gerganov and the GGML/llama.cpp contributors](https://github.com/ggml-org/llama.cpp) for GGUF architecture and quantization tooling.