---
title: Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF
canonical_url: "https://www.modelscope.cn/models/RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF"
md_url: "https://www.modelscope.cn/models/RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF.md"
repository: RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF
chinese_name: "Qwen3.8-27B OrcaRouter GSQ/RCO IQ3_XXS GGUF 量化"
last_updated: 2026-09-15
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen3_5
architectures:
  - Qwen3_5ForConditionalGeneration
base_model:
  - orcarouter/Qwen3.8-27B
base_model_relation: quantized
library_name:
  - gguf
language:
  - zh
  - en
downloads: 179
stars: 2
tags:
  - qwen3
  - qwen3.8
  - orcarouter
  - gsq
  - rco
  - gguf
  - quantized
  - speculative-decoding
  - mtp
  - text-generation
  - long-context
  - tool-use
  - 16gb-vram
  - image-text-to-text
  - vision
  - multimodal
  - mmproj
---

# Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF

> Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF - RentedNoodle 在 ModelScope 开源的模型。Qwen3.8-27B GGUF (OrcaRouter instruction-tuned base), IQ3_XXS GSQ/RCO quant + native MTP head

RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 orcarouter/Qwen3.8-27B 构建。

- **Repository**: RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: orcarouter/Qwen3.8-27B
- **Tags**: qwen3, qwen3.8, orcarouter, gsq, rco, gguf, quantized, speculative-decoding, mtp, text-generation, long-context, tool-use, 16gb-vram, image-text-to-text, vision, multimodal, mmproj
- **Downloads**: 179
- **Stars**: 2
- **Last updated**: 2026-09-15

Source: https://www.modelscope.cn/models/RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF

---

# Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF

本仓库提供两个 **Qwen3.8-27B GGUF 量化文件**（GSQ/RCO 混合精度量化，IQ3_XXS trunk + Q6_K MTP 头），面向单张 16 GB 显存显卡运行。

**基座模型**：`orcarouter/Qwen3.8-27B`（instruction-tuned variant）。本仓库是量化版本，不是新的微调。底层模型为 Qwen3.8-27B；指令微调变体来自 OrcaRouter；量化工作遵循 ISTA-DASLab 的 GSQ/RCO 方法。

## 概览

| | |
|---|---|
| 架构 | Qwen3.8（GDN hybrid） |
| 基座 | `orcarouter/Qwen3.8-27B`（instruction-tuned variant） |
| 上下文 | 原生架构 262K；发布配置在 32K 运行 |
| Trunk 量化 | IQ3_XXS，~3.06 bpw |
| MTP 头 | Q6_K（原生 MTP 头，S1 训练 draft 头）；用于推测解码 |
| 对话模板 | `froggeric-qwen3.8-tool-use.jinja` |
| 许可证 | Apache-2.0 |

原生 MTP（Multi-Token Prediction）头用于推测解码：配合 llama.cpp 的 `--spec-type draft-mtp` 可提升服务吞吐。

## 文件

| 文件 | 大小 | SHA-256（前缀） | bpw | 适用区间 |
|---|---|---|---|---|
| `Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf` | 10,466,439,424 字节 (9.75 GiB) | `41ad7dfb` | ~3.06 | ≤128K |
| `Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0-qatfa.gguf` | 10,442,846,464 字节 (9.73 GiB) | `7e181f30` | 3.0579 | >128K |
| `mmproj/mmproj-Qwen3.8-27B-BF16.gguf` | 931,146,528 字节 (0.87 GiB) | `13cb7beb` | — | 视觉输入 |
| `mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf` | 629,247,648 字节 (0.59 GiB) | `b280c2cb` | — | 视觉输入 |
| `REF-IQ3_XXS-mtp.rco-allocation.txt` | 24,583 字节 | `2e690030` | — | 量化分配表 |

完整 SHA-256 见本仓库 `SHA256SUMS.txt`。

`REF-IQ3_XXS-mtp.rco-allocation.txt` 为逐张量量化分配表（RCO allocation map），记录生成该量化文件所用的混合精度分配。

`v2.0-qatfa.gguf` 与 `v2.0.gguf` 除 MTP 草稿头区块外逐字节相同；它带有 **FA 对齐 MTP 头**（@3300）与 **ssm_alpha BF16 修复**（×48），针对 FlashAttention KV 路径校准。

## 快速开始

```bash
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

`--spec-type draft-mtp --spec-draft-n-max 2` 启用原生 MTP 推测解码。`-b 2048 -ub 2048` 为检索保真度所需（更小的 batch 会切分检索跨度，导致 depth-0.5 needle 丢失）。Froggeric 模板为预期工具/推理行为所需。

## 视觉输入（Vision）

本仓库包含 `mmproj` 视觉投影器，配合支持视觉的 Qwen3.8 构建可接受图像输入。两个投影器任选其一：

| 文件 | 大小 | 说明 |
|---|---|---|
| `mmproj/mmproj-Qwen3.8-27B-BF16.gguf` | 931,146,528 字节 (0.87 GiB) | BF16，保真度更高 |
| `mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf` | 629,247,648 字节 (0.59 GiB) | Q8_0，体积更小 |

图像输入用法（在文本启动命令基础上增加 `--mmproj`）：

```bash
# BF16 投影器
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

```bash
# Q8_0 投影器（体积更小）
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --mmproj mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

BF16 投影器保真度更高；Q8_0 投影器体积更小。视觉输出不保证事实准确。

## 适用区间（Regime guidance）

| 上下文 | 文件 | 关键 flags |
|---|---|---|
| ≤128K | `v2.0.gguf` | `-fa on`（快速路径） |
| >128K | `v2.0-qatfa.gguf` | `-fa on -ctk q8_0 -ctv q4_0`（FA KV 路径） |

启动命令（>128K）：

```bash
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0-qatfa.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 131072 -b 2048 -ub 2048 -fa on -ctk q8_0 -ctv q4_0 -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

## 实测结果

（RTX 5070 Ti 16 GB，llama.cpp，本仓库文件）

| 测试 | 结果 | 说明 |
|---|---|---|
| Needle 检索 | 5/6 | A/B 测试两腿均为 5/6；needle-05（depth 0.5）稳定未命中 |
| WikiText-2 困惑度 | ~6.17 | 40×512 |
| 服务吞吐中位数 | v2.0 62.21 t/s；qatfa 73.67 t/s | 同会话 A/B |
| Toolcall（14 例） | 9/14 | v2.0 卡片测量 |
| GPQA-Diamond | 75.25% | thinking ON，temp 0，198 题，harness-conditional |

**qatfa 相对 v2.0 的 A/B（build2+v2.0 对照 vs build-fa+qatfa 处理组，flags 相同）：**

| 指标 | 对照 | 处理组 | Δ |
|---|---|---|---|
| CUM 接受率（canonical） | 0.6957 | 0.6852 | **−0.0105** |
| CUM 接受率（clean-clean） | 0.6957 | 0.6761 | −0.0196 |
| 服务 t/s 中位数 | 62.21 | 73.67 | **+18.4%** |
| PPL | 6.1745 | 6.1717 | −0.045% |
| Needle 1–6 @ b2048 | 5/6 | 5/6 | parity |

如实说明：CUM 逐次运行噪声 ≈ 0.009。另有一次独立测量中 v2.0 needle 为 6/6（15K 词 haystack，build2 配置）；本次 A/B 两腿结果为 5/6。

## 局限与预期用途（Limitations & Intended Use）

- **相对 stock Qwen，本模型的拒绝过滤已减弱**，可能生成未经标准安全过滤的内容。
- 仅供研究与实验用途，不适用于生产或面向公众的部署。
- 输出不保证事实准确；用户须自行验证。视觉输出同样不保证事实准确。
- 使用者需自行负责遵守适用法律与平台政策。
- 激进 3-bit 量化——不要期待 BF16 表现；知识回忆弱于更高 bit 版本。
- MTP 输出在量化目标上可能与串行生成不同；需要逐 token 一致时关闭推测解码。

## 许可证

Apache-2.0（与基座声明保持一致）。

> 完整英文说明见下方。

---

[⬇️ 完整英文文档 / Full English Documentation Below ⬇️](#english)

<a id="english"></a>

---

# Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-GGUF

A pair of **Qwen3.8-27B GGUF quants** (mixed-precision GSQ/RCO, IQ3_XXS trunk + Q6_K MTP head), built to run on a single 16 GB GPU.

**Base model:** `orcarouter/Qwen3.8-27B` (instruction-tuned variant). This repo is a quantization, not a new fine-tune. The underlying model is Qwen3.8-27B; the instruction-tuned variant comes from OrcaRouter; the quantization work here follows the GSQ/RCO methodology from ISTA-DASLab.

## Overview

| | |
|---|---|
| Architecture | Qwen3.8 (GDN hybrid) |
| Base | `orcarouter/Qwen3.8-27B` (instruction-tuned variant) |
| Context | Native architecture 262K; ship profile runs at 32K |
| Trunk quant | IQ3_XXS, ~3.06 bpw |
| MTP head | Q6_K (native MTP head, S1-trained draft head) for speculative decoding |
| Template | `froggeric-qwen3.8-tool-use.jinja` |
| License | Apache-2.0 |

The native MTP (Multi-Token Prediction) head is used for speculative decoding: with llama.cpp's `--spec-type draft-mtp` it raises serve throughput.

## Files

| File | Size | SHA-256 (prefix) | bpw | Regime |
|---|---|---|---|---|
| `Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf` | 10,466,439,424 bytes (9.75 GiB) | `41ad7dfb` | ~3.06 | ≤128K |
| `Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0-qatfa.gguf` | 10,442,846,464 bytes (9.73 GiB) | `7e181f30` | 3.0579 | >128K |
| `mmproj/mmproj-Qwen3.8-27B-BF16.gguf` | 931,146,528 bytes (0.87 GiB) | `13cb7beb` | — | vision |
| `mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf` | 629,247,648 bytes (0.59 GiB) | `b280c2cb` | — | vision |
| `REF-IQ3_XXS-mtp.rco-allocation.txt` | 24,583 bytes | `2e690030` | — | allocation map |

Full SHA-256 hashes are in `SHA256SUMS.txt` in this repo.

`REF-IQ3_XXS-mtp.rco-allocation.txt` is the per-tensor quantization allocation map used to produce this quant.

`v2.0-qatfa.gguf` is byte-identical to `v2.0.gguf` **except the MTP draft head block**; it carries a **FA-aligned MTP head** (@3300) and the **ssm_alpha BF16 fix** (×48), calibrated for the FlashAttention KV path.

## Quickstart

```bash
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

`--spec-type draft-mtp --spec-draft-n-max 2` enables the native MTP speculative decoding. `-b 2048 -ub 2048` is required for retrieval fidelity (a smaller batch splits the retrieval span and the depth-0.5 needles are lost). The Froggeric template is required for the intended tool/reasoning behavior.

## Vision

This repo includes `mmproj` vision projectors so the quant accepts image input with a vision-capable Qwen3.8 build. Pick either projector:

| File | Size | Notes |
|---|---|---|
| `mmproj/mmproj-Qwen3.8-27B-BF16.gguf` | 931,146,528 bytes (0.87 GiB) | BF16, higher fidelity |
| `mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf` | 629,247,648 bytes (0.59 GiB) | Q8_0, smaller |

Image input (add `--mmproj` to the text command):

```bash
# BF16 projector
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

```bash
# Q8_0 projector (smaller)
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --mmproj mmproj/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

BF16 is the higher-fidelity projector; Q8_0 is smaller. Vision output is not factually guaranteed.

## Regime guidance

| Context | File | Key flags |
|---|---|---|
| ≤128K | `v2.0.gguf` | `-fa on` (fast path) |
| >128K | `v2.0-qatfa.gguf` | `-fa on -ctk q8_0 -ctv q4_0` (FA KV path) |

Serve command (>128K):

```bash
llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0-qatfa.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 131072 -b 2048 -ub 2048 -fa on -ctk q8_0 -ctv q4_0 -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
```

## Validation summary

(RTX 5070 Ti 16 GB, llama.cpp, these exact files)

| Suite | Result | Notes |
|---|---|---|
| Needle retrieval | 5/6 | both A/B legs 5/6; needle-05 (depth 0.5) consistently misses |
| WikiText-2 perplexity | ~6.17 | 40×512 |
| Serve t/s median | v2.0 62.21; qatfa 73.67 | same-session A/B |
| Toolcall (14-case) | 9/14 | v2.0 card measurement |
| GPQA-Diamond | 75.25% | thinking ON, temp 0, 198 questions, harness-conditional |

**qatfa vs v2.0 A/B (build2+v2.0 control vs build-fa+qatfa treatment, same flags):**

| Metric | Control | Treatment | Δ |
|---|---|---|---|
| CUM accept (canonical) | 0.6957 | 0.6852 | **−0.0105** |
| CUM accept (clean-clean) | 0.6957 | 0.6761 | −0.0196 |
| Serve t/s median | 62.21 | 73.67 | **+18.4%** |
| PPL | 6.1745 | 6.1717 | −0.045% |
| Needle 1–6 @ b2048 | 5/6 | 5/6 | parity |

Stated plainly: CUM run-to-run noise is ≈0.009. One separate measurement recorded needle 6/6 for v2.0 (15K-word haystack, build2 profile); on this A/B battery the result is 5/6 on both legs.

## Limitations & Intended Use

- **Reduced refusal filtering relative to stock Qwen** — the model may produce content without standard safety filtering.
- Research / experimental use only; not intended for production or public-facing deployment.
- Outputs are not factually guaranteed; verify before relying on them. Vision output is likewise not factually guaranteed.
- The user is responsible for compliance with applicable law and platform policy.
- Aggressive 3-bit quant — do not expect BF16 behavior; knowledge recall is weaker than higher-bit variants.
- MTP output may differ from serial generation on quantized targets; disable speculative decoding when exact serial behavior matters.

## License

Apache-2.0, matching the base's declared license.