---
title: Qwen3.8-14B-Instruct-Turbo-GGUF
canonical_url: "https://www.modelscope.cn/models/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF"
md_url: "https://www.modelscope.cn/models/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF.md"
repository: ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF
last_updated: 2026-08-31
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - ewinregirgojr/Qwen3.8-14B-Instruct-Turbo
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 1170
stars: 4
tags:
  - gguf
  - llama.cpp
  - qwen
  - qwen3
  - 14b
  - deltanet
  - linear-attention
  - hybrid-attention
  - distillation
  - pruned
  - reasoning
  - tool-calling
  - agent
  - coding
---

# Qwen3.8-14B-Instruct-Turbo-GGUF

> Qwen3.8-14B-Instruct-Turbo-GGUF - ewinregirgojr 在 ModelScope 开源的模型。STABLE VERSION (https://huggingface.co/ewinregirgojr/Qwen3.8-19B-Pruned)

ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 ewinregirgojr/Qwen3.8-14B-Instruct-Turbo 构建。

- **Repository**: ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: ewinregirgojr/Qwen3.8-14B-Instruct-Turbo
- **Tags**: gguf, llama.cpp, qwen, qwen3, 14b, deltanet, linear-attention, hybrid-attention, distillation, pruned, reasoning, tool-calling, agent, coding
- **Downloads**: 1170
- **Stars**: 4
- **Last updated**: 2026-08-31

Source: https://www.modelscope.cn/models/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF

---

# STABLE VERSION (https://huggingface.co/ewinregirgojr/Qwen3.8-19B-Pruned)

# Qwen3.8-14B-Instruct-Turbo-GGUF

Official GGUF quantizations of [**Qwen3.8-14B-Instruct-Turbo**](https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo), a 14.2B parameter pruned and distilled hybrid reasoning and agent model derived from Qwen3.8-27B.

This repository provides high-performance quantized `.gguf` binaries optimized for local inference across CPU, Apple Silicon (Metal), and NVIDIA CUDA hardware via **`llama.cpp`**, **Ollama**, **LM Studio**, and **Text-Generation-WebUI**.

---

## 📦 Provided GGUF Files

| Filename | Quant Method | File Size | Recommended Hardware / VRAM | Use Case |
| :--- | :---: | :---: | :--- | :--- |
| [**`Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf`**](https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF/blob/main/Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf) | `Q4_K_M` | **9.16 GB** | **8GB - 12GB VRAM** / 16GB RAM | **Recommended default.** Optimal balance of speed, low memory, and high generation quality. |
| [**`Qwen3.8-14B-Instruct-Turbo-Q5_K_M.gguf`**](https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF/blob/main/Qwen3.8-14B-Instruct-Turbo-Q5_K_M.gguf) | `Q5_K_M` | **10.60 GB** | **12GB - 16GB VRAM** / 24GB RAM | High-precision quantization for complex coding and chained multi-step reasoning. |
| [**`Qwen3.8-14B-Instruct-Turbo-Q8_0.gguf`**](https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF/blob/main/Qwen3.8-14B-Instruct-Turbo-Q8_0.gguf) | `Q8_0` | **14.58 GB** | **16GB - 24GB VRAM** / 32GB RAM | Near-lossless master precision matching FP16 quality. |

---

## 📊 Empirical Benchmark Evaluation Results

The model was evaluated using greedy decoding on NVIDIA T4 GPU hardware across official standardized benchmark splits:

| Benchmark Capability | Evaluation Dataset / Metric | Evaluated Count | **Qwen3.8-14B-Turbo** (Empirical) | **Qwen3.8-27B** (Baseline) | Evaluation Protocol |
| :--- | :--- | :---: | :---: | :---: | :--- |
| **Scientific Reasoning** | **GPQA Diamond** | 50 Questions | **70.0%** (35 / 50) | 89.2% | Exact-match multiple choice on expert graduate-level science problems. |
| **Agent / Function Calling** | **BFCL / XLAM Tool Use** | 50 Test Cases | **22.0%** (11 / 50) | 67.1% | Strict AST argument parsing and function dispatch validation. |
| **Coding & Program Synthesis** | **HumanEval / LiveCode** | 6 Evaluation Sets | **7.56 – 7.60 tok/s** | ~5.5 tok/s (FP16) | Measured greedy decoding throughput on NVIDIA T4 GPU. |
| **Knowledge Distillation** | **Multi-Turn Curricula** | 200 Iterations | **-82.58% Loss Drop** | N/A | Loss dropped from `4.7109` to `0.8207` ($r=32, lpha=64$). |

---

## 🚀 Quickstart & Usage

### 1. Using with `llama.cpp` CLI

Run inference directly on your GPU using `llama-cli`:

```bash
# Run with Q4_K_M offloaded to CUDA GPU
llama-cli -m Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 0.2 \
  -p "<|im_start|>system\nYou are an expert assistant.<|im_end|>\n<|im_start|>user\nWrite a Python function for prefix trie search.<|im_end|>\n<|im_start|>assistant\n<think>\n"
```

### 2. Using with `llama-server` (OpenAI-Compatible API)

Start a local OpenAI-compatible API server on port 8080:

```bash
llama-server -m Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --port 8080 \
  --host 0.0.0.0
```

You can now connect any OpenAI-compatible client (e.g. LangChain, AutoGen, Open-WebUI) to `http://localhost:8080/v1`.

### 3. Using with Ollama

Create a `Modelfile`:

```dockerfile
FROM ./Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
<think>
{{ .Response }}"""
PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"
```

Then build and run:

```bash
ollama create qwen3.8-14b-turbo -f Modelfile
ollama run qwen3.8-14b-turbo
```

---

## 📜 Citation & Attribution

```bibtex
@misc{qwen3.8_14b_instruct_turbo_gguf_2026,
  author = {ewinregirgojr},
  title = {Qwen3.8-14B-Instruct-Turbo-GGUF: Optimized 14.2B Hybrid Linear-Attention Reasoning Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF}}
}
```