---
title: Swift-Qwen3.8-27B-GGUF
canonical_url: "https://www.modelscope.cn/models/ukisai/Swift-Qwen3.8-27B-GGUF"
md_url: "https://www.modelscope.cn/models/ukisai/Swift-Qwen3.8-27B-GGUF.md"
repository: ukisai/Swift-Qwen3.8-27B-GGUF
last_updated: 2026-09-17
license: other
pipeline_tag: image-text-to-text
tasks:
- image-text-to-text
base_model:
- ukisai/Swift-Qwen3.8-27b
base_model_relation: quantized
library_name:
- gguf
- pytorch
frameworks:
- pytorch
downloads: 536
stars: 2
tags:
- gguf
- llama.cpp
- qwen3_8
- qwen3_5
- efficient-thinking
- reasoning
- token-efficient
---
# Swift-Qwen3.8-27B-GGUF
> Swift-Qwen3.8-27B-GGUF - ukisai 在 ModelScope 开源的模型。Website • Learn more • BF16 model • Enterprise licensing
ukisai/Swift-Qwen3.8-27B-GGUF 是 ModelScope 魔搭社区上的image-text-to-text模型，采用 other 许可，基于 ukisai/Swift-Qwen3.8-27b 构建。
- **Repository**: ukisai/Swift-Qwen3.8-27B-GGUF
- **License**: other
- **Tasks**: image-text-to-text
- **Base model**: ukisai/Swift-Qwen3.8-27b
- **Tags**: gguf, llama.cpp, qwen3_8, qwen3_5, efficient-thinking, reasoning, token-efficient
- **Downloads**: 536
- **Stars**: 2
- **Last updated**: 2026-09-17
Source: https://www.modelscope.cn/models/ukisai/Swift-Qwen3.8-27B-GGUF
---
# Swift-Qwen3.8-27B GGUF
Swift-Qwen3.8-27B is UkisAI's reasoning-efficient derivative of Qwen3.8-27B,
using **58.3% fewer thinking tokens** while maintaining near-identical performance
(**<1% loss**) and as a result getting a **x1.95 speed-up** on several tasks.
The prompt is a sample from LiveCodeBench v6
## Evaluation scope
> All results below compare the Qwen3.8-27B BF16 base with the same base plus the
> Swift adapter.
## Benchmarks
| Benchmark |
Score |
Mean tokens |
Median tokens |
| Base |
Swift |
Base |
Swift |
Reduction |
Reduction |
| General reasoning |
| GPQA-Diamond | 88.38% | 88.28% | 15,014 | 8,855 | ↓ 41.0% | ↓ 58.3% |
| MMLU-Pro | 85.47% | 84.95% | 2,980 | 1,603 | ↓ 46.2% | ↓ 28.3% |
| C-Eval | 90.00% | 90.62% | 1,492 | 804 | ↓ 46.1% | ↓ 19.3% |
| IFBench | 73.53% | 71.80% | 8,052 | 4,657 | ↓ 42.2% | ↓ 50.5% |
| Mathematics |
| AIME 2026 | 98.67% | 94.00% | 22,014 | 16,143 | ↓ 26.7% | ↓ 50.2% |
| HMMT (Nov 2025) | 99.33% | 96.00% | 22,032 | 15,189 | ↓ 31.1% | ↓ 45.9% |
| Multimodal |
| ERQA | 67.45% | 66.30% | 4,137 | 2,045 | ↓ 50.6% | ↓ 54.6% |
| Agentic coding |
| Terminal-Bench 2.1 | 66.74% | 65.84% | 37,086 | 27,272 | ↓ 26.5% | ↓ 38.7% |
| LiveCodeBench v6 | 76.76% | 81.55% | 11,374 | 8,615 | ↓ 24.3% | ↓ 45.8% |
How to reproduce
Serving: BF16 · vLLM 0.27.1 · Qwen3 parser · context 262,144 · thinking xhigh.
Sampling: temperature 1.0 · top_p 0.95 · top_k 20 · min_p 0 · presence_penalty 0 · repetition_penalty 1.
Benchmarks: averages over five seeds (0–4) per model; five trials per task for Terminal-Bench.
| Benchmark | Output cap |
|---|
| GPQA-Diamond | 100,000 |
| MMLU-Pro | 100,000 |
| C-Eval | 16,384 |
| IFBench | 81,920 |
| AIME 2026 | 250,000 |
| HMMT Nov 2025 | 250,000 |
| ERQA | 100,000 |
| Terminal-Bench 2.1 | Agent/task limits |
| LiveCodeBench v6 | 32,768 |
## Quantized evaluations
Quantized deployment is the intended use for Swift: lower-memory weights paired with
shorter reasoning. The INT4 evaluations below come from the
[source model card](https://huggingface.co/ukisai/Swift-Qwen3.8-27b) and were run on
W4A16 and AWQ checkpoints, not on this F16 GGUF. They retain token savings across GPQA,
IFBench, and AIME. On AIME, Swift matches or improves accuracy and reduces output-cap
failures by **31–33%**.
| Benchmark / quantization |
Base accuracy |
Swift accuracy |
Mean token reduction |
Median token reduction |
GPQA-Diamond
Mixed-precision quant W4A16 · thinking tokens | 88.69% | 88.38% | ↓ 32.1% | ↓ 50.2% |
IFBench
Mixed-precision quant W4A16 · completion tokens | 72.58% | 71.25% | ↓ 30.1% | ↓ 38.0% |
AIME 2026
Mixed-precision quant W4A16 · completion tokens | 84.00% | 84.00% | ↓ 19.0% | ↓ 37.5% |
AIME 2026
AWQ INT4 · completion tokens | 82.67% | 84.00% | ↓ 22.8% | ↓ 34.8% |
Quantized evaluation settings
Each row compares the same quantized base with and without the Swift adapter.
GPQA and AIME use five seeds; IFBench uses four samples per prompt and strict scoring.
Output caps: GPQA 100,000; IFBench 81,920; AIME 32,768. GPQA and IFBench use saved
historical base runs. AIME uses template-default effort and counts truncated answers
as incorrect. Its shorter cap makes it a separate comparison from the BF16 table.
## GGUF quantizations
| File |
Size |
KLD wikitext @512 |
KLD wikitext @32k |
KLD held-out @32k |
Top-p @32k |
| Q8_0 | 29.1 GB | 0.0009 | 0.0035 | 0.0579 | 97.92% |
Q6_K_L
new tier | 25.2 GB | 0.0015 | — | — | 98.24% |
Q6_K_S
new tier | 23.1 GB | 0.0018 | — | — | 98.09% |
| Q6_K | 22.9 GB | 0.0020 | 0.0069 | 0.0782 | 96.85% |
| Q5_K_M | 20.2 GB | 0.0056 | 0.0135 | 0.1251 | 95.60% |
Q5_K_S
new tier | 19.8 GB | 0.0057 | — | — | 96.70% |
Q4_K_L
new tier | 19.0 GB | 0.0102 | — | — | 95.77% |
| Q4_K_M | 18.0 GB | 0.0120 | 0.0211 | 0.1496 | 94.30% |
IQ4_NL
new tier | 17.6 GB | 0.0141 | — | — | 95.11% |
Q4_1
new tier | 17.5 GB | 0.0194 | — | — | 94.09% |
Q4_K_S
new tier | 16.6 GB | 0.0150 | — | — | 94.95% |
Q4_0
new tier | 16.0 GB | 0.0278 | — | — | 92.59% |
IQ4_XS
new tier | 15.7 GB | 0.0165 | — | — | 94.66% |
IQ3_M
new tier | 15.1 GB | 0.0390 | — | — | 91.76% |
Q3_K_L
new tier | 14.3 GB | 0.0412 | — | — | 91.26% |
Q3_K_M
new tier | 13.6 GB | 0.0552 | — | — | 89.91% |
IQ3_XS
new tier | 13.0 GB | 0.0555 | — | — | 89.91% |
Q3_K_S
new tier | 12.9 GB | 0.0631 | — | — | 89.30% |
IQ3_XXS
new tier | 12.5 GB | 0.0724 | — | — | 88.81% |
Q2_K
new tier | 11.0 GB | 0.1617 | — | — | 84.05% |
IQ2_M
new tier | 10.7 GB | 0.1469 | — | — | 84.47% |
IQ2_S
new tier | 9.9 GB | 0.2060 | — | — | 81.29% |
IQ2_XS
new tier | 9.3 GB | 0.2354 | — | — | 80.04% |
IQ2_XXS
new tier | 9.1 GB | 0.2852 | — | — | 78.09% |
Mean KL divergence against the BF16 source, lower is better. Tiers marked *new tier* were added on 2026-09-13 and carry the wikitext @512 measurement and 512-token top-token agreement; their 32k columns will be filled as those runs complete. `wikitext` is wikitext-2 test; `held-out`
is our own chat and long-document set, reserved before the importance matrix was fitted. `Top-p` is
top-token agreement with BF16 at 32k on the held-out set.
**Read the two 32k columns together.** On this hybrid architecture (48 of 64 blocks are recurrent), a
small fraction of positions (about 0.1%) diverge sharply at long context for every tier, including `Q8_0`,
and the same is true of the public `Q4_K_M` and `Q8_0` builds of the base Qwen3.8-27B measured on the same
harness. Those rare positions dominate the held-out mean; the median divergence at 32k is within 10% of the
512-token value for every tier. Typical-token quality does not degrade with context. The pick below follows
the 99th-percentile tail on the held-out set: 2.60 for `Q4_K_M`, 1.75 for `Q5_K_M`, 0.46 for `Q6_K`,
0.23 for `Q8_0`.
| Use case |
Pick |
| 24 GB cards, everyday use | Q4_K_M |
| Long agentic runs, strict tool-call formatting | Q6_K or higher |
| Maximum fidelity | Q8_0 |
Recipe
All tiers use the same importance matrix (8,016 chunks of domain, prompt and long-document text) and pin
the recurrent gate projections `ssm_alpha` and `ssm_beta` to F32 and the MTP head to Q8_0. `Q4_K_M`
additionally lifts `ssm_out`, `attn_gate`, `output` and `token_embd` to Q6_K; `Q5_K_M` and `Q6_K` lift
`attn_gate` to Q8_0. The lifts cost about 1.1 GB on `Q4_K_M` and reduce its KL divergence by roughly 20%
against a plain llama.cpp `Q4_K_M` of the same model.
The tiers added on 2026-09-13 (`IQ2_XXS` through `Q6_K_L`) use the same importance matrix and the same
`ssm_alpha`/`ssm_beta` F32 and MTP Q8_0 pins, with per-tensor type layouts computed for this model by
[bartowski's quantization-config](https://github.com/bartowski1182/quantization-config) instead of
llama.cpp's built-in heuristic (`--tensor-type-file`). `Q4_K_L`, `Q6_K_S` and `Q6_K_L` are the large and
small layouts of `Q4_K_M` and `Q6_K`. All files were built with llama.cpp release b10896 from a BF16
conversion of the published safetensors and checked against BF16 on the harness above.
### KV cache
Only 16 of the 64 blocks are full attention, so the cache stays small for a 27B:
```
16 layers x 4 kv-heads x 256 head_dim x 2 (K+V) x 2 bytes = 64 KiB per token
```
| Context |
KV cache |
| 8k | 0.5 GB |
| 32k | 2.0 GB |
| 64k | 4.0 GB |
| 128k | 8.0 GB |
## Training approach
We built Swift by identifying reasoning-marker tokens that, in our analysis, trigger overthinking in Qwen’s
reasoning rollouts. We then fine-tuned Qwen by penalizing usage of those tokens while it reasons.
Swift produces shorter reasoning traces. In our testing, we also observe fewer overthinking errors.
For maximum gains, Swift also includes a transfer component derived from
[BottleCap AI's ThinkingCap-Qwen3.6-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B).
## How to use
### llama.cpp
These files run with [llama.cpp](https://github.com/ggml-org/llama.cpp), installable in one line via [llama.app](https://llama.app/).
The command below is the llama.cpp counterpart of the vLLM configuration on the source card: full
262,144 context, thinking on at reasoning effort `xhigh`, reasoning and tool calls parsed from the
embedded chat template, and Qwen3.8's thinking-mode sampling.
```
curl -LsSf https://llama.app/install.sh | sh
llama-server -hf ukisai/Swift-Qwen3.8-27B-GGUF:Q4_K_M \
--jinja -fa on -ngl 99 \
-c 262144 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 \
--presence-penalty 0 --repeat-penalty 1.0 \
--port 8000
```
llama-server exposes an OpenAI-compatible API and a built-in chat web UI on the port above.
Swap `Q4_K_M` for any tier in the table above (`IQ2_XXS` up to `Q8_0`) or `F16`; `-hf` fetches the tier and
the vision projector automatically. The KV cache costs 64 KiB per token (16 GB at the full
262,144 context), so lower `-c` if it does not fit. Use a recent llama.cpp release with
Qwen3.5/Qwen3.8 architecture support. The same sampling values are stored in the GGUF header
and `xhigh` is the template default; the flags above make the configuration explicit.
They also work in [LM Studio](https://lmstudio.ai/), [koboldcpp](https://github.com/LostRuins/koboldcpp)
and [Jan AI](https://www.jan.ai/). In those apps set the same sampling values by hand and a
context length of at least 65,536 tokens; the default 4,096-token window overflows on long
reasoning and looks like an endless loop.
#### Multimodal
This model supports image input. Alongside the quants, this repo includes the multimodal
projector file [mmproj-Swift-Qwen3.8-27B-F16.gguf](https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF/blob/main/mmproj-Swift-Qwen3.8-27B-F16.gguf),
which pairs with any tier above. llama.cpp downloads the mmproj automatically when using
`-hf` as shown above; if you are loading files manually, pass it with `--mmproj`.
#### MTP
This model has MTP (Multi-Token Prediction) layers, and they are included in every tier,
stored at Q8_0. MTP layers act as a built-in draft model, letting llama.cpp run speculative
decoding for faster generation. To use them, add the following flag to your llama.cpp command:
```
--spec-type draft-mtp --spec-draft-n-max 3
```
This is the counterpart of the vLLM `--speculative-config '{"method":"mtp","num_speculative_tokens":3}'` option.
### Ollama
```
ollama create swift -f <(curl -fsSL https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF/resolve/main/Modelfile) && ollama run swift
```
Use Ollama 0.33 or newer. The [`Modelfile`](https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF/blob/main/Modelfile)
in this repo pulls the `Q4_K_M` tier together with the vision projector, applies the sampling
values above, and sets Ollama's built-in Qwen3.8 renderer and parser, which separate reasoning
from the answer and parse tool calls. Swift's embedded chat template is identical to Qwen3.8's.
For another tier, download the Modelfile, change the tag after `FROM`, and run
`ollama create swift -f Modelfile`.
`ollama run hf.co/ukisai/Swift-Qwen3.8-27B-GGUF:Q4_K_M` also works without a Modelfile. It
runs the embedded chat template through llama.cpp and reads the sampling values from the
[`params`](https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF/blob/main/params) file.
Hugging Face cannot set Ollama's renderer and parser, so in this mode reasoning may appear
inline with the answer.
Ollama sizes the context window from VRAM: 262,144 tokens with 48 GB or more, 32,768 with
24 GB, and 4,096 below that, which is too short for long reasoning. To raise it, start the
server with `OLLAMA_CONTEXT_LENGTH=65536 ollama serve` or run `/set parameter num_ctx 65536`
in the chat. To use the MTP layers, run `/set parameter draft_num_predict 3`.
### UkisAI API
If you would rather not run the weights yourself, Swift is served through an
OpenAI-compatible API at `https://ukisai.com/api/swift/v1`. It is **free for research
purposes** and needs no API key. The model id is `swift`.
```bash
curl https://ukisai.com/api/swift/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "swift", "messages": [{"role": "user", "content": "Hello, Swift."}]}'
```
## Validation
The converted files passed a finite-tensor check and a CPU text-generation smoke test.
Multimodal generation and the full benchmark suite have **not** been re-evaluated on
this GGUF release. The results above and on the
[source model card](https://huggingface.co/ukisai/Swift-Qwen3.8-27b) come from the BF16
and INT4 checkpoints named there, not from these files.
## License and access
Swift weights are distributed through gated access under the **Swift Open License v1.0**.
Personal, research, educational, evaluation, and commercial use are free for individuals
and organizations with annual recurring revenue, including affiliates, of up to
US$1,000,000. Above that threshold, commercial use requires a separate **Swift Enterprise
License**. Contact [UkisAI](https://ukisai.com/contact) for terms.
## Citation
```bibtex
@misc{swift-qwen3.8-27b,
title = {Swift-Qwen3.8-27B},
author = {UkisAI},
year = {2026},
url = {https://huggingface.co/ukisai/Swift-Qwen3.8-27b}
}
```
