--- title: Spark-X2.6-4B-LowBit-GGUF canonical_url: "https://www.modelscope.cn/models/Dalvlad/Spark-X2.6-4B-LowBit-GGUF" md_url: "https://www.modelscope.cn/models/Dalvlad/Spark-X2.6-4B-LowBit-GGUF.md" repository: Dalvlad/Spark-X2.6-4B-LowBit-GGUF last_updated: 2026-10-27 license: apache-2.0 pipeline_tag: text-generation tasks: - text-generation base_model: - XHToken/Spark-X2.6-4B base_model_relation: quantized library_name: - gguf language: - en - zh downloads: 47 stars: 0 tags: - spark - sparkx2_5 - gguf - imatrix - low-bit - quantization - agent - coding --- # Spark-X2.6-4B-LowBit-GGUF > Spark-X2.6-4B-LowBit-GGUF - Dalvlad 在 ModelScope 开源的模型。IQ3_M low-bit GGUF of Spark-X2.6-4B (imatrix-calibrated on a zh-balanced corpus): 2.04 GB, 3.66 BPW, PPL 30.49 vs F16 26.53. Real-chat verified on llama.cpp. Dalvlad/Spark-X2.6-4B-LowBit-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 XHToken/Spark-X2.6-4B 构建。 - **Repository**: Dalvlad/Spark-X2.6-4B-LowBit-GGUF - **License**: apache-2.0 - **Tasks**: text-generation - **Base model**: XHToken/Spark-X2.6-4B - **Tags**: spark, sparkx2_5, gguf, imatrix, low-bit, quantization, agent, coding - **Downloads**: 47 - **Stars**: 0 - **Last updated**: 2026-10-27 Source: https://www.modelscope.cn/models/Dalvlad/Spark-X2.6-4B-LowBit-GGUF --- # Spark-X2.6-4B-LowBit (imatrix-calibrated IQ3_M + IQ4_XS) Two imatrix-calibrated quantizations of [XHToken/Spark-X2.6-4B](https://www.modelscope.cn/models/XHToken/Spark-X2.6-4B), selected from a full quantization ladder evaluated with complete perplexity runs and real-chat capability checks on llama.cpp: - **IQ3_M (3.66 BPW, 2.04 GB)** — the balanced pick, and the one to choose for **Chinese-heavy use**: it produced a fully correct quicksort in the zh code check, including equal-element handling. - **IQ4_XS (4.69 BPW, 2.38 GB)** — the quality-first pick: perplexity statistically identical to the upstream Q4_K_M at two-thirds the size. Caveat: in the zh code check its quicksort had subtle ordering bugs in both thinking modes (pivot misplacement / dropped duplicates — see below), so prefer IQ3_M when Chinese code generation matters. ## Files | file | size | notes | |---|---|---| | `Spark-X2.6-4B-IQ3_M-LowBit.gguf` | 2,043,301,120 B (2.04 GB) | IQ3_M ("IQ3_S mix", 3.66 BPW per llama.cpp) | | `Spark-X2.6-4B-IQ4_XS-LowBit.gguf` | 2,376,046,080 B (2.38 GB) | IQ4_XS, 4.69 BPW — max quality below Q4_K_M size | | `spark-x2.6-lowbit.imatrix` | 3,572,704 B | the importance matrix used for both — kept so you can re-quantize any other bit-width yourself | | `README.md` | this file | — | That is the complete list: **GGUF weights only** — no safetensors, no calibration dataset text, no eval harness. ## Why IQ3_M? (the ladder, measured) All perplexities below are **complete** wikitext-2-raw-test runs (512-token chunks, llama.cpp b393, 24 CPU threads), not sampled estimates: | variant | size | PPL (512 ctx) | Δ vs F16 | in this repo | |---|---|---|---|---| | F16 source (upstream) | 8.23 GB | 26.53 ± 0.24 | — | — | | Q4_K_M (upstream repo, no imatrix) | 3.54 GB | 27.81 ± 0.25 | +4.8% | — | | **IQ4_XS + imatrix** | **2.38 GB** | **27.97 ± 0.26** | **+5.4%** | ✓ | | Q3_K_M + imatrix | 2.17 GB | 36.75 | +38.5% — excluded | — | | **IQ3_M + imatrix** | **2.04 GB** | **30.49 ± 0.27** | **+14.9%** | ✓ | | IQ3_XXS + imatrix | 1.80 GB | 30.65 ± 0.28 | +15.5% (failed real-chat zh check) | — | | IQ2_M + imatrix | 1.55 GB | 60.03 ± 0.56 | +126% — excluded | — | IQ3_XXS is statistically tied with IQ3_M on PPL but failed the real-chat Chinese check (ran out of its thinking budget mid-answer and produced a duplicate-dropping quicksort), so **IQ3_M is the lowest bit-width that actually preserves this model's capability**. ## Imatrix provenance Built with `llama-imatrix` (512-token context) on a ~113K-token slice, round-robin balanced from: wikitext-2-raw validation, bartowski's calibration_datav5 (Dampf enhancedV3 + Kalomaze + edaddario), 13 chapters of the d2l-zh Chinese deep-learning textbook (25% zh density), and C++/Python sources from llama.cpp. The corpus was paragraph-filtered against a safety blocklist (adult, gambling, SEO/pharma spam, weapons/drugs, malware, slurs) — 17 paragraphs dropped, counts only. Final slice: 12.2% Chinese characters (Spark-X2.6 is an en/zh model; a first imatrix built without zh measured worse on every rung — e.g. IQ4_XS 29.55 vs 27.97 here). ## Real-chat verification (llama.cpp b393, greedy, both quants) | check | IQ3_M (2.04 GB) | IQ4_XS (2.38 GB) | |---|---|---| | math: 17 × 24 with steps | ✓ 408, clean steps | ✓ 408 | | coding: prime function + tests | ✓ correct, with passing asserts | ✓ correct | | Chinese: quicksort explained + implemented | ✓ **fully correct** (equal elements handled) — simulation-verified on 8 arrays | ⚠ grammatical zh, but buggy code in both modes: no-think misplaces the pivot (3/8 arrays wrong), thinking-on drops duplicates (4/8 wrong) | | agentic: read_file-based CSV plan | ✓ structured multi-step plan, tool use, input validation | ✓ 6-step structured plan | | loop-resistance (800-token gen) | ✓ no repetition loops | ✓ no repetition loops | | generation speed (24 CPU threads) | ~22–23 tok/s | ~26 tok/s | Both quants share the upstream thinking-verbosity trait (identical on F16): a 500–900-token budget can be consumed by the `[Start thinking]` block. Disable thinking for short direct answers (`--chat-template-kwargs '{"enable_thinking": false}'` in llama.cpp, or `"chat_template_kwargs": {"enable_thinking": false}` on OpenAI-compatible servers) and give thinking-mode generations a generous `max_tokens`. **Bottom line:** IQ4_XS is the pick for maximum quality (English-heavy workloads, best PPL below 3.5 GB); IQ3_M is the pick when size matters most or when Chinese code generation is part of the job — it was the only low-bit rung whose zh code survived simulation testing. ## Known upstream trait (not quantization damage) Spark-X2.6's chat template defaults to high thinking-effort, which is **verbose**: with thinking enabled, a 500–900-token budget can be consumed entirely by the `[Start thinking]` block before the final answer — this is identical on the unquantized F16. For short direct answers, disable thinking per request: `--chat-template-kwargs '{"enable_thinking": false}'` in llama.cpp, or `"chat_template_kwargs": {"enable_thinking": false}` on OpenAI-compatible servers; and give thinking-mode generations a generous `max_tokens`. ## Usage llama.cpp b393+ (`spark2_5` arch): ```bash llama-cli -m Spark-X2.6-4B-IQ3_M-LowBit.gguf -c 32768 -t 16 -cnv # short answers without the thinking phase: llama-cli -m Spark-X2.6-4B-IQ3_M-LowBit.gguf -c 32768 -t 16 \ --chat-template-kwargs '{"enable_thinking": false}' ``` Re-quantize to any other width using the shipped imatrix: ```bash llama-quantize --imatrix spark-x2.6-lowbit.imatrix \ Spark-X2.6-4B-F16.gguf your-choice.gguf IQ4_XS ``` ## Credits - **[XHToken/Spark-X2.6-4B](https://www.modelscope.cn/models/XHToken/Spark-X2.6-4B)** — the upstream model (Apache-2.0). This repo re-quantizes the upstream F16 GGUF; all capability is theirs. - Calibration corpus: [bartowski's calibration_datav5](https://gist.github.com/bartowski1182/82ae9b520227f57d79ba04add13d0d0d), [wikitext-2](https://huggingface.co/datasets/Salesforce/wikitext), [d2l-zh](https://github.com/d2l-ai/d2l-zh), [llama.cpp](https://github.com/ggml-org/llama.cpp) sources. - Methodology: [llama.cpp imatrix discussion #5263](https://github.com/ggml-org/llama.cpp/discussions/5263).