--- title: yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF canonical_url: "https://www.modelscope.cn/models/hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF" md_url: "https://www.modelscope.cn/models/hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF.md" repository: hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF last_updated: 2026-07-18 license: apache-2.0 pipeline_tag: text-generation tasks: - text-generation base_model: - google/gemma-4-12B-it base_model_relation: quantized library_name: - gguf - pytorch frameworks: - pytorch downloads: 1266 stars: 25 tags: - gemma4 - coding - code - reasoning - thinking - gguf - llama.cpp - local-llm --- # yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF > yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF - hf 在 ModelScope 开源的模型。💻 Gemma4-12B-Coder (GGUF) — Composer 2.5 × Fable 5 ✨ 🐣 Tiny footprint, big brain — a local coding model for everyone hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 google/gemma-4-12B-it 构建。 - **Repository**: hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF - **License**: apache-2.0 - **Tasks**: text-generation - **Base model**: google/gemma-4-12B-it - **Tags**: gemma4, coding, code, reasoning, thinking, gguf, llama.cpp, local-llm - **Downloads**: 1266 - **Stars**: 25 - **Last updated**: 2026-07-18 Source: https://www.modelscope.cn/models/hf/yuxinlu1-gemma-4-12B-coder-fable5-composer2.5-v1-GGUF --- # 💻 Gemma4-12B-Coder (GGUF) — Composer 2.5 × Fable 5 ✨ ### 🐣 Tiny footprint, big brain — a local **coding** model for *everyone* > **No matter your GPU. No matter your RAM.** If you've got **~4.5 GB** of VRAM *or* unified memory free, > you can run your own private, offline coding assistant right now. 🚀 > This is the **v1 / code edition** — distilled from **real chain-of-thought** so it *thinks through* a problem > before writing the solution. 🧠💻 All local, all yours, no API, no cloud. ### 🎯 What it is A focused fine-tune of Gemma 4 12B on **verifiable Python coding** data — every training example's reasoning leads to code that **actually passed its tests**. The result reasons in the open (edge cases, complexity, approach) and then emits a clean, runnable solution. 💚 --- ## 📌 Announcements **🚀🔥 IT'S HERE — v2 is OUT NOW!** v2 has shipped — the **GGUF quants are live and ready to run** → **[grab v2 here](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF)**. 🎉 The **full `safetensors` master** (build / fine-tune on top) goes up **tomorrow**. v2 is **agentic + coding** focused — the piece v1 was missing. **Here's the result that got me most excited.** When I saw v2's **tau2-bench `telecom`** result — an agentic tool-use benchmark where the model has to *diagnose → fix → verify*, exactly like real terminal/debugging work — I literally got **launched out of my chair** (…okay, *kidding* 😄). The jump in **actually solving the problem** is wild: | tau2-bench **telecom** · local, same harness, **Q8_0** | score | |---|---| | official `gemma-4-12B-it` (base) | **~15%** | | 🟢 **v2 (this release)** | **~55%** | The base model tends to **give up early** (hands the problem off to a human); **v2 keeps going** and works it the way a much bigger model would. Full benchmark details are in the **[v2 card](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF)** now. 🔧 **✅ safetensors master (this v1 model) is UP.** Full-precision weights are live → **[yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1](https://huggingface.co/yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1)** — roll your own GGUF / MLX / AWQ quants or fine-tune straight from the master. 🎉 --- ## 📣 Context length fixed: now **256K** (was 131K) — thanks, community! 💚 A community member spotted that this model was reporting only a **131K** context window. That turned out to be the well-known upstream **Gemma 4 metadata bug** — Google's initial `config.json` shipped with `max_position_embeddings: 131072` instead of the real **262144 (256K)**, and that value got baked into a lot of downstream finetunes and quants (including this one) before it was fixed upstream. The weights were always fine — it was purely a metadata field. **All GGUF quants have been re-patched to the full 256K context** (`gemma4.context_length = 262144`). Just re-download if you grabbed an earlier copy. 🙏 --- ## 📚 Training data (the interesting part 🍳) This is a **distillation** of two complementary chain-of-thought sources, both over verifiable Python coding tasks (algorithmic / function-level problems that come with deterministic tests): - **🥇 Main set — Composer 2.5 *real* CoT.** Genuine, model-authored reasoning traces. The teacher solved each problem, its code was **run against the task's tests, and only the passing solutions were kept**. So the reasoning you're learning from leads to code that *actually works*. - **🥈 Aux set — Fable 5 (released today! 🎉).** A clever twist: we took the problems where **Composer 2.5 got it wrong** and handed them to **Fable 5** to *redo* — re-deriving a fresh, self-consistent chain-of-thought and a correct solution, again **gated on passing the tests**. This recovers the hard cases the main teacher missed. These traces are **synthetic** (rationalized CoT), and are tagged separately so the two sources stay distinguishable. The recipe: real CoT for the bulk of solid coverage, plus synthetic "second-attempt" CoT to patch the failures — both verified by execution before anything entered training. ✅ --- ## 📦 Pick your size (GGUF quants) | Quant | Size | Vibe | |------|------|------| | 🟢 **Q2_K** | **4.5 GB** | tiniest — runs almost anywhere | | 🟡 **Q3_K_M** | **5.7 GB** | great for 8 GB VRAM — much better than Q2 | | 🔵 **Q4_K_M** | **6.87 GB** | the sweet spot 👌 (recommended) | | 🟣 **Q6_K** | **9.11 GB** | near-lossless | | ⚪ **Q8_0** | **11.8 GB** | basically full quality | --- ## 🧮 "Will it fit?" — context length cheat-sheet Rough estimates 🤓 (assumes `q8_0` KV cache + ~1.5 GB overhead; **use `q4_0` KV cache for ≈2× more context!**). Max context is **256K**. "—" = won't fit, pick a smaller quant. ✂️ | Your VRAM / unified mem | 🟢 Q2_K (4.5G) | 🟡 Q3_K_M (5.7G) | 🔵 Q4_K_M (6.87G) | 🟣 Q6_K (9.11G) | ⚪ Q8_0 (11.8G) | |---|---|---|---|---|---| | **8 GB** | ~16K ctx | ~10K | tight (~2–4K) | — | — | | **12 GB** | ~48K | ~38K | ~30K | ~12K | — | | **16 GB** | ~80K | ~72K | ~64K | ~44K | ~22K | | **24 GB** | ~200K | ~160K | ~128K | ~110K | ~88K | | **32 GB** | 256K (max) 🎉 | 256K | 256K | ~230K | ~190K | > 💡 Apple Silicon / integrated GPUs with **unified memory** count too — same numbers, just slower than a dGPU. > 💡 Low on room? Drop a quant or switch KV cache to `q4_0` and your context roughly doubles. --- ## 🚀 How to run it (super easy) ### Option A — llama.cpp (recommended) 🦙 1. Grab a quant above (e.g. `…-Q4_K_M.gguf`) and `llama-server` from [llama.cpp](https://github.com/ggml-org/llama.cpp). > ⚠️ Needs a **recent llama.cpp** (this is the `gemma4_unified` architecture — older builds won't load it). 2. Run a server (Windows `.bat` shown — tweak `--port`, `--ctx-size` to taste): ```bat @echo off cd /d C:\llama.cpp llama-server.exe ^ -m C:\models\gemma4-coding-Q4_K_M.gguf ^ --ctx-size 16384 ^ --n-gpu-layers 99 ^ --no-mmap ^ -fa on ^ --cache-type-k q8_0 --cache-type-v q8_0 ^ --temp 1.0 --top-p 0.95 --top-k 64 ^ --host 0.0.0.0 --port 18080 pause ``` 3. Open `http://localhost:18080` and chat. 🎉 (Tip: bump `--ctx-size` per the table; use `q4_0` KV for more.) ### Option B — one-click apps 🖱️ Works in **LM Studio**, **Jan**, **Ollama**, etc. — just import the GGUF, pick your quant, go. 🐾 ### 🧠 Thinking mode This model thinks in Gemma's native thought channel before answering — exactly how it was trained. Keep **`enable_thinking=true`** (the default chat template handles it). Recommended sampling: `temp 1.0, top_p 0.95, top_k 64`. For coding you can also go greedy (`temp 0`) for more deterministic solutions. --- ## ⚠️ Good to know - **Reduced refusals:** the training data is task-focused with no safety hedging, so this refuses less than the base model. It is **not** safety-aligned — add your own guardrails for production. Use responsibly. 🙏 - Specialized for **Python / algorithmic** coding. Reasoning quality is strongest in that domain; general-knowledge facts/numbers should still be double-checked. - English-centric. --- ## 📚 Base & License - **License: Apache 2.0.** Gemma 4 is released by Google under **[Apache 2.0](https://ai.google.dev/gemma/apache_2)** (unlike the older Gemma 1/2/3 terms), so this fine-tune is **Apache 2.0** too — free to use, modify, and redistribute. 🎉 - **Base model:** [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it). - Personal/hobby project — shared as-is, no warranty. Have fun, and happy hacking! 🐾✨