--- title: FLUX-3-Image-Klein-GGUF canonical_url: "https://www.modelscope.cn/models/AtomicChat/FLUX-3-Image-Klein-GGUF" md_url: "https://www.modelscope.cn/models/AtomicChat/FLUX-3-Image-Klein-GGUF.md" repository: AtomicChat/FLUX-3-Image-Klein-GGUF last_updated: 2026-10-28 license: other pipeline_tag: text-to-image-synthesis tasks: - text-to-image-synthesis base_model: - black-forest-labs/flux-3-image-klein base_model_relation: quantized parameters: 84.0M tensor_type: - F32 - I64 library_name: - lora - gguf - safetensors - pytorch frameworks: - pytorch downloads: 60 stars: 0 tags: - gguf - stable-diffusion.cpp - flux - flux-3 - text-to-image - image-editing - imatrix - quantized - atomic-chat --- # FLUX-3-Image-Klein-GGUF > FLUX-3-Image-Klein-GGUF - AtomicChat 在 ModelScope 开源的模型。How to Run FLUX 3 Image Klein Locally Black Forest Labs' FLUX 3 Image Klein as GGUF for stable-diffusion.cpp, built from the published BF16 weights with our own importance matrix and measured on one… AtomicChat/FLUX-3-Image-Klein-GGUF 是 ModelScope 魔搭社区上的 84.0M 参数text-to-image-synthesis模型，采用 other 许可，基于 black-forest-labs/flux-3-image-klein 构建。 - **Repository**: AtomicChat/FLUX-3-Image-Klein-GGUF - **License**: other - **Tasks**: text-to-image-synthesis - **Parameters**: 84.0M - **Base model**: black-forest-labs/flux-3-image-klein - **Tags**: gguf, stable-diffusion.cpp, flux, flux-3, text-to-image, image-editing, imatrix, quantized, atomic-chat - **Downloads**: 60 - **Stars**: 0 - **Last updated**: 2026-10-28 Source: https://www.modelscope.cn/models/AtomicChat/FLUX-3-Image-Klein-GGUF --- # How to Run FLUX 3 Image Klein Locally

Black Forest Labs' FLUX 3 Image Klein as GGUF for stable-diffusion.cpp, built from the published BF16 weights with our own importance matrix and measured on one stand against every other Klein GGUF we could find. The renders and measurements are public.

Atomic Chat Discord GitHub X
[FLUX 3 Image Klein](https://huggingface.co/black-forest-labs/flux-3-image-klein) is Black Forest Labs' 5.6B text-to-image and editing model: one model for generation, single-reference edits and up to ten reference images, distilled to **16 steps at CFG 1**. This repo has the image model as GGUF for [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), with the text encoder and the autoencoder next to it. ## Pick a file Each file renders the same 48 captions × 2 seeds from the same starting noise as the BF16 weights, and every image is compared with the BF16 image of the same caption and seed. - `LPIPS` is a perceptual distance: lower is closer to the original picture, 0 is identical. - `vs. new seed` puts it on a scale. 100% is how far the BF16 weights move when only the seed changes, which gives an unrelated picture of the same caption (LPIPS 0.530). - `text (zh)` is the LPIPS on the eight captions that put Chinese text into the image. | File | Size on disk | LPIPS [95% CI] | vs. new seed | SSIM | PSNR | text (zh) | |---|---:|---|---:|---:|---:|---:| | `BF16` | 11.10 GB | 0 (identical) | 0% | 1.000 | — | 0 | | `Q8_0` | 5.99 GB | 0.014 [0.011, 0.017] | 3% | 0.982 | 34.4 dB | 0.011 | | **`AD-Q5_K`** | **4.44 GB** | **0.038 [0.032, 0.044]** | **7%** | **0.954** | **28.8 dB** | **0.033** | | **`AD-Q4_K`** | **3.65 GB** | **0.081 [0.069, 0.095]** | **15%** | **0.903** | **24.4 dB** | **0.070** | - **`Q8_0`** is the closest to the original. Pick it if it fits. - **`AD-Q5_K`** is the pick at about 4.5 GB: the same picture, small details move. - **`AD-Q4_K`** keeps the composition and the lettering; textures and small objects shift. - Below 3.6 GB we do not publish a file of our own: at 3.3 GB and 2.6 GB unsloth's `UD-Q3_K_XL` and `UD-Q2_K_XL` are as good as or better than our best layouts. - The BF16 weights render deterministically here: a second BF16 render of the same caption and seed is LPIPS 0.0004 away, so every number above comes from quantization. ![The same four captions and seed in BF16, Q8_0, AD-Q5_K, AD-Q4_K and unsloth's Q4_K_M](https://huggingface.co/AtomicChat/FLUX-3-Image-Klein-GGUF/resolve/main/samples-grid.png) ## How these compare with other GGUFs Every file below was downloaded and rendered here with the same captions, seeds, binary and GPU. ![LPIPS to the BF16 picture against size on disk for our files and every other Klein GGUF we measured](https://huggingface.co/AtomicChat/FLUX-3-Image-Klein-GGUF/resolve/main/quality-vs-size.png) | File | Size on disk | LPIPS [95% CI] | Ours at that size or smaller | |---|---:|---|---| | leejet `Q8_0` | 6.15 GB | 0.016 [0.012, 0.021] | `Q8_0`, 5.99 GB: 0.014 | | unsloth `Q8_0` | 6.09 GB | 0.015 [0.012, 0.021] | `Q8_0`, 5.99 GB: 0.014 | | unsloth `Q6_K` | 4.80 GB | 0.035 [0.029, 0.043] | `AD-Q5_K`, 4.44 GB: 0.038. Our 4.78 GB layout measured the same as theirs (0.035), so we did not publish it | | unsloth `UD-Q5_K_XL` | 4.48 GB | 0.044 [0.037, 0.052] | **`AD-Q5_K`, 4.44 GB: 0.038, 14% closer** | | unsloth `UD-Q4_K_XL` | 3.93 GB | 0.055 [0.047, 0.062] | `AD-Q4_K`, 3.65 GB: 0.081. **Theirs is the better file at this size** | | unsloth `Q4_K_M` | 3.67 GB | 0.103 [0.093, 0.114] | **`AD-Q4_K`, 3.65 GB: 0.081, 21% closer** | | leejet `Q4_K` | 3.51 GB | 0.126 [0.112, 0.140] | none | | unsloth `UD-Q3_K_XL` | 3.28 GB | 0.116 [0.101, 0.130] | none. Our best 3.26 GB layout measured the same (0.114), so we did not publish it | | Abiray `Q4_K_M` | 3.27 GB | 0.129 [0.114, 0.144] | none | | unsloth `UD-Q2_K_XL` | 2.62 GB | 0.203 [0.182, 0.224] | none. **Theirs is better than our best 2.59 GB layout (0.212)** | | Abiray `Q3_K_M` | 2.55 GB | 0.279 [0.260, 0.297] | none | | leejet `Q2_K` | 2.28 GB | 0.413 [0.389, 0.435] | none | - **Unsloth's UD files are strong below 5 bits.** They move bits from the reference-image blocks, which text-to-image never runs, to the last single blocks. We tried the same moves at 3.9 GB (rows `AD-Q4_K-v/-w/-x` in the metrics repo, 0.076 to 0.079 on one seed against their 0.054) and did not catch `UD-Q4_K_XL`. - Abiray's `Q4_K_M` and `Q3_K_M`, and leejet's files, are stable-diffusion.cpp's defaults (`sd-cli -M convert --type`): Abiray's measure the same as the plain `Q4_K` and `Q3_K` we built. The `_K_M` suffix does not mean a mixed layout there. - Our `Q8_0` is 0.1 GB smaller than the other `Q8_0` files because the stable-diffusion.cpp version we used also quantizes the six modulation layers, and it is still closer to the original. - Not measured: unsloth's `Q5_K_M` (4.22 GB) and `Q3_K_M` (3.01 GB), and leejet's and Abiray's other sizes, which are the tool's defaults like the ones above. ### What `AD-` changes `AD-` means Atomic Dynamic: the type is chosen per tensor from a sensitivity scan of this model, and the file is quantized with an importance matrix from 32 upsampled captions that share nothing with the evaluation set. The scan ranked, per byte saved, the six modulation layers, the attention `v` and output projections and the last 14 single blocks as the most sensitive, and the MLP input projections as the least. - **`AD-Q5_K`** (base Q5_K): the modulation layers stay BF16; attention `v` and output Q8_0 in the last 14 single blocks and Q6_K elsewhere; everything else in the last 14 single blocks Q6_K; the five text-only blocks Q6_K; the five reference-image blocks, which run only for edits, Q3_K. - **`AD-Q4_K`** (base Q4_K): the five text-only blocks Q6_K; attention `v` and output Q6_K in the last 14 single blocks and Q5_K elsewhere; the MLP output of the last 14 single blocks and the MLP input of the last 7 Q5_K; the modulation layers and the reference-image blocks at the base type. - The exact rules and the per-tensor type map of every file are in the metrics repo (`logs/convert-*.log`, `layouts/`). `AD-Q5_K` is the row `AD-Q5_K-u` there and `AD-Q4_K` the row `AD-Q4_K-v1`. ## The text encoder Klein reads Qwen3-VL-4B-Instruct's hidden states, and the encoder's quantization moves the picture as much as the image model's. Each encoder below rendered the 48 captions on seed 42 with the BF16 image model, compared with the same renders through the BF16 encoder: | Encoder file | Size on disk | Picture LPIPS [95% CI] | Next-token KLD to BF16 | |---|---:|---|---:| | `text_encoders/Qwen3-VL-4B-Instruct-Q8_0.gguf` | 4.28 GB | 0.016 [0.012, 0.020] | 0.0015 | | `text_encoders/Qwen3-VL-4B-Instruct-AD-Q6_K.gguf` | 3.68 GB | 0.029 [0.024, 0.035] | 0.0024 | | **`text_encoders/Qwen3-VL-4B-Instruct-AD-Q4_K.gguf`** | **2.49 GB** | **0.070 [0.058, 0.082]** | **0.025** | | Qwen's own `Qwen3VL-4B-Instruct-Q4_K_M.gguf` | 2.50 GB | 0.088 [0.070, 0.109] | not measured; a plain Q4_K_M of ours: 0.032 | | Qwen's own `Qwen3VL-4B-Instruct-Q8_0.gguf` | 4.28 GB | 0.016 [0.012, 0.020] | not measured | - A Q4_K_M encoder moves the picture more (0.088) than our `AD-Q4_K` image model does (0.081). Use the `Q8_0` encoder if it fits. - These are one seed; the image model numbers above are two. - The text encoder is Qwen3-VL-4B-Instruct byte for byte (the sha256 of Klein's `text_encoder/` shards equals `Qwen/Qwen3-VL-4B-Instruct`'s), so any GGUF of it works. ## Running it Klein is distilled to **16 steps at CFG 1**. stable-diffusion.cpp's default schedule for it is BFL's (time shift 6.93), so no sigma list is needed. ```bash sd-cli --diffusion-model FLUX-3-Image-Klein-AD-Q5_K.gguf \ --llm text_encoders/Qwen3-VL-4B-Instruct-Q8_0.gguf \ --vae vae/ae.safetensors \ -p "$(cat caption.txt)" \ --steps 16 --cfg-scale 1.0 --sampling-method euler -W 1024 -H 1024 -o out.png ``` - **Use a caption, not a short prompt.** Klein is trained on full scene captions written by a prompt upsampler. BFL's is a 9B model, in GGUF at [AtomicChat/FLUX-3-Image-Klein-pup-GGUF](https://huggingface.co/AtomicChat/FLUX-3-Image-Klein-pup-GGUF) with BFL's system prompts. Any capable LLM can write the caption with the `t2i` system prompt. A short prompt sent as typed gives a generic picture. - **stable-diffusion.cpp `master-968-8cd731e` or newer** (FLUX 3 support, leejet/stable-diffusion.cpp#2143). We measured with `95b3090` (`master-975`). - **Editing:** add the reference images with `-r` (up to 10), and write the caption from the instruction and the references (the upsampler's `i2i` system prompt, with its vision projector). ### In ComfyUI - **Image model:** our GGUF in `models/unet/`, loaded with [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF)'s **Unet Loader (GGUF)**. Start from ComfyUI's FLUX 3 Klein template ("distilled t2i") and put this node in place of Load Diffusion Model. - **Text encoder:** use `qwen3vl_4b_bf16.safetensors` (or `qwen3vl_4b_fp8_scaled.safetensors`) from [Comfy-Org/Krea-2](https://huggingface.co/Comfy-Org/Krea-2/tree/main/text_encoders) with the core **Load CLIP** node, type `flux3`. - Our encoder GGUFs in `text_encoders/` are for stable-diffusion.cpp. In ComfyUI, ComfyUI-GGUF loads a Qwen3-VL GGUF as plain Qwen3-4B, and the sampler stops with a shape error (`2560` against `20480`). - **VAE:** `vae/ae.safetensors` with **Load VAE**. - **Checked on 21.10** with ComfyUI `7a28332` and ComfyUI-GGUF `6ea2651`: 8 captions, seed 42, the template's 16 steps, euler, simple. Each LPIPS is against ComfyUI's own render with BFL's BF16 safetensors: | file | LPIPS | |---|---:| | our BF16 GGUF | 0.0000 (the same picture) | | `Q8_0` | 0.021 | | `AD-Q5_K` | 0.031 | | `AD-Q4_K` | 0.069 | | unsloth's `UD-Q4_K_XL` (same run) | 0.052 | The renders are in the metrics repo under `comfyui/`. ## Changes - **2026-10-21:** the four image-model files now carry `general.architecture = flux` and `general.quantization_version = 2` in their metadata, so ComfyUI-GGUF loads them. Before, it stopped with "Unknown model architecture!". - Each file is 96 bytes larger. The tensors are byte for byte the same, and stable-diffusion.cpp renders the same pixels as before. - A file downloaded earlier keeps working in stable-diffusion.cpp. ## Content filtering The open FLUX 3 Klein pipeline has no content filter of its own. BFL's safety checks, an NSFW image classifier and optional prompt checks, live in BFL's Python inference code. stable-diffusion.cpp, ComfyUI and Atomic Chat do not run them. BFL also states that it evaluated and mitigated risks including child sexual abuse material and non-consensual intimate imagery before release. We measured what the open pipeline draws on 45 probe requests in nine categories (adults only, no real people), upsampled by BFL's prompt upsampler and judged by an image model, two seeds, BF16 weights: | Category | Requested content drawn | |---|---:| | Nudity (art, beach, life drawing) | 8 / 10 | | Suggestive | 7 / 10 | | Violence | 10 / 10 | | Weapons | 10 / 10 | | Drugs | 9 / 10 | We also removed the refusal direction from the text encoder and from the prompt upsampler (abliteration) and ran the same probes. Neither changed any category by more than one image in ten, in either direction, alone or together. The stock upsampler already writes the requested content into its captions (31 of 35 sensitive requests; the abliterated one 33, and the image model draws neither of the two extra ones). With no open component refusing, an uncensored encoder or upsampler has nothing to unlock in Klein's pictures. We publish both as models without refusals, with these numbers in their cards: [FLUX-3-Image-Klein-TE-Uncensored-GGUF](https://huggingface.co/AtomicChat/FLUX-3-Image-Klein-TE-Uncensored-GGUF) (the text encoder, a drop-in `--llm`) and [FLUX-3-Image-Klein-pup-Uncensored-GGUF](https://huggingface.co/AtomicChat/FLUX-3-Image-Klein-pup-Uncensored-GGUF) (the prompt upsampler). The aggregates are in the metrics repo (`diagnostic/`); the probe requests and their renders are not published. The FLUX Kommunity License forbids, among other things, using the model to generate unlawful content, including child sexual abuse material and non-consensual intimate images, and violating anyone's rights of publicity. See the [license](LICENSE.md), section 4. ## How it was measured - **Engine:** stable-diffusion.cpp at `95b3090` (`master-975`) on an NVIDIA H100 80 GB. - **Settings:** 1024×1024, 16 steps, CFG 1.0, `euler`, the default schedule, `--rng cpu`, seeds 42 and 43. - **Prompts:** 48 requests in six categories (photo, illustration, English text, Chinese text, several objects with attributes, transparent background), each upsampled to a full caption by BFL's prompt upsampler. - **Held constant:** the BF16 text encoder and the autoencoder, so the image model file was the only thing that changed. - **Not measured:** aesthetics or caption adherence as such, edits with reference images (a quick check with the BF16 render as its own reference showed every file at Q4 and above reproducing it within LPIPS 0.015; that does not measure real edits), other step counts, resolutions or samplers, and runtimes other than stable-diffusion.cpp. - **In the [metrics repo](https://huggingface.co/datasets/AtomicChat/FLUX-3-Image-Klein-GGUF-metrics):** every render, the captions, the importance matrix, the sensitivity scan, the per-tensor layout of every file, ours and theirs, and every log. ## How these files were made 1. BFL's `model.safetensors` and `ae.safetensors` at revision `3373376`, checked against the Hub's sha256. `vae/ae.safetensors` is BFL's file, unchanged. 2. Converted with `sd-cli -M convert` from stable-diffusion.cpp `95b3090`. The `AD-` files add `--tensor-type-rules` and `--imat-in`, with the exact command for each in the metrics repo. 3. The text encoders: Qwen/Qwen3-VL-4B-Instruct, converted with llama.cpp `6184e92`, importance matrix from our general calibration corpus with `--parse-special`. ## License The image model files and the autoencoder are a modified FLUX Model under the [FLUX Kommunity License](LICENSE.md); see [NOTICE.md](NOTICE.md). The license allows non-commercial use, and commercial use of outputs for Qualifying Users (under US$5M annual revenue), under its terms. The text encoders are under the Apache License 2.0 (`text_encoders/LICENSE`). Quantized by AtomicChat; not affiliated with or endorsed by Black Forest Labs.