--- title: refcontrol-FLUX.2-klein-9B-reference-canny-lora canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora" md_url: "https://www.modelscope.cn/models/AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora.md" repository: AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora last_updated: 2026-07-18 license: apache-2.0 pipeline_tag: image-to-image tasks: - image-to-image base_model: - black-forest-labs/FLUX.2-klein-base-9B base_model_relation: adapter parameters: 82.8M tensor_type: - BF16 library_name: - lora - safetensors - pytorch frameworks: - pytorch supports_inference: img2img downloads: 140 stars: 1 tags: - flux - flux2 - klein - canny - controlnet - refcontrol - img2img - image - editing - lora --- # refcontrol-FLUX.2-klein-9B-reference-canny-lora > refcontrol-FLUX.2-klein-9B-reference-canny-lora - AI-ModelScope 在 ModelScope 开源的模型。RefControl FLUX.2 Klein 9B – Reference Canny LoRA AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora 是 ModelScope 魔搭社区上的 82.8M 参数image-to-image模型，采用 apache-2.0 许可，基于 black-forest-labs/FLUX.2-klein-base-9B 构建，并支持在线推理（img2img）。 - **Repository**: AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora - **License**: apache-2.0 - **Tasks**: image-to-image - **Parameters**: 82.8M - **Base model**: black-forest-labs/FLUX.2-klein-base-9B - **Online inference**: img2img - **Tags**: flux, flux2, klein, canny, controlnet, refcontrol, img2img, image, editing, lora - **Downloads**: 140 - **Stars**: 1 - **Last updated**: 2026-07-18 Source: https://www.modelscope.cn/models/AI-ModelScope/refcontrol-FLUX.2-klein-9B-reference-canny-lora --- # RefControl FLUX.2 Klein 9B – Reference Canny LoRA ## 📝 Short description A LoRA for **FLUX.2 Klein 9B Base** that fuses a **reference image** (identity) with a **canny edge map** (structure / contours). It preserves **identity and style** from the reference while following the **shape and composition** from the canny control map. **Trigger word:** `refcontrol` --- ## 📊 Examples Each preview is a **single combined image** from ComfyUI: **Canny → Reference → Result** (left to right).
Canny → Reference → Result
--- ## 📖 Extended description This LoRA was primarily trained on **humans**, but it also works with **stylized characters** and some objects. Its main purpose is to **preserve identity** — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the **structure and contours** defined by the canny edge map. **FLUX.2 Klein 9B Base** already handles reference + edge-guided transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and **better preserves character identity and edge fidelity** than the base model without LoRA. Part of the **RefControl** family: reference + control fusion for consistent, controllable generation on **FLUX.2 Klein 9B Base**. --- ## ⚙️ How to use 1. Use the **first image** as the **canny edge map** (structure / contours). 2. Use the **second image** as the **reference** (character, person, or object). 3. Add the trigger word **`refcontrol`** in your prompt. 4. Adjust LoRA weight (recommended **0.8–1.0**) depending on how strongly you want to preserve identity. ## ComfyUI requirenments Canny extraction in the included workflow uses **CannyEdgePreprocessor** (via [comfyui_controlnet_aux](https://github.com/Fannovel16/comfyui_controlnet_aux)): https://github.com/Fannovel16/comfyui_controlnet_aux You can disable the built-in canny preprocessor if your input is already a canny/edge image. ### Base model Trained on and recommended with [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B). The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, `guidance_scale` ~4.0). The LoRA also works with the **4-step distilled** [`black-forest-labs/FLUX.2-klein-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-9B) for faster inference (~4 steps, `guidance_scale` ~1.0), but **quality may be slightly lower** — especially for identity and edge fidelity — compared to the Base model. ### ✅ Example prompt ``` refcontrol ``` --- ## 🎯 What it does - Preserves **character or object identity** across generations. - Adapts the subject to a **new structure or composition** defined by the canny map. - Works best when the edge map has **similar proportions and scale** to the reference. --- ## ⚡ Tips - Best results when the canny map is not drastically different in body scale or framing from the reference. - Combine with **text prompts** to refine background, lighting, or mood. - **Canny edge maps** on a **black background** work well as control input. --- ## 📌 Use cases - Character restyling while keeping identity and edge-defined structure. - Consistent **character design** across different compositions. - Illustration and storyboard generation with edge-guided layout. - Object transformations with contour-guided placement. --- ## 📦 Files - **Weights:** [`flux2_klein_9b_refcontrol_canny.safetensors`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora/blob/main/flux2_klein_9b_refcontrol_canny.safetensors) - **ComfyUI workflow:** [`refcontrol_canny_flux_klein_9b.json`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora/blob/main/refcontrol_canny_flux_klein_9b.json) - **Repo:** [thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora)