---
title: Qwen-Image-In-Context-Control-Union
canonical_url: "https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-In-Context-Control-Union"
md_url: "https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-In-Context-Control-Union.md"
repository: DiffSynth-Studio/Qwen-Image-In-Context-Control-Union
chinese_name: "Qwen-Image 图像结构控制模型-In Context Controlnet"
last_updated: 2025-08-21
license: "Apache License 2.0"
pipeline_tag: text-to-image-synthesis
tasks:
  - text-to-image-synthesis
base_model:
  - Qwen/Qwen-Image
base_model_relation: adapter
parameters: 471.9M
tensor_type:
  - BF16
library_name:
  - pytorch
  - safetensors
frameworks:
  - Pytorch
downloads: 2016
stars: 14
---

# Qwen-Image-In-Context-Control-Union

> Qwen-Image-In-Context-Control-Union - DiffSynth-Studio 在 ModelScope 开源的模型。本模型是基于 Qwen-Image 训练的图像结构控制 LoRA，采用的 In Context 的技术路线。可以支持多种条件: canny, depth, lineart, softedge, normal 和 openpose。训练框架基于 DiffSynth-Studio 构建，采用的数据集是…

DiffSynth-Studio/Qwen-Image-In-Context-Control-Union 是 ModelScope 魔搭社区上的 471.9M 参数text-to-image-synthesis模型，采用 Apache License 2.0 许可，基于 Qwen/Qwen-Image 构建。

- **Repository**: DiffSynth-Studio/Qwen-Image-In-Context-Control-Union
- **License**: Apache License 2.0
- **Tasks**: text-to-image-synthesis
- **Parameters**: 471.9M
- **Base model**: Qwen/Qwen-Image
- **Downloads**: 2016
- **Stars**: 14
- **Last updated**: 2025-08-21

Source: https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-In-Context-Control-Union

---

# Qwen-Image 图像结构控制模型

![](assets/title.png)

## 模型介绍

本模型是基于 [Qwen-Image](https://www.modelscope.cn/models/Qwen/Qwen-Image) 训练的图像结构控制 LoRA，采用的 In Context 的技术路线。可以支持多种条件: canny, depth, lineart, softedge, normal 和 openpose。训练框架基于 [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) 构建，采用的数据集是 [Qwen-Image-Self-Generated-Dataset](https://www.modelscope.cn/datasets/DiffSynth-Studio/Qwen-Image-Self-Generated-Dataset)。输入 Prompt 推荐以 "Context_Control. "开头。

请注意，在使用 Openpose 控制时，由于这一控制类型的特殊性，无法做到与其他控制类型相似的“点对点”控制效果。

## 效果展示

|控制条件|控制图像|生成图1|生成图2|
|-|-|-|-|
|canny|![](./assets/1_canny.png)|![](./assets/canny_image_seed_1_blue.png)|![](./assets/canny_image_seed_3_pink.png)|
|depth|![](./assets/1_depth.png)|![](./assets/depth_image_seed_2_blue.png)|![](./assets/depth_image_seed_2_pink_1.png)|
|lineart|![](./assets/1_lineart.png)|![](./assets/lineart_image_seed_1_blue.png)|![](./assets/lineart_image_seed_2_pink_1.png)|
|softedge|![](./assets/1_softedge.png)|![](./assets/softedge_image_seed_2_blue.png)|![](./assets/softedge_image_seed_2_pink_1.png)|
|normal|![](./assets/1_normal.png)|![](./assets/normal_image_seed_2_blue.png)|![](./assets/normal_image_seed_2_pink_1.png)|
|openpose|![](./assets/1_openpose.png)|![](./assets/openpose_image_seed_1_blue.png)|![](./assets/openpose_image_seed_4_pink.png)|


## 推理代码
```
git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .
```

```python
from PIL import Image
import torch
from modelscope import dataset_snapshot_download, snapshot_download
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
from diffsynth.controlnets.processors import Annotator

allow_file_pattern = ["sk_model.pth", "sk_model2.pth", "dpt_hybrid-midas-501f0c75.pt", "ControlNetHED.pth", "body_pose_model.pth", "hand_pose_model.pth", "facenet.pth", "scannet.pt"]
snapshot_download("lllyasviel/Annotators", local_dir="models/Annotators", allow_file_pattern=allow_file_pattern)

pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    tokenizer_config=ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="tokenizer/"),
)
snapshot_download("DiffSynth-Studio/Qwen-Image-In-Context-Control-Union", local_dir="models/DiffSynth-Studio/Qwen-Image-In-Context-Control-Union", allow_file_pattern="model.safetensors")
pipe.load_lora(pipe.dit, "models/DiffSynth-Studio/Qwen-Image-In-Context-Control-Union/model.safetensors")

dataset_snapshot_download(dataset_id="DiffSynth-Studio/examples_in_diffsynth", local_dir="./", allow_file_pattern=f"data/examples/qwen-image-context-control/image.jpg")
origin_image = Image.open("data/examples/qwen-image-context-control/image.jpg").resize((1024, 1024))
annotator_ids = ['openpose', 'canny', 'depth', 'lineart', 'softedge', 'normal']
for annotator_id in annotator_ids:
    annotator = Annotator(processor_id=annotator_id, device="cuda")
    control_image = annotator(origin_image)
    control_image.save(f"{annotator.processor_id}.png")

    control_prompt = "Context_Control. "
    prompt = f"{control_prompt}一个穿着淡蓝色的漂亮女孩正在翩翩起舞，背景是梦幻的星空，光影交错，细节精致。"
    negative_prompt = "网格化，规则的网格，模糊, 低分辨率, 低质量, 变形, 畸形, 错误的解剖学, 变形的手, 变形的身体, 变形的脸, 变形的头发, 变形的眼睛, 变形的嘴巴"
    image = pipe(prompt, seed=1, negative_prompt=negative_prompt, context_image=control_image, height=1024, width=1024)
    image.save(f"image_{annotator.processor_id}.png")
```