---
title: Qwen-Image-EliGen
canonical_url: "https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-EliGen"
md_url: "https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-EliGen.md"
repository: DiffSynth-Studio/Qwen-Image-EliGen
chinese_name: "Qwen-Image Eligen精确分区控制模型"
last_updated: 2025-09-28
license: "Apache License 2.0"
pipeline_tag: text-to-image-synthesis
tasks:
  - text-to-image-synthesis
base_model:
  - Qwen/Qwen-Image
base_model_relation: adapter
parameters: 235.9M
tensor_type:
  - BF16
library_name:
  - pytorch
  - safetensors
frameworks:
  - Pytorch
downloads: 1298
stars: 25
---

# Qwen-Image-EliGen

> Qwen-Image-EliGen - DiffSynth-Studio 在 ModelScope 开源的模型。本模型是基于 Qwen-Image 训练的精确分区控制模型，模型结构为 LoRA，可以通过输入每个实体的文本和区域条件（蒙版图）来控制每个实体的位置和形状。训练框架基于 DiffSynth-Studio 构建，采用的数据集是 DiffSynth-Studio/EliGenTrainSet。

DiffSynth-Studio/Qwen-Image-EliGen 是 ModelScope 魔搭社区上的 235.9M 参数text-to-image-synthesis模型，采用 Apache License 2.0 许可，基于 Qwen/Qwen-Image 构建。

- **Repository**: DiffSynth-Studio/Qwen-Image-EliGen
- **License**: Apache License 2.0
- **Tasks**: text-to-image-synthesis
- **Parameters**: 235.9M
- **Base model**: Qwen/Qwen-Image
- **Downloads**: 1298
- **Stars**: 25
- **Last updated**: 2025-09-28

Source: https://www.modelscope.cn/models/DiffSynth-Studio/Qwen-Image-EliGen

---

# Qwen-Image 精确分区控制模型

![](./assets/title.png)

## 模型介绍

本模型是基于 [Qwen-Image](https://www.modelscope.cn/models/Qwen/Qwen-Image) 训练的精确分区控制模型，模型结构为 LoRA，可以通过输入每个实体的文本和区域条件（蒙版图）来控制每个实体的位置和形状。训练框架基于 [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) 构建，采用的数据集是 [DiffSynth-Studio/EliGenTrainSet](https://www.modelscope.cn/datasets/DiffSynth-Studio/EliGenTrainSet)。


## 效果展示

|实体控制条件|生成图|
|-|-|
|![eligen_example_1_0](./assets/samples/poster_region.png)|![eligen_example_1_mask_0](./assets/samples/poster.png)|
|![eligen_example_1_0](./assets/samples/eligen_example_1_mask.png)|![eligen_example_1_mask_0](./assets/samples/eligen_example_1.png)|
|![eligen_example_1_0](./assets/samples/eligen_example_2_mask.png)|![eligen_example_1_mask_0](./assets/samples/eligen_example_2.png)|
|![eligen_example_1_0](./assets/samples/eligen_example_3_mask.png)|![eligen_example_1_mask_0](./assets/samples/eligen_example_3.png)|
|![eligen_example_1_0](./assets/samples/eligen_example_4_mask.png)|![eligen_example_1_mask_0](./assets/samples/eligen_example_4.png)|
|![eligen_example_1_0](./assets/samples/eligen_example_5_mask.png)|![eligen_example_1_mask_0](./assets/samples/eligen_example_5.png)|


## 推理代码
```
git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .
```

```python
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
from modelscope import dataset_snapshot_download, snapshot_download
import torch
from PIL import Image


pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    tokenizer_config=ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="tokenizer/"),
)
snapshot_download("DiffSynth-Studio/Qwen-Image-EliGen", local_dir="models/DiffSynth-Studio/Qwen-Image-EliGen", allow_file_pattern="model.safetensors")
pipe.load_lora(pipe.dit, "models/DiffSynth-Studio/Qwen-Image-EliGen/model.safetensors")

global_prompt =  "Qwen-Image-EliGen魔法咖啡厅的宣传海报，主体是两杯魔法咖啡，一杯冒着火焰，一杯冒着冰锥，背景是浅蓝色水雾，海报写着“Qwen-Image-EliGen魔法咖啡厅”、“新品上市”"
entity_prompts = ["一杯红色魔法咖啡，杯中火焰燃烧", "一杯红色魔法咖啡，杯中冰锥环绕", "字：“新品上市”", "字：“Qwen-Image-EliGen魔法咖啡厅”"]

dataset_snapshot_download(dataset_id="DiffSynth-Studio/examples_in_diffsynth", local_dir="./", allow_file_pattern=f"data/examples/eligen/qwen-image/example_6/*.png")
masks = [Image.open(f"./data/examples/eligen/qwen-image/example_6/{i}.png").convert('RGB').resize((1328, 1328)) for i in range(len(entity_prompts))]

image = pipe(
    prompt=global_prompt,
    seed=0,
    eligen_entity_prompts=entity_prompts,
    eligen_entity_masks=masks,
)
image.save("image.jpg")
```
## 引用
如果您觉得我们的工作对您有所帮助，欢迎引用我们的成果。
```
@article{zhang2025eligen,
  title={Eligen: Entity-level controlled image generation with regional attention},
  author={Zhang, Hong and Duan, Zhongjie and Wang, Xingjun and Chen, Yingda and Zhang, Yu},
  journal={arXiv preprint arXiv:2501.01097},
  year={2025}
}
```