---
title: cv_stable-diffusion_paint-by-example
canonical_url: "https://www.modelscope.cn/models/iic/cv_stable-diffusion_paint-by-example"
md_url: "https://www.modelscope.cn/models/iic/cv_stable-diffusion_paint-by-example.md"
repository: iic/cv_stable-diffusion_paint-by-example
chinese_name: "图像示例替换"
last_updated: 2023-11-30
license: "Apache License 2.0"
pipeline_tag: image-paintbyexample
tasks:
  - image-paintbyexample
model_type:
  - Stablediffusion-Paintbyexample
library_name:
  - pytorch
frameworks:
  - pytorch
domain:
  - cv
downloads: 4746
stars: 48
tags:
  - Alibaba
  - "Paint by Example"
  - "Stable Diffusion"
---

# cv_stable-diffusion_paint-by-example

> cv_stable-diffusion_paint-by-example - iic 在 ModelScope 开源的模型。给定一张原图和对应的mask区域，将给定的参考图自适应地替换上去

iic/cv_stable-diffusion_paint-by-example 是 ModelScope 魔搭社区上的image-paintbyexample模型，采用 Apache License 2.0 许可。

- **Repository**: iic/cv_stable-diffusion_paint-by-example
- **License**: Apache License 2.0
- **Tasks**: image-paintbyexample
- **Tags**: Alibaba, Paint by Example, Stable Diffusion
- **Downloads**: 4746
- **Stars**: 48
- **Last updated**: 2023-11-30

Source: https://www.modelscope.cn/models/iic/cv_stable-diffusion_paint-by-example

---

# Paint by Example 图像示例替换

本模型选自Paint by Example， 根据示例图片对原始图片的mask区域进行自适应地替换。

<img src="teaser.png" width=70% />

## 模型描述

Paint by Example 是基于stable diffusion 模型的一个图像编辑算法，根据示例图片对原始图片的mask区域进行自适应地替换。

## 期望模型使用方式以及适用范围

本模型适用范围为室外自然场景；

### 如何使用

在ModelScope框架上，提供输入图片，即可通过简单的Pipeline调用来使用。

#### 环境安装

安装好基础modelscope环境后，安装paint-ldm：

pip install paint-ldm -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html


#### 代码范例


- 推理(仅支持GPU)：


```python
from PIL import Image
import cv2
from modelscope.outputs import OutputKeys
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

input_location = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_paint_by_example/image/example_1.png'
input_mask_location = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_paint_by_example/mask/example_1.png'
reference_location = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_paint_by_example/reference/example_1.jpg'
input = {
        'img':input_location,
        'mask':input_mask_location,
        'reference':reference_location,
}

paintbyexample = pipeline(Tasks.image_paintbyexample, model='damo/cv_stable-diffusion_paint-by-example')
result = paintbyexample(input)
vis_img = result[OutputKeys.OUTPUT_IMG]
cv2.imwrite("result.png", vis_img)
```


### 模型局限性以及可能的偏差

- 人脸图片暂不支持
- 当前版本在python 3.8环境测试通过，其他环境下可用性待测试
- 当前版本fine-tune在cpu和单机单gpu环境测试通过，单机多gpu等其他环境待测试

## 训练数据介绍

- [ImageNet](https://www.image-net.org/)：包括1M训练数据




## 引用
如果你觉得这个该模型对有所帮助，请考虑引用下面的相关的论文：

```BibTeX
@article{yang2022paint,
  title={Paint by Example: Exemplar-based Image Editing with Diffusion Models},
  author={Binxin Yang and Shuyang Gu and Bo Zhang and Ting Zhang and Xuejin Chen and Xiaoyan Sun and Dong Chen and Fang Wen},
  journal={arXiv preprint arXiv:2211.13227},
  year={2022}
}
```