---
title: cv_masactrl_image-editing
canonical_url: "https://www.modelscope.cn/models/iic/cv_masactrl_image-editing"
md_url: "https://www.modelscope.cn/models/iic/cv_masactrl_image-editing.md"
repository: iic/cv_masactrl_image-editing
chinese_name: "MasaCtrl图像合成与编辑"
last_updated: 2023-08-16
license: "Apache License 2.0"
parameters: 1.4B
tensor_type:
  - F32
  - I64
library_name:
  - safetensors
  - diffusers
  - pytorch
frameworks:
  - Pytorch
downloads: 348
stars: 2
---

# cv_masactrl_image-editing

> cv_masactrl_image-editing - iic 在 ModelScope 开源的模型。一种用于非刚性一致图像合成与编辑的方法。关键思想是将源图像中的内容迁移到由文本或其他附加控制生成的布局上。

iic/cv_masactrl_image-editing 是 ModelScope 魔搭社区上的 1.4B 参数机器学习模型，采用 Apache License 2.0 许可。

- **Repository**: iic/cv_masactrl_image-editing
- **License**: Apache License 2.0
- **Parameters**: 1.4B
- **Downloads**: 348
- **Stars**: 2
- **Last updated**: 2023-08-16

Source: https://www.modelscope.cn/models/iic/cv_masactrl_image-editing

---

## MasaCtrl: Tuning-free <span style="text-decoration: underline"><font color="Tomato">M</font></span>utu<span style="text-decoration: underline"><font color="Tomato">a</font></span>l <span style="text-decoration: underline"><font color="Tomato">S</font></span>elf-<span style="text-decoration: underline"><font color="Tomato">A</font></span>ttention <span style="text-decoration: underline"><font color="Tomato">Control</font></span> for Consistent Image Synthesis and Editing

<div align="center">
<img src="https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/public/ModelScope/modelcard/masactrl/overview.gif">
<br />
<i> MasaCtrl enables performing various consistent non-rigid image synthesis and editing without fine-tuning and optimization. </i>
</div>

### 模型描述
MasaCtrl，一个针对非刚性图像合成与编辑的免训练方法。主要方法是结合*源图像*的`内容`和*文本题词、附加控制*生成的`布局`来生成期望的图像。

a tuning-free method for non-rigid consistent image synthesis and editing. The key idea is to combine the `contents` from the *source image* and the `layout` synthesized from *text prompt and additional controls* into the desired synthesized or edited image, with **Mutual Self-Attention Control**.

<div align="center">
<img src="https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/public/ModelScope/modelcard/masactrl/corgi_all_step4_layer10.png">
<br />
<i> (a) source; (b) reconstructed_source; (c) without masactrl; (d) masactrl</i>
<br />
<i> a photo of a running corgi </i>
<br />
<img src="https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/public/ModelScope/modelcard/masactrl/man_all_step4_layer10.png">
<br />
<i> (a) source; (b) reconstructed_source; (c) without masactrl; (d) masactrl </i>
<br />
<i> a photo of a person, black t-shirt, raising right hand </i>
</div>

### 期望模型使用方式以及使用范围

#### 如何使用
##### 代码范例
```python
import cv2
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.utils.test_utils import test_level

prompts = [
    "",                           # source prompt
    "a photo of a running corgi"  # target prompt
]
output_image_path = './result.png'
img = 'https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/public/ModelScope/test/images/corgi.jpg'
input = {'img': img, 'prompts': prompts}

pipe = pipeline(Tasks.image_editing,
    model='damo/cv_masactrl_image-editing')

output = pipe(input)['output_img']
cv2.imwrite(output_image_path, output)
print('pipeline: the output image path is {}'.format(output_image_path))
```

### 相关论文以及引用信息
```bibtex
@misc{cao2023masactrl,
      title={MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing}, 
      author={Mingdeng Cao and Xintao Wang and Zhongang Qi and Ying Shan and Xiaohu Qie and Yinqiang Zheng},
      year={2023},
      eprint={2304.08465},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
    }
```

#### Clone with HTTP
```bash
 git clone https://www.modelscope.cn/damo/cv_masactrl_image-editing.git
```