---
title: flux-controlnet-canny
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/flux-controlnet-canny"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/flux-controlnet-canny.md"
repository: AI-ModelScope/flux-controlnet-canny
last_updated: 2024-08-14
license: other
pipeline_tag: text-to-image-synthesis
tasks:
  - text-to-image-synthesis
base_model:
  - black-forest-labs/FLUX.1-dev
base_model_relation: finetune
parameters: 743.8M
tensor_type:
  - BF16
library_name:
  - lora
  - diffusers
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 2598
stars: 2
tags:
  - Text-to-Image
  - ControlNet
  - "Stable Diffusion"
  - diffusers
---

# flux-controlnet-canny

> flux-controlnet-canny - AI-ModelScope 在 ModelScope 开源的模型。This repository provides a checkpoint with trained ControlNet Canny model for FLUX.1-dev model by Black Forest Labs ComfyUI

AI-ModelScope/flux-controlnet-canny 是 ModelScope 魔搭社区上的 743.8M 参数text-to-image-synthesis模型，采用 other 许可，基于 black-forest-labs/FLUX.1-dev 构建。

- **Repository**: AI-ModelScope/flux-controlnet-canny
- **License**: other
- **Tasks**: text-to-image-synthesis
- **Parameters**: 743.8M
- **Base model**: black-forest-labs/FLUX.1-dev
- **Tags**: Text-to-Image, ControlNet, Stable Diffusion, diffusers
- **Downloads**: 2598
- **Stars**: 2
- **Last updated**: 2024-08-14

Source: https://www.modelscope.cn/models/AI-ModelScope/flux-controlnet-canny

---

![ControlNet Canny for Flux](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/light/controlnet-canny-header-rev1.png?raw=true)

This repository provides a checkpoint with trained ControlNet Canny model for
[FLUX.1-dev model](https://huggingface.co/black-forest-labs/FLUX.1-dev) by Black Forest Labs
# ComfyUI

[See our github](https://github.com/XLabs-AI/x-flux-comfyui) for comfy ui workflows.
![Example Picture 1](https://github.com/XLabs-AI/x-flux-comfyui/blob/main/assets/image1.png?raw=true)
# Training details
[XLabs AI](https://github.com/XLabs-AI) team is happy to publish fune-tuning Flux scripts, including:
- **LoRA** 🔥
- **ControlNet** 🔥

[See our github](https://github.com/XLabs-AI/x-flux) for train script and train configs.

# Training dataset
Dataset has the following format for the training process:
```
├── images/
│    ├── 1.png
│    ├── 1.json
│    ├── 2.png
│    ├── 2.json
│    ├── ...
```
A .json file contains "caption" field with a text prompt.

# Inference

To test our checkpoints, use commands presented below.
```bash
python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a handsome viking man with white hair, cinematic, MM full HD"
```
![Example Picture 1](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/picture-3-rev1.png?raw=true)
```bash
python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a dark evil mysterius house with ghosts, cinematic, MM full HD"
```

![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/picture-2-rev1.png?raw=true)


# License
controlnet.safetensors falls under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/>