---
title: MiniMax-H3-Image-Songyu-LoRA
canonical_url: "https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Image-Songyu-LoRA"
md_url: "https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Image-Songyu-LoRA.md"
repository: mibei0804/MiniMax-H3-Image-Songyu-LoRA
chinese_name: "MiniMax-H3-Image-宋玉人物LoRA"
last_updated: 2026-11-17
license: "Apache License 2.0"
pipeline_tag: text-to-image-synthesis
tasks:
  - text-to-image-synthesis
base_model:
  - MiniMax/MiniMax-H3-Image
base_model_relation: adapter
parameters: 155.1M
tensor_type:
  - BF16
library_name:
  - pytorch
  - lora
  - safetensors
frameworks:
  - Pytorch
supports_inference: txt2img
downloads: 74
stars: 3
tags:
  - LoRA
  - MiniMax-H3-Image
  - text-to-image
  - character
---

# MiniMax-H3-Image-Songyu-LoRA

> MiniMax-H3-Image-Songyu-LoRA - mibei0804 在 ModelScope 开源的模型。MiniMax-H3-Image Songyu LoRA

mibei0804/MiniMax-H3-Image-Songyu-LoRA 是 ModelScope 魔搭社区上的 155.1M 参数text-to-image-synthesis模型，采用 Apache License 2.0 许可，基于 MiniMax/MiniMax-H3-Image 构建，并支持在线推理（txt2img）。

- **Repository**: mibei0804/MiniMax-H3-Image-Songyu-LoRA
- **License**: Apache License 2.0
- **Tasks**: text-to-image-synthesis
- **Parameters**: 155.1M
- **Base model**: MiniMax/MiniMax-H3-Image
- **Online inference**: txt2img
- **Tags**: LoRA, MiniMax-H3-Image, text-to-image, character
- **Downloads**: 74
- **Stars**: 3
- **Last updated**: 2026-11-17

Source: https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Image-Songyu-LoRA

---

# MiniMax-H3-Image Songyu LoRA

给 [MiniMax-H3-Image](https://www.modelscope.cn/models/MiniMax/MiniMax-H3-Image) 注入「宋玉」（《凡人修仙传》）角色身份的人物 LoRA。加载后，在 prompt 里写上触发词即可生成该角色的图像。

本模型同时是 [MiniMax-H3-Image-TrainingAdapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter)（DeCFG Training Adapter）方案的一个完整验证案例。

LoRA 权重文件：`model.safetensors`

## 触发词与 Prompt 写法

触发词：

```
宋玉 凡人修仙传,宋玉 (凡人修仙传)，
```

把触发词放在 prompt 开头，后面接正常的画面描述即可。完整示例：

```text
宋玉 凡人修仙传,宋玉 (凡人修仙传)，3D guochan xianxia CG animation style, high-precision CG rendering, Unreal Engine 5 quality, cinematic lighting. Half-body close shot of a beautiful young female cultivator standing in a peach blossom grove, one hand gently holding a flowering peach branch across her shoulder, the other hand relaxed at her side, turning her head toward the camera with clear open eyes and a soft amused half-smile. Pale-cyan and white immortal dress with sheer wide sleeves, white openwork crown and white wing-shaped hair ornaments, red cord necklace with a jade pendant. Pale pink blossoms fill the foreground and background as bokeh, a drift of petals caught in the air, warm late-afternoon sunlight filtering through the branches, honeyed rim light in her hair, shallow depth of field, delicate spring atmosphere.
```

## 快速开始

### 1. 安装 DiffSynth-Studio

```bash
pip install diffsynth
```

或从源码安装：

```bash
git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
pip install -e .
```

### 2. 推理

```python
import torch
from diffsynth.pipelines.minimax_h3_image import MiniMaxH3ImagePipeline, ModelConfig

pipe = MiniMaxH3ImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/text_encoder/model*.safetensors"),
        ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="transformer/model*.safetensors"),
        ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="pdd_fused_heads.safetensors"),
        ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    processor_config=ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="processor/"),
    vram_limit=torch.cuda.mem_get_info("cuda")[1] / (1024 ** 3) - 2,
)

pipe.load_lora(pipe.dit, ModelConfig(
    model_id="mibei0804/MiniMax-H3-Image-Songyu-LoRA",
    origin_file_pattern="model.safetensors",
))

prompt = (
    "宋玉 凡人修仙传,宋玉 (凡人修仙传)，3D guochan xianxia CG animation style, high-precision CG rendering, "
    "Unreal Engine 5 quality, cinematic lighting. Half-body close shot of a beautiful young female cultivator "
    "standing in a peach blossom grove, one hand gently holding a flowering peach branch across her shoulder, "
    "the other hand relaxed at her side, turning her head toward the camera with clear open eyes and a soft "
    "amused half-smile. Pale-cyan and white immortal dress with sheer wide sleeves, white openwork crown and "
    "white wing-shaped hair ornaments, red cord necklace with a jade pendant. Pale pink blossoms fill the "
    "foreground and background as bokeh, a drift of petals caught in the air, warm late-afternoon sunlight "
    "filtering through the branches, honeyed rim light in her hair, shallow depth of field, delicate spring atmosphere."
)

image = pipe(prompt=prompt, height=4096, width=4096, seed=0)
image.save("songyu.jpg")

# To unload the LoRA:
# pipe.clear_lora()
```

## 训练方法

### 使用 [MiniMax-H3-Image-TrainingAdapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter) 作为 preset lora

MiniMax-H3-Image 基座是经过 CFG 蒸馏的（6 步、无 CFG 采样），直接在上面做 LoRA 微调容易不稳定，或者破坏蒸馏后的 CFG-free 行为。[MiniMax-H3-Image-TrainingAdapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter) 在训练期间把蒸馏后的 DiT 临时拉回接近蒸馏前的状态，为新 LoRA 提供更好的优化面。

**Training Adapter 只在训练时用，推理时不要加载**——本仓库发布的 `model.safetensors` 是已经脱离 Training Adapter 的独立 LoRA，直接加载到原版 MiniMax-H3-Image 即可。

### 训练数据

8 张「宋玉」角色截图（大部分为 2560×1440），每张配一条以触发词开头的英文画面描述。

### 完整训练脚本

先下载 Training Adapter：

```bash
modelscope download --model DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter --local_dir ./models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter
```

**Stage 1：数据预处理**（缓存 prompt embedding 与 VAE latent，此阶段 DiT 被 offload，不需要 Training Adapter）

```bash
accelerate launch examples/minimax_h3_image/model_training/train.py \
  --dataset_base_path "" \
  --dataset_metadata_path "<your_dataset_dir>/metadata.jsonl" \
  --data_file_keys "image" \
  --max_pixels 4194304 \
  --dataset_repeat 1 \
  --model_id_with_origin_paths "MiniMax/MiniMax-H3:FL2VA/text_encoder/model*.safetensors,MiniMax/MiniMax-H3-Image:transformer/model*.safetensors,MiniMax/MiniMax-H3-Image:pdd_fused_heads.safetensors,MiniMax/MiniMax-H3-Image:vae/diffusion_pytorch_model.safetensors" \
  --learning_rate 1e-4 \
  --num_epochs 1 \
  --remove_prefix_in_ckpt "pipe.dit." \
  --output_path "./models/train/songyu_cache" \
  --lora_base_model "dit" \
  --lora_target_modules "attn.qkv_proj,attn.out_proj,mlp.fc1,mlp.fc2" \
  --lora_rank 32 \
  --use_gradient_checkpointing \
  --task "sft:data_process" \
  --offload_models "MiniMax/MiniMax-H3-Image:transformer/model*.safetensors,MiniMax/MiniMax-H3-Image:pdd_fused_heads.safetensors"
```

**Stage 2：训练 LoRA**

```bash
accelerate launch examples/minimax_h3_image/model_training/train.py \
  --dataset_base_path "./models/train/songyu_cache" \
  --data_file_keys "image" \
  --max_pixels 4194304 \
  --dataset_repeat 50 \
  --model_id_with_origin_paths "MiniMax/MiniMax-H3:FL2VA/text_encoder/model*.safetensors,MiniMax/MiniMax-H3-Image:transformer/model*.safetensors,MiniMax/MiniMax-H3-Image:pdd_fused_heads.safetensors,MiniMax/MiniMax-H3-Image:vae/diffusion_pytorch_model.safetensors" \
  --learning_rate 1e-4 \
  --num_epochs 20 \
  --remove_prefix_in_ckpt "pipe.dit." \
  --output_path "./models/train/songyu" \
  --lora_base_model "dit" \
  --lora_target_modules "attn.qkv_proj,attn.out_proj,mlp.fc1,mlp.fc2" \
  --lora_rank 32 \
  --use_gradient_checkpointing \
  --task "sft:train" \
  --offload_models "MiniMax/MiniMax-H3:FL2VA/text_encoder/model*.safetensors,MiniMax/MiniMax-H3-Image:vae/diffusion_pytorch_model.safetensors" \
  --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter/model.safetensors" \
  --preset_lora_model "dit"
```

本仓库发布的 `model.safetensors` 即 Stage 2 输出的 `epoch-9.safetensors`（训练 10 个 epoch）。

### Note

1. **如何加 Training Adapter**：在 Stage 2 的训练命令里加上 `--preset_lora_path <training_adapter>.safetensors --preset_lora_model dit`，训练框架会在训练开始前把 Training Adapter 融进 DiT。训练结束后只保留新训出的 LoRA，Training Adapter 不再需要，推理时也不要加载它。
2. **ComfyUI 格式的 DiT**：如果训练用的是 ComfyUI 格式（如 `Comfy-Org/MiniMax-H3-Image`）的 DiT，Training Adapter 请改用 `model_for_comfy_dit.safetensors`。
3. **分辨率**：`--max_pixels 4194304` 让每个样本最多约 400 万像素（约 2K），超过的图片会等比缩小。推荐使用 2K 及以上分辨率的图像训练，MiniMax-H3-Image 在小分辨率下训练效果不好。
4. **LoRA 注入位置**：`--lora_target_modules` 覆盖全部 50 个 DiT block 与 2 个 token-refiner block 的 `attn.qkv_proj`、`attn.out_proj`、`mlp.fc1`、`mlp.fc2`，rank 为 32。

## 效果示例

使用本仓库的 `model.safetensors`（`epoch-9`）生成，图像分辨率为 4096×4096。

<table>
  <tr>
    <td width="50%"><img src="assets/0.jpg" alt="桃花林" width="100%"></td>
    <td width="50%"><img src="assets/1.jpg" alt="御剑云海" width="100%"></td>
  </tr>
  <tr>
    <td width="50%"><img src="assets/2.jpg" alt="古城夜市" width="100%"></td>
    <td width="50%"><img src="assets/3.jpg" alt="月下药园" width="100%"></td>
  </tr>
</table>

## 相关链接

- 基座模型：[MiniMax/MiniMax-H3-Image](https://www.modelscope.cn/models/MiniMax/MiniMax-H3-Image)
- Training Adapter：[DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-TrainingAdapter)
- 训练框架：[DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)
