---
title: EditReward-Bench
canonical_url: "https://www.modelscope.cn/datasets/TIGER-Lab/EditReward-Bench"
md_url: "https://www.modelscope.cn/datasets/TIGER-Lab/EditReward-Bench.md"
repository: TIGER-Lab/EditReward-Bench
last_updated: 2025-12-01
license: apache-2.0
storage_size: "2.6 GB"
downloads: 276
stars: 0
---
# EditReward-Bench
> EditReward-Bench - TIGER-Lab 在 ModelScope 开源的数据集。EditReward-Bench: A Human-Aligned Reward Model Benchmark for Instruction-Guided Image Editing
TIGER-Lab/EditReward-Bench 是 ModelScope 魔搭社区上的数据集，存储大小 2.6 GB，采用 apache-2.0 许可。
- **Repository**: TIGER-Lab/EditReward-Bench
- **License**: apache-2.0
- **Storage size**: 2.6 GB
- **Downloads**: 276
- **Stars**: 0
- **Last updated**: 2025-12-01
Source: https://www.modelscope.cn/datasets/TIGER-Lab/EditReward-Bench
---
# EditReward-Bench: A Human-Aligned Reward Model Benchmark for Instruction-Guided Image Editing
[](https://tiger-ai-lab.github.io/EditReward/)
[](https://arxiv.org/abs/2509.26346)
[](https://huggingface.co/collections/TIGER-Lab/editreward-68ddf026ef9eb1510458abc6)
[](https://huggingface.co/datasets/TIGER-Lab/EditReward-Data)
[](https://github.com/TIGER-AI-Lab/EditReward)
This repository contains **EditReward-Bench**, a new benchmark introduced in the paper [EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing](https://huggingface.co/papers/2509.26346).
## Introduction
Recent advances in image editing with natural language instructions have shown promising progress, particularly with closed-source models. However, open-source models often lag due to the lack of a reliable reward model for scaling up high-quality synthetic training data. To address this critical bottleneck, the paper introduces **EditReward**, a human-aligned reward model trained with a new large-scale human preference dataset called **EditReward-Data** (over 200K preference pairs).
To rigorously assess **EditReward** and future models, we also introduce **EditReward-Bench**, a new benchmark built upon high-quality annotations, which includes more difficult multi-way preference prediction tasks. EditReward demonstrates superior alignment with human preferences in instruction-guided image editing tasks, achieving state-of-the-art human correlation on established benchmarks and on EditReward-Bench itself.
* **Paper**: https://huggingface.co/papers/2509.26346
* **Project Page**: https://tiger-ai-lab.github.io/EditReward
* **Code**: https://github.com/TIGER-AI-Lab/EditReward
### Dataset Statistics
| Dataset | Original Samples | Pairwise Comparisons | File Size |
|---------|-----------------|---------------------|-----------|
| 2-pair | 133 | 133 | ~333 MB |
| 3-pair | 150 | 450 | ~833 MB |
| 4-pair | 148 | 888 | ~1.5 GB |
| **Total** | **431** | **1471** | **~2.7 GB** |
- **Total unique instructions**: 431
- **Score range**: 2.0 - 8.0 (Overall score = Instruction Following + Visual Quality)
- **Average score difference**: 2.23
- **Total images**: 1,739 (all embedded in parquet files)
## Download EditReward-Bench
You can download the **EditReward-Bench** dataset using the Hugging Face CLI:
```bash
huggingface-cli download --repo-type dataset TIGER-Lab/EditReward-Bench --local-dir /your-local-dataset-path
```
## Usage
Please refer to the github code for evaluation! The EditReward-bench's evaluation is available at [EditReward/evaluate](https://github.com/TIGER-AI-Lab/EditReward/tree/main/EditReward/evaluate).
## Benchmark Results: Alignment with Humans
The following tables present the experimental results from the paper, showcasing various models' alignment with human preferences on established benchmarks and on **EditReward-Bench**.
Experimental Results: Alignment with Humans
| Method | GenAI-Bench | AURORA-Bench | ImagenHub | EditReward-Bench (Overall) |
| :--- | :--- | :--- | :--- | :--- |
| Random | 25.90 | 33.43 | -- | 13.84 |
| Human-to-Human | -- | -- | 41.84 | -- |
| ***Proprietary Models*** | | | | |
| GPT-4o | 53.54 | 50.81 | 38.21 | 28.31 |
| GPT-5 | 59.61 | 47.27 | 40.85 | 37.81 |
| Gemini-2.0-Flash | 53.32 | 44.31 | 23.69 | 33.47 |
| Gemini-2.5-Flash | 57.01 | 47.63 | **41.62** | 38.02 |
| ***Open-Source VLMs*** | | | | |
| Qwen2.5-VL-3B-Inst | 42.76 | 30.69 | -2.54 | 26.86 |
| Qwen2.5-VL-7B-Inst | 40.48 | 38.62 | 18.59 | 29.75 |
| Qwen2.5-VL-32B-Inst | 39.28 | 37.06 | 26.87 | 28.72 |
| MiMo-VL-7B-SFT-2508 | 57.89 | 30.43 | 22.14 | 31.19 |
| ADIEE | 59.96 | 55.56 | 34.50 | -- |
| ***Reward Models (Ours)*** | | | | |
| EditReward (on Qwen2.5-VL-7B) | 63.97 | 59.50 | 36.18 | 36.78 |
| EditReward (on MiMo-VL-7B) | **65.72** | **63.62** | 35.20 | **38.42** |
---
EditReward-Bench Results (K-way preference prediction)
| Method | EditReward-Bench (K=2) | EditReward-Bench (K=3) | EditReward-Bench (K=4) | EditReward-Bench (Overall) |
| :--- | :--- | :--- | :--- | :--- |
| Random | 25.81 | 11.33 | 1.35 | 13.84 |
| Human-to-Human | -- | -- | -- | -- |
| ***Proprietary Models*** | | | | |
| GPT-4o | 45.69 | 27.33 | 7.31 | 28.31 |
| GPT-5 | 57.53 | 38.51 | 12.84 | 37.81 |
| Gemini-2.0-Flash | 52.43 | 33.33 | **13.51** | 33.47 |
| Gemini-2.5-Flash | **58.61** | 39.86 | 12.16 | 38.02 |
| ***Open-Source VLMs*** | | | | |
| Qwen2.5-VL-3B-Inst | 51.07 | 20.27 | 2.71 | 26.86 |
| Qwen2.5-VL-7B-Inst | 52.69 | 24.67 | 3.38 | 29.75 |
| Qwen2.5-VL-32B-Inst | 50.54 | 25.27 | 4.05 | 28.72 |
| MiMo-VL-7B-SFT-2508 | 49.46 | 30.41 | 9.46 | 31.19 |
| ADIEE | -- | -- | -- | -- |
| ***Reward Models (Ours)*** | | | | |
| EditReward (on Qwen2.5-VL-7B) | 56.99 | 36.00 | 10.81 | 36.78 |
| EditReward (on MiMo-VL-7B) | 56.45 | **42.67** | 11.49 | **38.42** |
## Citation
Please kindly cite our paper if you use our code, data, models or results:
```bibtex
@article{wu2025editreward,
title={EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing},
author={Wu, Keming and Jiang, Sicong and Ku, Max and Nie, Ping and Liu, Minghao and Chen, Wenhu},
journal={arXiv preprint arXiv:2509.26346},
year={2025}
}
```