---
title: VL-RewardBench
canonical_url: "https://www.modelscope.cn/datasets/MMInstruction/VL-RewardBench"
md_url: "https://www.modelscope.cn/datasets/MMInstruction/VL-RewardBench.md"
repository: MMInstruction/VL-RewardBench
last_updated: 2025-05-19
license: "Apache License 2.0"
storage_size: "282 MB"
downloads: 1223
stars: 2
---

# VL-RewardBench

> VL-RewardBench - MMInstruction 在 ModelScope 开源的数据集。Dataset Card for VLRewardBench

MMInstruction/VL-RewardBench 是 ModelScope 魔搭社区上的数据集，存储大小 282 MB，采用 Apache License 2.0 许可。

- **Repository**: MMInstruction/VL-RewardBench
- **License**: Apache License 2.0
- **Storage size**: 282 MB
- **Downloads**: 1223
- **Stars**: 2
- **Last updated**: 2025-05-19

Source: https://www.modelscope.cn/datasets/MMInstruction/VL-RewardBench

---

# Dataset Card for VLRewardBench

Project Page:

https://vl-rewardbench.github.io

## Dataset Summary
VLRewardBench is a comprehensive benchmark designed to evaluate vision-language generative reward models (VL-GenRMs) across visual perception, hallucination detection, and reasoning tasks. The benchmark contains 1,250 high-quality examples specifically curated to probe model limitations.

## Dataset Structure

Each instance consists of multimodal queries spanning three key domains:

- General multimodal queries from real users
- Visual hallucination detection tasks
- Multimodal knowledge and mathematical reasoning

### Data Fields

Key fields:

- `id`: instance id

- `query`: text query of the multimodal prompt

- `image`: image input of the multimodal prompt

- `response`: list of two candidate responses generated by models;

- `human_ranking`: rank of the two responses `[0, 1]` denotes the first one is preferred; `[1, 0]` denotes the second one is better;

- `models`: the corresponding models generating the response. useful for instances from `wildvision` subset

- `query_source` : source dataset for the instance :

  - WildVision
  - POVID
  - RLAIF-V
  - RLHF-V
  - MMMU-Pro
  - MathVerse



## Annotations
- Small LVLMs were used to filter challenging samples
- Strong commercial models generated responses with explicit reasoning paths
- GPT-4o performed quality assessment
- All preference labels underwent human verification


## Usage
Intended Uses
The dataset is intended for research use only, specifically for:

- Evaluating and improving vision-language reward models
- Studying model limitations in visual perception and reasoning
- Developing better multimodal AI systems

## License
Research use only. Usage is restricted by the license agreements of GPT-4o and Claude.


## Citation Information

```bibtex
@article{VLRewardBench,
  title={VLRewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models},
  author={Lei Li and Yuancheng Wei and Zhihui Xie and Xuqing Yang and Yifan Song and Peiyi Wang and Chenxin An and Tianyu Liu and Sujian Li and Bill Yuchen Lin and Lingpeng Kong and Qi Liu},
  year={2024},
  journal={arXiv preprint arXiv:2411.17451}
}
```