---
title: "VisualPRM: An Effective Process Reward Model for Multimodal Reasoning"
canonical_url: "https://www.modelscope.cn/papers/126372"
md_url: "https://www.modelscope.cn/papers/126372.md"
arxiv_id: 2503.10291
published: 2025-03-13
last_updated: 2025-03-13
authors:
  - "Weiyun Wang"
  - "Zhangwei Gao"
  - "Lianjie Chen"
  - "Zhe Chen"
  - "Jinguo Zhu"
  - "Xiangyu Zhao"
  - "Yangzhou Liu"
  - "Yue Cao"
  - "Shenglong Ye"
  - "Xizhou Zhu"
  - "Lewei Lu"
  - "Haodong Duan"
  - "Yu Qiao"
  - "Jifeng Dai"
  - "Wenhai Wang"
model_name: VisualPRM
model_developer: "上海人工智能实验室, 复旦大学, 上海交通大学, 清华大学, 南京大学, 香港中文大学, 商汤科技"
domain:
  - "计算机视觉"
  - "自然语言处理"
  - "深度学习"
type:
  - "计算机视觉"
  - "自然语言处理"
  - "深度学习"
  - "Computer Vision and Pattern Recognition (cs.CV)"
  - "Computation and Language (cs.CL)"
arxiv_url: "https://arxiv.org/abs/2503.10291"
pdf_url: "https://arxiv.org/pdf/2503.10291.pdf"
code_link: "https://internvl.github.io/blog/2025-03-13-VisualPRM/"
---

# VisualPRM: An Effective Process Reward Model for Multimodal Reasoning

> We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN)…

「VisualPRM: An Effective Process Reward Model for Multimodal Reasoning」是 ModelScope 魔搭社区收录的论文，arXiv 2503.10291，作者为 Weiyun Wang, Zhangwei Gao, Lianjie Chen et al.，发表于 2025-03-13，属于 计算机视觉、自然语言处理、深度学习 领域。

- **ArXiv**: 2503.10291
- **Published**: 2025-03-13
- **Authors**: Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, Lewei Lu, Haodong Duan, Yu Qiao, Jifeng Dai, Wenhai Wang
- **Model**: VisualPRM
- **Developer**: 上海人工智能实验室, 复旦大学, 上海交通大学, 清华大学, 南京大学, 香港中文大学, 商汤科技
- **Domain**: 计算机视觉, 自然语言处理, 深度学习
- **ArXiv URL**: https://arxiv.org/abs/2503.10291
- **PDF**: https://arxiv.org/pdf/2503.10291.pdf
- **Code**: https://internvl.github.io/blog/2025-03-13-VisualPRM/

Source: https://www.modelscope.cn/papers/126372

---

> VisualPRM：大幅提升多模态推理能力的过程奖励模型

## 摘要

本文提出了一种名为VisualPRM的多模态过程奖励模型（PRM），该模型具有80亿参数，旨在通过Best-of-N (BoN)评估策略提升多模态大语言模型（MLLMs）的推理能力。具体而言，VisualPRM能够改进三种类型的MLLMs和四种不同规模模型的推理性能，并在应用于InternVL2.5-78B时，在七个多模态推理基准测试中实现了5.9分的提升。实验结果表明，与结果奖励模型（ORMs）和自一致性方法相比，VisualPRM在BoN评估中的表现更为优越。

为支持VisualPRM的训练，作者构建了一个包含约40万个多模态过程监督数据的VisualPRM400K数据集，其中每个样本包括一张图像、一个问题、逐步解决方案以及每一步的正确性标注。此外，为了评估多模态PRMs，作者提出了VisualProcessBench基准测试，该基准包含2866个样本和26950个人工标注的逐步正确性标签，用于衡量PRMs和MLLMs检测多模态推理任务中错误步骤的能力。

通过广泛的实验验证，VisualPRM显著提升了MiniCPM-V2.6、QwenVL2.5-7B、InternVL2.5-8B和InternVL2.5-78B等模型在七个多模态推理基准上的整体性能，分别提高了8.0、3.7、8.4和5.9分。此外，实验还表明PRMs在BoN评估中始终优于ORMs和自一致性方法。这些成果不仅证明了VisualPRM的有效性，还填补了多模态PRMs和评估基准的空白，为未来的研究提供了重要参考。

## Abstract

We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies. Specifically, our model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. Experimental results show that our model exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation. To facilitate the training of multimodal PRMs, we construct a multimodal process supervision dataset VisualPRM400K using an automated data pipeline. For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels, to measure the abilities of PRMs to detect erroneous steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark are released in this https URL.