---
title: Visual-ERM
canonical_url: "https://www.modelscope.cn/models/Shanghai_AI_Laboratory/Visual-ERM"
md_url: "https://www.modelscope.cn/models/Shanghai_AI_Laboratory/Visual-ERM.md"
repository: Shanghai_AI_Laboratory/Visual-ERM
last_updated: 2026-03-25
license: apache-2.0
model_type:
- qwen3_vl
architectures:
- Qwen3VLForConditionalGeneration
base_model:
- Qwen/Qwen3-VL-8B-Instruct
base_model_relation: finetune
parameters: 8.8B
tensor_type:
- BF16
library_name:
- safetensors
- pytorch
frameworks:
- pytorch
downloads: 74
stars: 0
---
# Visual-ERM
> Visual-ERM - Shanghai_AI_Laboratory 在 ModelScope 开源的模型。Visual-ERM is a multimodal generative reward model for vision-to-code tasks. It evaluates outputs directly in the rendered visual space and produces fine-grained, interpretable, and task-agnostic…
Shanghai_AI_Laboratory/Visual-ERM 是 ModelScope 魔搭社区上的 8.8B 参数机器学习模型，采用 apache-2.0 许可，基于 Qwen/Qwen3-VL-8B-Instruct 构建。
- **Repository**: Shanghai_AI_Laboratory/Visual-ERM
- **License**: apache-2.0
- **Parameters**: 8.8B
- **Base model**: Qwen/Qwen3-VL-8B-Instruct
- **Downloads**: 74
- **Stars**: 0
- **Last updated**: 2026-03-25
Source: https://www.modelscope.cn/models/Shanghai_AI_Laboratory/Visual-ERM
---
# Visual-ERM
Visual-ERM is a **multimodal generative reward model** for **vision-to-code** tasks.
It evaluates outputs directly in the **rendered visual space** and produces **fine-grained**, **interpretable**, and **task-agnostic** discrepancy feedback for structured visual reconstruction.
📄 Paper |
💻 GitHub |
📊 VC-RewardBench
## Model Overview
Existing rewards for vision-to-code usually fall into two categories:
1. **Text-based rewards** such as edit distance or TEDS, which ignore important visual cues like layout, spacing, alignment, and style.
2. **Vision embedding rewards** such as DINO similarity, which are often coarse-grained and can be vulnerable to reward hacking.
Visual-ERM addresses this by directly comparing:
- the **ground-truth image**, and
- the **rendered image** produced from a model prediction,
and then generating **structured discrepancy annotations** that can be converted into reward signals or used for reflection-based refinement.
## What this model does
Visual-ERM is designed to judge whether a predicted result is **visually equivalent** to the target.
Given a pair of images, it can identify discrepancies such as:
- **category**
- **severity**
- **location**
- **description**
This makes Visual-ERM useful not only as a reward model for RL, but also as a **visual critic** for test-time reflection and revision.
## Supported Tasks
Visual-ERM is designed for structured visual reconstruction tasks, including:
- **Chart-to-Code**
- **Table-to-Markdown**
- **SVG-to-Code**
## Key Features
- **Visual-space reward modeling**
Evaluates predictions in rendered visual space instead of relying only on text matching or coarse embedding similarity.
- **Fine-grained and interpretable feedback**
Produces structured discrepancy annotations rather than a single black-box score.
- **Task-agnostic reward supervision**
A unified reward model that generalizes across multiple vision-to-code tasks.
- **Useful for both training and inference**
Can be used as a reward model in RL and as a visual critic during test-time refinement.
## VC-RewardBench
We also release **VisualCritic-RewardBench (VC-RewardBench)**, a benchmark for evaluating fine-grained image-to-image discrepancy judgment on structured visual data.
### Benchmark Features
- Covers **charts**, **tables**, and **SVGs**
- Contains **1,335** carefully curated instances
- Each instance includes:
- a ground-truth image
- a corrupted / rendered counterpart
- fine-grained discrepancy annotations
Dataset link:
https://huggingface.co/datasets/internlm/VC-RewardBench
## How to Use
Visual-ERM is fine-tuned from **Qwen/Qwen3-VL-8B-Instruct** and follows the same multimodal interface.
### Input
Visual-ERM takes as input:
- a **reference / ground-truth image**
- a **rendered prediction image**
- a **prompt** asking the model to identify fine-grained visual discrepancies
### Output
The model outputs structured discrepancy annotations, which can then be:
- converted into a scalar reward for RL
- used as feedback for reflection-and-revision
- evaluated directly on VC-RewardBench
A typical output format is:
```json
{
"errors": [
{
"category": "structure_error",
"severity": 3,
"location": "legend area",
"description": "The legend is placed outside the plot area in the prediction."
},
{
"category": "style_error",
"severity": 2,
"location": "bar colors",
"description": "The colors differ from those in the reference image."
}
]
}
```
### Inference / Evaluation / RL
For full inference scripts, RL training pipelines, evaluation code, and prompt templates, please refer to the official repository:
https://github.com/InternLM/Visual-ERM
## Intended Use
Visual-ERM is intended for:
- **reward modeling** in vision-to-code RL pipelines
- **visual discrepancy judgment** between target and predicted renderings
- **reflection-based refinement** at inference time
- **research on visual reward modeling** and multimodal RL
## Citation
If you find this model useful, please consider citing:
```bibtex
@article{liu2026visual,
title={Visual-ERM: Reward Modeling for Visual Equivalence},
author={Liu, Ziyu and Ding, Shengyuan and Fang, Xinyu and Dai, Xuanlang and Yang, Penghui and Liang, Jianze and Wang, Jiaqi and Chen, Kai and Lin, Dahua and Zang, Yuhang},
journal={arXiv preprint arXiv:2603.13224},
year={2026}
}
```
## Contact
If you are interested in **visual reward modeling**, **vision-to-code**, or **reinforcement learning for multimodal models**, feel free to reach out.