---
title: RPC-Bench
canonical_url: "https://www.modelscope.cn/datasets/ZhipuAI/RPC-Bench"
md_url: "https://www.modelscope.cn/datasets/ZhipuAI/RPC-Bench.md"
repository: ZhipuAI/RPC-Bench
chinese_name: RPC-Bench
last_updated: 2026-06-26
license: other
storage_size: "93 GB"
downloads: 2278
stars: 0
---

# RPC-Bench

> RPC-Bench - ZhipuAI 在 ModelScope 开源的数据集。RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

ZhipuAI/RPC-Bench 是 ModelScope 魔搭社区上的数据集，存储大小 93 GB，采用 other 许可。

- **Repository**: ZhipuAI/RPC-Bench
- **License**: other
- **Storage size**: 93 GB
- **Downloads**: 2278
- **Stars**: 0
- **Last updated**: 2026-06-26

Source: https://www.modelscope.cn/datasets/ZhipuAI/RPC-Bench

---

<div align="center">

# RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

</div>

<p align="center">
    🌐 <a href="https://rpc-bench.github.io/" target="_blank">Project Page</a> •
    💻 <a href="https://github.com/zai-org/RPC-Bench" target="_blank">GitHub</a> •
    📖 <a href="https://arxiv.org/abs/2601.14289" target="_blank">Paper</a>
</p>

<div align="center">
    <img src="assets/pipeline.png" width="100%" />
</div>

RPC-Bench is a fine-grained benchmark for research paper comprehension. It is built from review-rebuttal exchanges of high-quality academic papers and supports both text-only and visual evaluation through complementary paper representations.

## Data Structure

RPC-Bench is organized into `train`, `dev`, and `test` subsets. Split assignments are recorded in `manifest.jsonl`, and the original split JSON files are provided in `split_metadata/` (`train.json`, `dev.json`, `test.json`).

`md/` contains Markdown files parsed from each paper by MinerU. These files provide the text input for LLM-oriented evaluation.

`parse/` contains the full MinerU parsing outputs for each paper, including structured layout and content artifacts.

`pdf/` contains the original paper PDFs.

`vlm/` contains page images rendered from the PDFs with PyMuPDF at 200 DPI for VLM-oriented evaluation.

```text
RPC-Bench/
├── README.md
├── manifest.jsonl
├── split_metadata/
│   ├── train.json
│   ├── dev.json
│   └── test.json
├── parse/
│   ├── train/
│   │   └── <paper_id>/
│   ├── dev/
│   │   └── <paper_id>/
│   └── test/
│       └── <paper_id>/
├── md/
│   ├── train/
│   │   └── <paper_id>/
│   │       └── <paper_id>.md
│   ├── dev/
│   │   └── <paper_id>/
│   │       └── <paper_id>.md
│   └── test/
│       └── <paper_id>/
│           └── <paper_id>.md
├── pdf/
│   ├── train/
│   │   └── <paper_id>.pdf
│   ├── dev/
│   │   └── <paper_id>.pdf
│   └── test/
│       └── <paper_id>.pdf
└── vlm/
    ├── train/
    │   └── <paper_id>/
    ├── dev/
    │   └── <paper_id>/
    └── test/
        └── <paper_id>/
```

## Practical Uses

RPC-Bench can be used to try paper-centric systems that require broader document understanding rather than local snippet matching.

- Research paper comprehension: try models on full-paper understanding, including core concepts, methods, and experimental findings.
- Long-context evaluation: try whether longer context windows or long-context architectures improve document-level reasoning.
- Multimodal reasoning: try models that combine textual evidence with page-level figures, tables, and diagrams in the original PDF layout.
- RAG system diagnosis: try retrieval, chunking, and evidence-fusion strategies for paper-centric workflows beyond snippet-level retrieval accuracy.

## Citation

```bibtex
@article{chen2026rpc,
  title={RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension},
  author={Chen, Yelin and Zhang, Fanjin and Sun, Suping and Pang, Yunhe and Wang, Yuanchun and Song, Jian and Li, Xiaoyan and Hou, Lei and Zhao, Shu and Tang, Jie and others},
  journal={arXiv preprint arXiv:2601.14289},
  year={2026}
}
```