---
title: LongPIBench
canonical_url: "https://www.modelscope.cn/datasets/evalscope/LongPIBench"
md_url: "https://www.modelscope.cn/datasets/evalscope/LongPIBench.md"
repository: evalscope/LongPIBench
last_updated: 2026-10-26
license: mit
storage_size: "9.7 MB"
downloads: 59
stars: 0
---

# LongPIBench

> LongPIBench - evalscope 在 ModelScope 开源的数据集。LongPIBench is a benchmark release containing 100 linked examples. Each example is identified by an integer from 0 through 99 and contains:

evalscope/LongPIBench 是 ModelScope 魔搭社区上的数据集，存储大小 9.7 MB，采用 mit 许可。

- **Repository**: evalscope/LongPIBench
- **License**: mit
- **Storage size**: 9.7 MB
- **Downloads**: 59
- **Stars**: 0
- **Last updated**: 2026-10-26

Source: https://www.modelscope.cn/datasets/evalscope/LongPIBench

---

# LongPIBench

LongPIBench is a benchmark release containing 100 linked examples. Each example
is identified by an integer from `0` through `99` and contains:

- a complete multi-file LaTeX paper project in `papers/<id>/`;
- structured personal-profile data in `person_info/<id>.json`;
- an email exchange and attachment text in `emails/<id>.json`; and
- a before/after programming task in `code_changes/<id>.json`.

The directory structure is part of the dataset. In particular, each paper is a
raw LaTeX project whose PDF is intended to be generated dynamically by the
benchmark code. The repository therefore preserves source files instead of
flattening the examples into a CSV or Parquet table.

## Download

Install the lightweight Hub client:

```bash
pip install huggingface_hub
```

Download the complete, version-pinned dataset:

```python
from huggingface_hub import snapshot_download

dataset_root = snapshot_download(
    repo_id="RainWatcher/LongPIBench",
    repo_type="dataset",
    revision="v1.0.0",
    local_dir="LongPIBench",
)
print(dataset_root)
```

To download a single linked example:

```python
from huggingface_hub import snapshot_download

example_id = 0
snapshot_download(
    repo_id="RainWatcher/LongPIBench",
    repo_type="dataset",
    revision="v1.0.0",
    local_dir="LongPIBench",
    allow_patterns=[
        f"papers/{example_id}/*",
        f"person_info/{example_id}.json",
        f"emails/{example_id}.json",
        f"code_changes/{example_id}.json",
        "manifest.jsonl",
        "checksums.sha256",
        "README.md",
    ],
)
```

The lightweight manifest can also be inspected with `datasets`:

```python
from datasets import load_dataset

manifest = load_dataset(
    "RainWatcher/LongPIBench",
    data_files="manifest.jsonl",
    split="test",
    revision="v1.0.0",
)
print(manifest[0])
```

`load_dataset()` loads the index, not the complete LaTeX project trees. Use
`snapshot_download()` when running experiments.

## Repository layout

```text
LongPIBench/
├── manifest.jsonl
├── checksums.sha256
├── papers/
│   └── <id>/
│       ├── paper.tex
│       ├── abstract.tex
│       ├── intro.tex
│       ├── rw.tex
│       ├── method.tex
│       ├── eval.tex
│       ├── dl.tex
│       ├── conclusion.tex
│       ├── reference.bib
│       ├── iclr2024_conference.sty
│       └── iclr2024_conference.bst
├── person_info/<id>.json
├── emails/<id>.json
└── code_changes/<id>.json
```

Empty `build/` directories are intentionally not stored. Code that compiles a
paper should create the directory first.

## Compiling a paper

A typical compilation command is:

```bash
cd papers/0
mkdir -p build
latexmk -pdf -interaction=nonstopmode -halt-on-error \
  -outdir=build paper.tex
```

The projects use a broad set of standard LaTeX packages. A full TeX Live
installation is recommended for reproducible execution. Compilation should be
performed in an isolated environment, without unrestricted shell escape or
access to sensitive host files.

## Integrity and versioning

`checksums.sha256` contains a SHA-256 digest for every released source-data
file. Verify a checkout from the repository root with:

```bash
shasum -a 256 -c checksums.sha256
```

Experiments should cite and download the immutable `v1.0.0` tag rather than the
moving `main` branch.

## Data notes

- All four components contain the same 100 IDs (`0` through `99`).
- Every paper directory contains the same eleven-file project layout.
- Generated PDFs and LaTeX build artifacts are excluded.
- Some examples contain names, organizations, email addresses, or other
  person-like details as benchmark content. They must not be interpreted as
  verified real-world identities or contact information.
- LaTeX and code content should be treated as untrusted input when evaluated.

## Third-party files

The paper projects include copies of the ICLR 2024 LaTeX style and an associated
BibTeX style. The BibTeX style states that it is distributed under the LaTeX
Project Public License. These third-party template files remain subject to
their respective upstream terms and are not relicensed by the dataset's main
license.

## License and citation

Except for third-party files described above, LongPIBench is released under the
MIT License. See `LICENSE` for the full text.

If you use LongPIBench, please cite the associated paper. Formal BibTeX metadata
will be added when the paper's public bibliographic record is available.