---
title: R-GroundBench
canonical_url: "https://www.modelscope.cn/datasets/Latent-Recast-Research/R-GroundBench"
md_url: "https://www.modelscope.cn/datasets/Latent-Recast-Research/R-GroundBench.md"
repository: Latent-Recast-Research/R-GroundBench
last_updated: 2026-09-29
license: other
storage_size: "208 MB"
downloads: 391
stars: 0
---

# R-GroundBench

> R-GroundBench - Latent-Recast-Research 在 ModelScope 开源的数据集。R-GroundBench is a diagnostic benchmark for evaluating R-group grounding in Markush molecular editing. It tests whether a model can connect R-group placeholders in a Markush scaffold to concrete…

Latent-Recast-Research/R-GroundBench 是 ModelScope 魔搭社区上的数据集，存储大小 208 MB，采用 other 许可。

- **Repository**: Latent-Recast-Research/R-GroundBench
- **License**: other
- **Storage size**: 208 MB
- **Downloads**: 391
- **Stars**: 0
- **Last updated**: 2026-09-29

Source: https://www.modelscope.cn/datasets/Latent-Recast-Research/R-GroundBench

---

# R-GroundBench

R-GroundBench is a diagnostic benchmark for evaluating R-group grounding in Markush molecular editing. It tests whether a model can connect R-group placeholders in a Markush scaffold to concrete substituents under natural-language instructions or property questions.

The release contains two tracks: multiple-choice visual question answering (VQA) and open-ended molecular generation. Every configuration contains a single `test` split. The benchmark records, their ordering, and the PNG assets are consolidated from the original VQA and Generation releases without regenerating questions or images.

## Configurations and verified sizes

| Track | Configuration | Rows |
|---|---|---:|
| VQA | `vqa_easy_basic` | 940 |
| VQA | `vqa_easy_advanced` | 634 |
| VQA | `vqa_medium_basic` | 940 |
| VQA | `vqa_medium_advanced` | 634 |
| VQA | `vqa_hard_basic` | 940 |
| VQA | `vqa_hard_advanced` | 634 |
| Generation | `generation_easy` | 500 |
| Generation | `generation_hard` | 500 |

The six VQA configurations contain 4,722 questions in total. The two Generation configurations contain 1,000 examples in total.

## VQA track

The VQA track presents a Markush scaffold, an instruction or property question, and four candidate answers. The expected answer is `A`, `B`, `C`, or `D`.

- **Basic** tasks test direct R-group substitution: localize the requested R-group, interpret the substituent instruction, and select the edited molecule.
- **Advanced** tasks test property-guided candidate selection using physicochemical, biological-activity, or indication/mechanism information.
- **Easy, Medium, and Hard** vary distractor construction and instruction style. Easy Basic uses cross-scaffold distractors, Medium Basic uses same-scaffold distractors, and Hard Basic uses high-similarity same-scaffold distractors with descriptive instructions. Advanced configurations follow the corresponding direct-to-descriptive difficulty progression.
- **NOTA robustness:** Easy and Medium Basic include “None of the above” cases. The `is_none_of_above`, `nova_answer_is_d`, and `nova_type` fields preserve the associated annotations.

Where supported by a model, the benchmark can be evaluated in four modality settings:

| Setting | Scaffold input | Candidate representation |
|---|---|---|
| `img_img` | Markush PNG | molecule PNGs |
| `img_smi` | Markush PNG | SMILES strings |
| `smi_img` | E-SMILES text | molecule PNGs |
| `smi_smi` | E-SMILES text | SMILES strings |

## Generation track

The Generation track supplies a Markush scaffold and a natural-language R-group substitution instruction. A model generates the complete edited molecule as SMILES without candidate options.

- `generation_easy` uses direct substituent naming.
- `generation_hard` uses descriptive paraphrases while retaining the same open-ended SMILES output format.

## Loading

```python
from pathlib import Path

from datasets import load_dataset
from modelscope import snapshot_download

config = "vqa_easy_basic"
root = Path(
    snapshot_download(
        repo_id="Latent-Recast-Research/R-GroundBench",
        repo_type="dataset",
    )
)
test = load_dataset(
    "parquet",
    data_files={
        "test": str(root / "data" / config / "test-00000-of-00001.parquet")
    },
)["test"]

# Image columns contain paths relative to the downloaded repository root.
original_image = root / test[0]["original_image_path"]
```

ModelScope does not map Hugging Face multi-config card metadata to a single
native loader argument. The eight logical configurations are therefore kept as
explicit directories under `data/<config>/`, each with one `test` Parquet file.
This preserves the exact validated Parquet bytes, row order, and image paths.

The same release is also available through Hugging Face's config-aware loader:

```python
from datasets import load_dataset

test = load_dataset(
    "latent-recast-research/R-GroundBench",
    "generation_hard",
    split="test",
)
```

## Important fields

### VQA

- `split`: original difficulty/task split name.
- `original_esmiles`: Markush scaffold in E-SMILES format.
- `original_image_path`: repository-relative Markush scaffold PNG path.
- `instruction`, `question`, and `property`: task text and property metadata where applicable.
- `correct_answer`: correct option label (`A`, `B`, `C`, or `D`).
- `correct_smiles`: SMILES for the correct molecule.
- `correct_prop_value` and `prop_values`: property values where applicable.
- `option_A_smiles` through `option_D_smiles`: candidate representations.
- `option_A_image` through `option_D_image`: repository-relative candidate PNG paths. An empty option-D image is retained for applicable NOTA records.
- `is_none_of_above`, `nova_answer_is_d`, and `nova_type`: NOTA annotations.
- Hard configurations may include analysis metadata such as `trick`, `n_rgroups`, and `is_same_group`.

### Generation

- `difficulty`: `easy` or `hard`.
- `original_esmiles`: input Markush scaffold in E-SMILES format.
- `original_image_path`: repository-relative scaffold PNG path.
- `instruction`: R-group substitution instruction.
- `correct_smiles`: ground-truth edited molecule SMILES.
- `scaffold_smiles_clean`: scaffold used by scaffold-preservation evaluation.
- `output_image_path`: repository-relative ground-truth molecule PNG path.
- Hard examples additionally retain `trick` and `n_rgroups` metadata.

E-SMILES is a Markush-aware extension of standard SMILES. Annotation tokens occur after a `<sep>` delimiter and should be removed before passing the scaffold to software that expects ordinary SMILES.

## Image paths

Image columns are deliberately stored as strings, not Hugging Face `Image` features. Each non-empty value is a POSIX-style path relative to this dataset repository, under `images/`. The release retains source-grouped canonical paths where possible, stores content-identical PNGs only once, and preserves the original PNG bytes exactly.

To download a referenced asset:

```python
from huggingface_hub import hf_hub_download

row = test[0]
local_png = hf_hub_download(
    repo_id="latent-recast-research/R-GroundBench",
    repo_type="dataset",
    filename=row["original_image_path"],
)
```

## Evaluation overview

VQA is evaluated with exact multiple-choice accuracy after parsing the model response as `A`, `B`, `C`, or `D`.

Generation evaluation reports:

- validity: whether the generated output is parseable as a molecule;
- Exact Match: canonical-SMILES equality with the ground truth;
- Tanimoto similarity: ECFP4 fingerprint similarity to the ground truth;
- Scaffold Match: whether the generated molecule preserves the Markush scaffold under the benchmark’s scaffold-matching criterion.

## Project links

- **Paper:** link to be added after the public paper URL is confirmed.
- **Homepage:** [https://latent-recast-research.github.io/R-GroundBench/](https://latent-recast-research.github.io/R-GroundBench/)
- **GitHub:** [https://github.com/CrisYing/R-GROUNDBENCH](https://github.com/CrisYing/R-GROUNDBENCH)
- **Hugging Face:** [https://huggingface.co/datasets/latent-recast-research/R-GroundBench](https://huggingface.co/datasets/latent-recast-research/R-GroundBench)

## License and source records

This benchmark is released for research use. Source molecular records remain subject to their original licensing terms.

## Citation

Citation metadata will be added when the paper’s final public bibliographic information is confirmed.