---
title: MedPIC-Bench
canonical_url: "https://www.modelscope.cn/datasets/ZhitianHou/MedPIC-Bench"
md_url: "https://www.modelscope.cn/datasets/ZhitianHou/MedPIC-Bench.md"
repository: ZhitianHou/MedPIC-Bench
last_updated: 2026-07-30
license: cc-by-4.0
downloads: 58
stars: 0
---

# MedPIC-Bench

> MedPIC-Bench - ZhitianHou 在 ModelScope 开源的数据集。MedPIC-Bench is an English multiple-choice benchmark for evaluating whether large language models apply medication-safety rules to patient-specific clinical contexts. Questions may have one or more correct options.

ZhitianHou/MedPIC-Bench 是 ModelScope 魔搭社区上的数据集，采用 cc-by-4.0 许可。

- **Repository**: ZhitianHou/MedPIC-Bench
- **License**: cc-by-4.0
- **Downloads**: 58
- **Stars**: 0
- **Last updated**: 2026-07-30

Source: https://www.modelscope.cn/datasets/ZhitianHou/MedPIC-Bench

---

# MedPIC-Bench

MedPIC-Bench is an English multiple-choice benchmark for evaluating whether
large language models apply medication-safety rules to patient-specific
clinical contexts. Questions may have one or more correct options.

## Dataset overview

The dataset contains 467 questions:

| Task family | Questions |
|---|---:|
| Guideline following | 284 |
| Counterfactual reasoning | 183 |
| **Total** | **467** |

## Loading the dataset

```python
from modelscope.msdatasets import MsDataset

dataset = MsDataset.load(
    "ZhitianHou/MedPIC-Bench",
    subset_name="default",
    split="test",
)
```

## Data fields

| Field | Type | Description |
|---|---|---|
| `instance_id` | string | Unique question identifier. |
| `patient_vignette` | string | Patient-specific clinical context. |
| `question` | string | Medication-safety question. |
| `options` | object | Mapping from option letters to answer text. |
| `answer` | list[string] | Correct option letter or letters. |
| `taxonomy_clinical_department` | string | Clinical department category. |
| `taxonomy_clinical_system` | string | Organ-system category. |
| `taxonomy_drug_categories` | list[string] | Drug classes represented in the question. |
| `taxonomy_patient_info_type` | string | Patient-information or condition type. |
| `taxonomy_population_source` | string | Population group represented by the question. |
| `taxonomy_reasoning_operation` | string | Required reasoning operation. |
| `benchmark_task_family` | string | `guideline_following` or `counterfactual`. |

Example:

```json
{
  "instance_id": "SS-HF-HFREF-001",
  "patient_vignette": "A 71-year-old man has heart failure with reduced ejection fraction (LVEF 32%), currently asymptomatic on guideline-directed therapy review. No other significant past medical history; renal and hepatic function normal; not on other prescription medications; no known drug allergies.",
  "question": "For this patient, which of the following drugs should be avoided?",
  "options": {
    "A": "Verapamil",
    "B": "Dextromethorphan-quinidine",
    "C": "Diltiazem",
    "D": "Amlodipine",
    "E": "Cilostazol"
  },
  "answer": [
    "A",
    "B",
    "C",
    "E"
  ],
  "taxonomy_clinical_department": "cardiology",
  "taxonomy_clinical_system": "cardiovascular system",
  "taxonomy_drug_categories": [
    "calcium-channel blockers",
    "antiarrhythmics",
    "pde3 inhibitors/peripheral vascular drugs"
  ],
  "taxonomy_patient_info_type": "disease presence",
  "taxonomy_population_source": "older adults",
  "taxonomy_reasoning_operation": "static risk identification",
  "benchmark_task_family": "guideline_following"
}
```

## Ethical statement and disclaimer

MedPIC-Bench is a research benchmark for evaluating model behavior. It is not
a clinical decision-support system, medical device, prescribing reference, or
source of medical advice. Benchmark questions and model outputs must not be
used to make decisions about diagnosis, treatment, medication selection, or
patient care.

The benchmark covers a limited set of medication-safety rules, clinical
contexts, and patient populations. It is not an exhaustive representation of
clinical practice. Performance on this dataset does not establish that a model
is medically accurate, safe, unbiased, or suitable for deployment. Model
outputs may contain clinically harmful errors even when aggregate benchmark
scores are high.

Users are responsible for appropriate expert review, risk assessment, and
compliance with applicable ethical, institutional, and legal requirements.

## Acknowledgments

We gratefully acknowledge Yuting Long (龙宇婷) and Xinyao Ma (马馨瑶) for
their valuable support in dataset validation. Their careful review helped
improve the consistency and reliability of the benchmark.