---
title: biomed-visual-instructions
canonical_url: "https://www.modelscope.cn/datasets/AdaptLLM/biomed-visual-instructions"
md_url: "https://www.modelscope.cn/datasets/AdaptLLM/biomed-visual-instructions.md"
repository: AdaptLLM/biomed-visual-instructions
last_updated: 2025-08-21
license: apache-2.0
storage_size: "1.3 GB"
downloads: 1481
stars: 0
---

# biomed-visual-instructions

> biomed-visual-instructions - AdaptLLM 在 ModelScope 开源的数据集。Adapting Multimodal Large Language Models to Domains via Post-Training (EMNLP 2025)

AdaptLLM/biomed-visual-instructions 是 ModelScope 魔搭社区上的数据集，存储大小 1.3 GB，采用 apache-2.0 许可。

- **Repository**: AdaptLLM/biomed-visual-instructions
- **License**: apache-2.0
- **Storage size**: 1.3 GB
- **Downloads**: 1481
- **Stars**: 0
- **Last updated**: 2025-08-21

Source: https://www.modelscope.cn/datasets/AdaptLLM/biomed-visual-instructions

---

# Adapting Multimodal Large Language Models to Domains via Post-Training (EMNLP 2025)

This repos contains the **biomedicine visual instructions for post-training MLLMs** in our paper: [On Domain-Specific Post-Training for Multimodal Large Language Models](https://huggingface.co/papers/2411.19930).

The main project page is: [Adapt-MLLM-to-Domains](https://huggingface.co/AdaptLLM/Adapt-MLLM-to-Domains)

## Data Information
Using our [visual instruction synthesizer](https://huggingface.co/AdaptLLM/visual-instruction-synthesizer), we generate visual instruction tasks based on the image-caption pairs from [PubMedVision](https://huggingface.co/datasets/FreedomIntelligence/PubMedVision) (referred to as `PMC_refined` in our paper). These synthetic tasks, combined with the original image captioning tasks, are used to train general MLLMs through a single-stage post-training process.

- **image_caption_pairs.json**: Derived from [PubMedVision_Alignment_VQA](https://huggingface.co/datasets/FreedomIntelligence/PubMedVision/blob/main/PubMedVision_Alignment_VQA.json) in PubMedVision, we include only single-image examples, resulting in 500K image-caption pairs.
- **synthetic_visual_tasks.json**: Synthesized based on the aforementioned image-caption pairs, containing 144K synthetic instruction-response pairs after applying our consistency-based data filter.
- **image_caption_and_synthetic_task.json**: This dataset is used to reproduce our `single-stage domain-specific post-training`, containing both image-captioning tasks and synthetic visual-instruction tasks, totaling 500K examples (equal to the number of image-caption pairs).

## To Download the Data
1. Set up dependencies:
```bash
pip install "huggingface_hub[cli]"
```

2. Download text data:
```bash
REPO="AdaptLLM/biomed-visual-instructions"

# The local directory where you intend to save the files
LOCAL_DIR="./biomed-visual-instructions"

# Choose from ["image_caption_and_synthetic_task.json", "image_caption_pairs.json", "synthetic_visual_tasks.json"]
FILE="image_caption_and_synthetic_task.json" # This is used for reproducing AdaMLLM in our paper.

huggingface-cli download --resume-download ${REPO} ${FILE} --local-dir ${LOCAL_DIR} --repo-type dataset
```

3. Download image data:
```bash
REPO="FreedomIntelligence/PubMedVision"

huggingface-cli download --resume-download ${REPO} --local-dir ${LOCAL_DIR} --repo-type dataset --include "images_*.zip"
```

4. Unzip the downloaded images:
```bash
cd ${LOCAL_DIR}

for ((i=0; i<20; i++))
do
    unzip -j images_$i.zip -d images/ & # Wait patiently, it takes a while...
done
```

## To reproduce the data

We have included detailed scripts to reproduce the data in [Synthesis.md](https://github.com/bigai-ai/QA-Synthesizer/blob/main/docs/Synthesis.md).

## To train MLLMs using the data

Our training data can be easily used to train MLLMs based on the `Llava` repository or the `LLaMA Factory` repository. Please refer to the [Post-Train Guide](https://github.com/bigai-ai/QA-Synthesizer/blob/main/docs/Post_Train.md).  


## Citation
If you find our work helpful, please cite us.

[Adapt MLLM to Domains](https://huggingface.co/papers/2411.19930) (EMNLP 2025 Findings)
```bibtex
@article{adamllm,
  title={On Domain-Adaptive Post-Training for Multimodal Large Language Models},
  author={Cheng, Daixuan and Huang, Shaohan and Zhu, Ziyu and Zhang, Xintong and Zhao, Wayne Xin and Luan, Zhongzhi and Dai, Bo and Zhang, Zhenliang},
  journal={arXiv preprint arXiv:2411.19930},
  year={2024}
}
```

[Adapt LLM to Domains](https://huggingface.co/papers/2309.09530) (ICLR 2024)
```bibtex
@inproceedings{
adaptllm,
title={Adapting Large Language Models via Reading Comprehension},
author={Daixuan Cheng and Shaohan Huang and Furu Wei},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=y886UXPEZ0}
}
```