---
title: FVQA
canonical_url: "https://www.modelscope.cn/datasets/lmms-lab/FVQA"
md_url: "https://www.modelscope.cn/datasets/lmms-lab/FVQA.md"
repository: lmms-lab/FVQA
last_updated: 2025-08-09
license: apache-2.0
storage_size: "2.2 GB"
downloads: 3745
stars: 0
---

# FVQA

> FVQA - lmms-lab 在 ModelScope 开源的数据集。Factual Visual Question Answering (FVQA)

lmms-lab/FVQA 是 ModelScope 魔搭社区上的数据集，存储大小 2.2 GB，采用 apache-2.0 许可。

- **Repository**: lmms-lab/FVQA
- **License**: apache-2.0
- **Storage size**: 2.2 GB
- **Downloads**: 3745
- **Stars**: 0
- **Last updated**: 2025-08-09

Source: https://www.modelscope.cn/datasets/lmms-lab/FVQA

---

## Factual Visual Question Answering (FVQA)

### Dataset Summary
FactualVQA (FVQA) is a multimodal Visual Question Answering dataset created for search-augmented training and evaluation. It emphasizes knowledge-intensive questions that require external information beyond the given image. Each entry includes an image, a question, and an answer (optionally accompanied by candidate answers), enabling models to develop and refine on-demand search strategies. Details of dataset construction are provided in the the [blog](https://www.lmms-lab.com/posts/mmsearch_r1/) or the [paper](https://arxiv.org/abs/2506.20670).

### Dataset Structure
- Data Fields
    The datasets are stored in Parquet format and include the following columns:
    - `data_id`: unique data id
    - `prompt`: The user question
    - `images`: Raw image data stored as bytes
    - `reward_model`: Ground truth and candidate answers used for reward calculation
    - `data_source`: Specifies which reward function to use in veRL (e.g., mmsearch_r1/fvqa_train, mmsearch_r1/fvqa_test)
    - `image_urls`: Optional field for potential use with the image search tool
    - `category`: search-required and search-free annotations
- Train/Test Split
    - Train: ~5k samples, with approximately 68% search-required and 32% search-free (estimated using Qwen2.5-VL-7B-Instruct)
    - Test: ~1.8k samples
- Source
    - Image Sources: Google Image Search, subset of [InfoSeek](https://github.com/open-vision-language/infoseek)'s training split
    - QA Sources: GPT4o-generated, Human-annotated(for test split), subset of InfoSeek's training split
- Cached Image Search Results of FVQA
    - Cached Image Search Results (relevant webpage titles and thumbnail-image-urls) of images of FVQA dataset, indexed by `data_id` 
        - fvqa_train_image_search_results_cache.pkl
        - fvqa_test_image_search_results_cache.pkl
    - Since the webpage thumbnail URLs from SerpAPI’s search results include both strings and `PIL.Image` objects (e.g., `<class 'PIL.JpegImagePlugin.JpegImageFile'>`), you may need to `pip3 install pillow==11.1.0` to load the pickle files

### Citation
```
@article{wu2025mmsearch,
  title={MMSearch-R1: Incentivizing LMMs to Search},
  author={Wu, Jinming and Deng, Zihao and Li, Wei and Liu, Yiding and You, Bo and Li, Bo and Ma, Zejun and Liu, Ziwei},
  journal={arXiv preprint arXiv:2506.20670},
  year={2025}
}
```