---
title: SPARK
canonical_url: "https://www.modelscope.cn/datasets/AI-ModelScope/SPARK"
md_url: "https://www.modelscope.cn/datasets/AI-ModelScope/SPARK.md"
repository: AI-ModelScope/SPARK
last_updated: 2024-11-21
license: apache-2.0
storage_size: "862 MB"
domain:
  - configs
  - size_categories
  - dataset_info
tasks:
  - "{config_name=default, data_files=[{split=train, path=data/train-*}]}"
  - "1K<n<10K"
  - dataset_size
downloads: 2741
stars: 0
---

# SPARK

> SPARK - AI-ModelScope 在 ModelScope 开源的数据集。⚡ SPARK (multi-vision Sensor Perception And Reasoning benchmarK)

AI-ModelScope/SPARK 是 ModelScope 魔搭社区上的{config_name=default, data_files=[{split=train, path=data/train-*}]}、1K<n<10K、dataset_size数据集，涉及 configs、size_categories、dataset_info 领域，存储大小 862 MB，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/SPARK
- **License**: apache-2.0
- **Tasks**: {config_name=default, data_files=[{split=train, path=data/train-*}]}, 1K<n<10K, dataset_size
- **Domain**: configs, size_categories, dataset_info
- **Storage size**: 862 MB
- **Downloads**: 2741
- **Stars**: 0
- **Last updated**: 2024-11-21

Source: https://www.modelscope.cn/datasets/AI-ModelScope/SPARK

---

# ⚡ SPARK (multi-vision Sensor Perception And Reasoning benchmarK)

[**🌐 github**](https://github.com/top-yun/SPARK) | [**🤗 Dataset**](https://huggingface.co/datasets/topyun/SPARK) | [**📃 Paper**](https://arxiv.org/abs/2408.12114)

## Dataset Details

<p align="center">
  <img src="https://raw.githubusercontent.com/top-yun/SPARK/main/resources/examples.png" :height="400px" width="800px">
</p>

SPARK can reduce the fundamental multi-vision sensor information gap between images and multi-vision sensors. We generated 6,248 vision-language test samples automatically to investigate multi-vision sensory perception and multi-vision sensory reasoning on physical sensor knowledge proficiency across different formats, covering different types of sensor-related questions.



## Uses

you can easily download the dataset as follows:
```python
from datasets import load_dataset
test_dataset = load_dataset("topyun/SPARK", split="train")
```

Additionally, we have provided two example codes for evaluation: Open Model([**test.py**](https://github.com/top-yun/SPARK/blob/main/test.py)) and Closed Model([**test_closed_models.py**](https://github.com/top-yun/SPARK/blob/main/test_closed_models.py)). You can easily run them as shown below.

If you have 4 GPUs and want to run the experiment with llava-1.5-7b, you can do the following:
```bash
accelerate launch --config_file utils/ddp_accel_fp16.yaml \
--num_processes=4 \
test.py \
--batch_size 1 \
--model llava \
```

When running the closed model, make sure to insert your API KEY into the [**config.py**](https://github.com/top-yun/SPARK/blob/main/config.py) file.
If you have 1 GPU and want to run the experiment with gpt-4o, you can do the following:
```bash
accelerate launch --config_file utils/ddp_accel_fp16.yaml \
--num_processes=$n_gpu \
test_closed_models.py \
--batch_size 8 \
--model gpt \
--multiprocess True \
```

### Tips
The evaluation method we've implemented simply checks whether 'A', 'B', 'C', 'D', 'yes', or 'no' appears at the beginning of the sentence.
So, if the model you're evaluating provides unexpected answers (e.g., "'B'ased on ..." or "'C'onsidering ..."), you can resolve this by adding "Do not include any additional text." at the end of the prompt.



### Source Data


#### Data Collection and Processing

These instructions are built from five public datasets: [MS-COCO](https://arxiv.org/abs/1405.0312), [M3FD](https://arxiv.org/abs/2203.16220v1), [Dog&People](https://public.roboflow.com/object-detection/thermal-dogs-and-people), [RGB-D scene dataset](https://arxiv.org/abs/2110.11590), and [UNIFESP X-ray Body Part Classifier Competition dataset](https://www.kaggle.com/competitions/unifesp-x-ray-body-part-classifier). 


## Citation 

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```bibtex
@misc{yu2024sparkmultivisionsensorperception,
      title={SPARK: Multi-Vision Sensor Perception and Reasoning Benchmark for Large-scale Vision-Language Models}, 
      author={Youngjoon Yu and Sangyun Chung and Byung-Kwan Lee and Yong Man Ro},
      year={2024},
      eprint={2408.12114},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2408.12114}, 
}
```

## Contact

[SangYun Chung](https://sites.google.com/view/sang-yun-chung/profile): jelarum@kaist.ac.kr