---
title: MIC
canonical_url: "https://www.modelscope.cn/datasets/BleachNick/MIC"
md_url: "https://www.modelscope.cn/datasets/BleachNick/MIC.md"
repository: BleachNick/MIC
last_updated: 2023-11-14
license: Unknown
storage_size: "281 GB"
domain:
  - source_datasets
tasks:
  - original
downloads: 1207
stars: 0
---

# MIC

> MIC - BleachNick 在 ModelScope 开源的数据集。Dataset Description Visual Language Models (VLMs) have made significant progress in various downstream tasks by developing large-scale multimodal models. However, they sometimes lack reasoning and contextual learning…

BleachNick/MIC 是 ModelScope 魔搭社区上的original数据集，涉及 source_datasets 领域，存储大小 281 GB，采用 Unknown 许可。

- **Repository**: BleachNick/MIC
- **License**: Unknown
- **Tasks**: original
- **Domain**: source_datasets
- **Storage size**: 281 GB
- **Downloads**: 1207
- **Stars**: 0
- **Last updated**: 2023-11-14

Source: https://www.modelscope.cn/datasets/BleachNick/MIC

---

## Dataset Description
Visual Language Models (VLMs) have made significant progress in various downstream tasks by developing large-scale multimodal models. However, they sometimes lack reasoning and contextual learning abilities. On the other hand, Large Language Models (LLMs) have revolutionized the NLP community with their strong reasoning and contextual learning capabilities. LLMs can quickly adapt to new tasks involving inference without fine-tuning pre-trained models or parameter updates, such as question answering and commonsense reasoning.

Studying in context learning abilities contributes to VLMs' ability to generalize new knowledge in lifelong learning environments, develop learnable capabilities, and advance artificial intelligence skills. Therefore, we propose the MIC(Multimodality In-Context Learning) dataset. This is a manually constructed instruction tuning dataset supports interleaved text-image inputs, inter-related multiple image inputs, and multimodal in-context learning inputs. By finetuning VLMs on MIC, we enable them to possess multimodal in-context learning capabilities and understand complex relationships between instructions and multiple images.

### Dataset Introduction
Using multiple data source such as: VQAv2, GQA, COCO, NLVR2, OKVQK, FILCKR, STVQA, MSRVTT, MSRVTTQA, FunQA, TextVQA, RefCOCO, Vizwiz_caption, Ln_COCO, Textcap, WikiArt, DiffusionDB, VSR, LLaVa-Instruct, MiniImagenet, we tranform them with our designed context schema to form those open source dataset into a unified multimodal in-context format. We stored it into jsonl files: It forms the all data in to multi instruction style with zero to few-shot form data. Image data can be found in the **data** folder stored in *.zip files.

You can refer to the [MIC_tool](https://github.com/HaozheZhao/MIC_tool) repo which is the tool we used to transform the open source datasets into MIC data format.

You can also save the processed data into arrow files, which are compatible with huggingface datasets.

For further details, please refer to the [GitHub repository](https://github.com/HaozheZhao/MIC) or consult the accompanying paper.

```
@misc{zhao2023mmicl,
      title={MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning}, 
      author={Haozhe Zhao and Zefan Cai and Shuzheng Si and Xiaojian Ma and Kaikai An and Liang Chen and Zixuan Liu and Sheng Wang and Wenjuan Han and Baobao Chang},
      year={2023},
      eprint={2309.07915},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
