---
title: SEED-Bench
canonical_url: "https://www.modelscope.cn/datasets/TencentARC/SEED-Bench"
md_url: "https://www.modelscope.cn/datasets/TencentARC/SEED-Bench.md"
repository: TencentARC/SEED-Bench
last_updated: 2024-11-21
license: cc-by-nc-4.0
storage_size: "23 GB"
domain:
  - size_categories
  - language
  - pretty_name
  - task_categories
tasks:
  - "10K<n<100K"
  - en
  - SEED-Bench
  - visual-question-answering
downloads: 4852
stars: 2
---

# SEED-Bench

> SEED-Bench - TencentARC 在 ModelScope 开源的数据集。Benchmark type: SEED-Bench is a large-scale benchmark to evaluate Multimodal Large Language Models (MLLMs). It consists of 19K multiple choice questions with accurate human annotations, which covers 12 evaluation…

TencentARC/SEED-Bench 是 ModelScope 魔搭社区上的10K<n<100K、en、SEED-Bench数据集，涉及 size_categories、language、pretty_name 领域，存储大小 23 GB，采用 cc-by-nc-4.0 许可。

- **Repository**: TencentARC/SEED-Bench
- **License**: cc-by-nc-4.0
- **Tasks**: 10K<n<100K, en, SEED-Bench, visual-question-answering
- **Domain**: size_categories, language, pretty_name, task_categories
- **Storage size**: 23 GB
- **Downloads**: 4852
- **Stars**: 2
- **Last updated**: 2024-11-21

Source: https://www.modelscope.cn/datasets/TencentARC/SEED-Bench

---

# SEED-Bench Card

## Benchmark details

**Benchmark type:**
SEED-Bench is a large-scale benchmark to evaluate Multimodal Large Language Models (MLLMs).
It consists of 19K multiple choice questions with accurate human annotations, which
covers 12 evaluation dimensions including the comprehension of both the image and video modality.


**Benchmark date:**
SEED-Bench was collected in July 2023.

**Paper or resources for more information:**
https://github.com/AILab-CVC/SEED-Bench

**License:**
Attribution-NonCommercial 4.0 International. It should abide by the policy of OpenAI: https://openai.com/policies/terms-of-use.

For the images of SEED-Bench, we use the data from Conceptual Captions Dataset (https://ai.google.com/research/ConceptualCaptions/)
following its license (https://github.com/google-research-datasets/conceptual-captions/blob/master/LICENSE).
Tencent does not hold the copyright for these images and the copyright belongs to the original owner of Conceptual Captions Dataset. 

For the videos of SEED-Bench, we use tha data from Something-Something v2 (https://developer.qualcomm.com/software/ai-datasets/something-something),
Epic-kitchen 100 (https://epic-kitchens.github.io/2023) and 
Breakfast (https://serre-lab.clps.brown.edu/resource/breakfast-actions-dataset/). We only provide the video name. Please download them in their official websites.

**Where to send questions or comments about the benchmark:**
https://github.com/AILab-CVC/SEED-Bench/issues

## Intended use
**Primary intended uses:**
The primary use of SEED-Bench is evaluate Multimodal Large Language Models on spatial and temporal understanding.

**Primary intended users:**
The primary intended users of the Benchmark are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.