---
title: cinepile
canonical_url: "https://www.modelscope.cn/datasets/swift/cinepile"
md_url: "https://www.modelscope.cn/datasets/swift/cinepile.md"
repository: swift/cinepile
last_updated: 2024-08-23
license: "Apache License 2.0"
storage_size: "122 MB"
domain:
  - task_categories
tasks:
  - video-text-to-text
downloads: 3497
stars: 1
---

# cinepile

> cinepile - swift 在 ModelScope 开源的数据集。CinePile: A Long Video Question Answering Dataset and Benchmark

swift/cinepile 是 ModelScope 魔搭社区上的video-text-to-text数据集，涉及 task_categories 领域，存储大小 122 MB，采用 Apache License 2.0 许可。

- **Repository**: swift/cinepile
- **License**: Apache License 2.0
- **Tasks**: video-text-to-text
- **Domain**: task_categories
- **Storage size**: 122 MB
- **Downloads**: 3497
- **Stars**: 1
- **Last updated**: 2024-08-23

Source: https://www.modelscope.cn/datasets/swift/cinepile

---

# CinePile: A Long Video Question Answering Dataset and Benchmark

CinePile is a question-answering-based, long-form video understanding dataset. It has been created using advanced large language models (LLMs) with human-in-the-loop pipeline leveraging existing human-generated raw data. It consists of approximately 300,000 training data points and 5,000 test data points. 



If you have any comments or questions, reach out to: [Ruchit Rawal](https://ruchitrawal.github.io/) or [Gowthami Somepalli](https://somepago.github.io/)

Other links - [Website](https://ruchitrawal.github.io/cinepile/)   [Paper](https://arxiv.org/abs/2405.08813)

## Version support and revisions

- October 2024: We refine both the training and test split using the adversarial refinement process described in detail [here](https://huggingface.co/blog/cinepile2). This refined version will be loaded by default when running `load_dataset("tomg-group-umd/cinepile")`. To load the previous version, use `load_dataset("tomg-group-umd/cinepile", "v1")`.


## Dataset Structure

Each row in the dataset consists of a `question` (dtype: string), five `choices` (dtype: list), and an `answer_key` (dtype: string). Auxiliary columns are included that store the movie's name, movie's genre, video clip titles, etc.

The train split of the dataset is intended for the instruction tuning of video-LLMs. The test split is designed for benchmarking video-LLMs and includes the `hard_split` column, which is "True" for particularly challenging questions and "False" otherwise. The `visual_reliance` column indicates whether a question likely requires integrating visual information to be answered correctly.

### Dataset Features

- **movie_name**: Name of the movie to which the video clip belongs.
- **year**: Release year of the movie.
- **genre**: Genre(s) of the movie.
- **yt_clip_title**: Title of the video clip as it appears on YouTube.
- **yt_clip_link**: URL link to the video clip on YouTube.
- **movie_scene**: Description of the movie scene, contains subtitles and visual descriptions.
- **subtitles**: Subtitles extracted from the movie scene.
- **question**: Question derived from the movie scene.
- **choices**: Multiple-choice options associated with the question.
- **answer_key**: The correct answer from the choices provided.
- **answer_key_position**: The index position of the correct answer within the choices list.
- **question_category**: The category to which the question belongs.
- **hard_split**: Indicates if the question is particularly challenging. "N/A" for the train set; applicable only in the test set.
- **visual_reliance**: Indicates if the question requires visual information for an accurate answer. "N/A" for the train set.

## Dataset Use and Starter Snippets

### Loading the dataset
You can load the dataset easily using the Datasets library:
```
from datasets import load_dataset

dataset = load_dataset("tomg-group-umd/cinepile")
```

### Retrieving questions from a specific clip
```
cinepile_test = load_dataset('tomg-group-umd/cinepile', token=True, split='test')
yt_clip_title = "Extraction (2015) - You're Crazy Scene (5/10) | Movieclips"

clip_test_dataset = cinepile_test.filter(lambda x: x['yt_clip_title'] == yt_clip_title)
```

### Loading the hard-split:
```
cinepile_test = load_dataset('tomg-group-umd/cinepile', token=True, split='test')
hard_split_test = cinepile_test.filter(lambda x: x['hard_split'] == "True")
```

Please refer to the accompanying [Colab notebook](https://colab.research.google.com/drive/1jDwvPoCsg9tck3dFhVCV-h3Ny6992wCr?usp=sharing) for more examples e.g. evaluating VLMs, extracting responses, etc.


### Cite us:

```
@article{rawal2024cinepile,
  title={CinePile: A Long Video Question Answering Dataset and Benchmark},
  author={Rawal, Ruchit and Saifullah, Khalid and Basri, Ronen and Jacobs, David and Somepalli, Gowthami and Goldstein, Tom},
  journal={arXiv preprint arXiv:2405.08813},
  year={2024}
}
```