---
title: VISTA-Mantis
canonical_url: "https://www.modelscope.cn/models/TIGER-Lab/VISTA-Mantis"
md_url: "https://www.modelscope.cn/models/TIGER-Lab/VISTA-Mantis.md"
repository: TIGER-Lab/VISTA-Mantis
last_updated: 2025-04-24
license: mit
model_type:
  - idefics2
architectures:
  - Idefics2ForConditionalGeneration
parameters: 8.4B
tensor_type:
  - BF16
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 1112
stars: 0
---

# VISTA-Mantis

> VISTA-Mantis - TIGER-Lab 在 ModelScope 开源的模型。This repo contains model checkpoints for VISTA-Mantis. VISTA is a video spatiotemporal augmentation method that generates long-duration and high-resolution video instruction-following data to enhance the video…

TIGER-Lab/VISTA-Mantis 是 ModelScope 魔搭社区上的 8.4B 参数机器学习模型，采用 mit 许可。

- **Repository**: TIGER-Lab/VISTA-Mantis
- **License**: mit
- **Parameters**: 8.4B
- **Downloads**: 1112
- **Stars**: 0
- **Last updated**: 2025-04-24

Source: https://www.modelscope.cn/models/TIGER-Lab/VISTA-Mantis

---

# VISTA-Mantis

This repo contains model checkpoints for **VISTA-Mantis**. [VISTA](https://huggingface.co/papers/2412.00927) is a video spatiotemporal augmentation method that generates long-duration and high-resolution video instruction-following data to enhance the video understanding capabilities of video LMMs.

[**🌐 Homepage**](https://tiger-ai-lab.github.io/VISTA/) | [**📖 arXiv**](https://arxiv.org/abs/2412.00927) | [**💻 GitHub**](https://github.com/TIGER-AI-Lab/VISTA) | [**🤗 VISTA-400K**](https://huggingface.co/datasets/TIGER-Lab/VISTA-400K) | [**🤗 Models**](https://huggingface.co/collections/TIGER-Lab/vista-674a2f0fab81be728a673193) | [**🤗 HRVideoBench**](https://huggingface.co/datasets/TIGER-Lab/HRVideoBench)

## Video Instruction Data Synthesis Pipeline
<p align="center">
<img src="https://tiger-ai-lab.github.io/VISTA/static/images/vista_main.png" width="900">
</p>

VISTA leverages insights from image and video classification data augmentation techniques such as CutMix, MixUp and VideoMix, which demonstrate that training on synthetic data created by overlaying or mixing multiple images or videos results in more robust classifiers. Similarly, our method spatially and temporally combines videos to create (artificial) augmented video samples with longer durations and higher resolutions, followed by synthesizing instruction data based on these new videos. Our data synthesis pipeline utilizes existing public video-caption datasets, making it fully open-sourced and scalable. This allows us to construct VISTA-400K, a high-quality video instruction-following dataset aimed at improving the long and high-resolution video understanding capabilities of video LMMs.



## Citation
If you find our paper useful, please cite us with
```
@misc{ren2024vistaenhancinglongdurationhighresolution,
      title={VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by Video Spatiotemporal Augmentation}, 
      author={Weiming Ren and Huan Yang and Jie Min and Cong Wei and Wenhu Chen},
      year={2024},
      eprint={2412.00927},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.00927}, 
}
```