---
title: "DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation"
canonical_url: "https://www.modelscope.cn/papers/124784"
md_url: "https://www.modelscope.cn/papers/124784.md"
arxiv_id: 2503.06053
published: 2025-03-08
last_updated: 2025-03-08
authors:
  - "Runze Zhang"
  - "Guoguang Du"
  - "Xiaochuan Li"
  - "Qi Jia"
  - "Liang Jin"
  - "Lu Liu"
  - "Jingjing Wang"
  - "Cong Xu"
  - "Zhenhua Guo"
  - "Yaqian Zhao"
  - "Xiaoli Gong"
  - "Rengang Li"
  - "Baoyu Fan"
model_name: DropletVideo
model_developer: "IEIT System Co., Ltd.、Nankai University、Tsinghua University"
domain:
  - "计算机视觉"
  - "人工智能"
  - "视频生成"
  - "时空一致性"
  - "扩散模型"
type:
  - "计算机视觉"
  - "人工智能"
  - "视频生成"
  - "时空一致性"
  - "扩散模型"
  - "Computer Vision and Pattern Recognition"
  - "Artificial Intelligence"
arxiv_url: "https://arxiv.org/abs/2503.06053"
pdf_url: "https://arxiv.org/pdf/2503.06053"
code_link: "https://dropletx.github.io"
---

# DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation

> Spatio-temporal consistency is a critical research topic in video generation. A qualified generated video segment must ensure plot plausibility and coherence while maintaining visual consistency of objects and scenes across varying viewpoints. Prior…

「DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation」是 ModelScope 魔搭社区收录的论文，arXiv 2503.06053，作者为 Runze Zhang, Guoguang Du, Xiaochuan Li et al.，发表于 2025-03-08，属于 计算机视觉、人工智能、视频生成 领域。

- **ArXiv**: 2503.06053
- **Published**: 2025-03-08
- **Authors**: Runze Zhang, Guoguang Du, Xiaochuan Li, Qi Jia, Liang Jin, Lu Liu, Jingjing Wang, Cong Xu, Zhenhua Guo, Yaqian Zhao, Xiaoli Gong, Rengang Li, Baoyu Fan
- **Model**: DropletVideo
- **Developer**: IEIT System Co., Ltd.、Nankai University、Tsinghua University
- **Domain**: 计算机视觉, 人工智能, 视频生成, 时空一致性, 扩散模型
- **ArXiv URL**: https://arxiv.org/abs/2503.06053
- **PDF**: https://arxiv.org/pdf/2503.06053
- **Code**: https://dropletx.github.io

Source: https://www.modelscope.cn/papers/124784

---

> DropletVideo：构建整体时空一致性的大规模视频生成数据集与模型

## 摘要

本文探讨了视频生成中的整体时空一致性问题，提出了一个新的数据集和模型。研究背景方面，现有的视频生成方法主要集中在时间或空间一致性上，忽略了两者之间的协同作用。特别是在涉及复杂相机运动的视频中，物体的引入或移除会叠加并影响先前的叙事，导致内容不连贯。为了解决这些问题，作者构建了一个名为DropletVideo-10M的数据集，该数据集包含1000万个带有动态相机运动和物体动作的视频片段，每个视频配有平均206字的详细描述。基于此数据集，作者开发并训练了DropletVideo模型，该模型在视频生成过程中能够保持整体时空一致性。实验结果表明，DropletVideo在生成具有自然相机运动和流畅场景转换的多情节视频方面表现出色。此外，作者开源了数据集、代码和模型权重，以促进公共领域的算法创新。

## Abstract

Spatio-temporal consistency is a critical research topic in video generation. A qualified generated video segment must ensure plot plausibility and coherence while maintaining visual consistency of objects and scenes across varying viewpoints. Prior research, especially in open-source projects, primarily focuses on either temporal or spatial consistency, or their basic combination, such as appending a description of a camera movement after a prompt without constraining the outcomes of this movement. However, camera movement may introduce new objects to the scene or eliminate existing ones, thereby overlaying and affecting the preceding narrative. Especially in videos with numerous camera movements, the interplay between multiple plots becomes increasingly complex. This paper introduces and examines integral spatio-temporal consistency, considering the synergy between plot progression and camera techniques, and the long-term impact of prior content on subsequent generation. Our research encompasses dataset construction through to the development of the model. Initially, we constructed a DropletVideo-10M dataset, which comprises 10 million videos featuring dynamic camera motion and object actions. Each video is annotated with an average caption of 206 words, detailing various camera movements and plot developments. Following this, we developed and trained the DropletVideo model, which excels in preserving spatio-temporal coherence during video generation. The DropletVideo dataset and model are accessible at https://dropletx.github.io.