---
title: tf-t2v
canonical_url: "https://www.modelscope.cn/models/iic/tf-t2v"
md_url: "https://www.modelscope.cn/models/iic/tf-t2v.md"
repository: iic/tf-t2v
last_updated: 2024-04-03
pipeline_tag: text-to-video-synthesis
tasks:
  - text-to-video-synthesis
library_name:
  - pytorch
frameworks:
  - Pytorch
downloads: 492
stars: 2
---

# tf-t2v

> tf-t2v - iic 在 ModelScope 开源的模型。This repo include the checkpoints for TF-T2V and VideoLCM.

- **Repository**: iic/tf-t2v
- **Tasks**: text-to-video-synthesis
- **Downloads**: 492
- **Stars**: 2
- **Last updated**: 2024-04-03

Source: https://www.modelscope.cn/models/iic/tf-t2v

---

# TF-T2V (CVPR-2024)

## This repo include the checkpoints for TF-T2V and VideoLCM.

"models/midas_v3_dpt_large.pth": the checkpoint for depth extraction.
"models/open_clip_pytorch_model.bin": the checkpoint for clip embedding.
"models/sketch_simplification_gan.pth": the checkpoint for sketch extraction.
"models/sr_step_110000_ema.pth": the checkpoint for super-resolution model.
"models/table5_pidinet.pth": the checkpoint for sketch extraction.
"models/tft2v_t2v_32frames_non_ema_560000.pth": the checkpoint for text-to-video generation in TF-T2V (32 frames).
"models/tft2v_t2v_non_ema_512000.pth": the checkpoint for text-to-video generation in TF-T2V (16 frames).
"models/tft2v_vcomposer_non_ema_254000.pth": the checkpoint for compositional video synthesis in TF-T2V (448x256 resulotion).
"models/tft2v_vcomposer_non_ema_270000_896x512.pth": the checkpoint for compositional video synthesis in TF-T2V (896x512 resulotion).
"models/v2-1_512-ema-pruned.ckpt": the checkpoint for Stable Diffusion.
"models/videolcm_t2v_non_ema_544000.pth":  the checkpoint for text-to-video generation in VideoLCM.
"models/videolcm_vcomposer_non_ema_914000.pth": the checkpoint for compositional video synthesis in VideoLCM.



## BibTeX

If this repo is useful to you, please cite our corresponding technical paper.

```bibtex
@article{2023videocomposer,
  title={VideoComposer: Compositional Video Synthesis with Motion Controllability},
  author={Wang, Xiang and Yuan, Hangjie and Zhang, Shiwei and Chen, Dayou and Wang, Jiuniu, and Zhang, Yingya, and Shen, Yujun, and Zhao, Deli and Zhou, Jingren},
  booktitle={arXiv preprint arXiv:2306.02018},
  year={2023}
}
@article{2023i2vgenxl,
  title={I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models},
  author={Zhang, Shiwei and Wang, Jiayu and Zhang, Yingya and Zhao, Kang and Yuan, Hangjie and Qing, Zhiwu and Wang, Xiang  and Zhao, Deli and Zhou, Jingren},
  booktitle={arXiv preprint arXiv:2311.04145},
  year={2023}
}
@article{wang2023modelscope,
  title={Modelscope text-to-video technical report},
  author={Wang, Jiuniu and Yuan, Hangjie and Chen, Dayou and Zhang, Yingya and Wang, Xiang and Zhang, Shiwei},
  journal={arXiv preprint arXiv:2308.06571},
  year={2023}
}
@article{dreamvideo,
  title={DreamVideo: Composing Your Dream Videos with Customized Subject and Motion},
  author={Wei, Yujie and Zhang, Shiwei and Qing, Zhiwu and Yuan, Hangjie and Liu, Zhiheng and Liu, Yu and Zhang, Yingya and Zhou, Jingren and Shan, Hongming},
  journal={arXiv preprint arXiv:2312.04433},
  year={2023}
}
@article{qing2023higen,
  title={Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation},
  author={Qing, Zhiwu and Zhang, Shiwei and Wang, Jiayu and Wang, Xiang and Wei, Yujie and Zhang, Yingya and Gao, Changxin and Sang, Nong },
  journal={arXiv preprint arXiv:2312.04483},
  year={2023}
}
@article{wang2023videolcm,
  title={VideoLCM: Video Latent Consistency Model},
  author={Wang, Xiang and Zhang, Shiwei and Zhang, Han and Liu, Yu and Zhang, Yingya and Gao, Changxin and Sang, Nong },
  journal={arXiv preprint arXiv:2312.09109},
  year={2023}
}
@article{ma2023dreamtalk,
  title={DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models},
  author={Ma, Yifeng and Zhang, Shiwei and Wang, Jiayu and Wang, Xiang and Zhang, Yingya and Deng Zhidong},
  journal={arXiv preprint arXiv:2312.09767},
  year={2023}
}
@article{2023InstructVideo,
  title={InstructVideo: Instructing Video Diffusion Models with Human Feedback},
  author={Yuan, Hangjie and Zhang, Shiwei and Wang, Xiang and Wei, Yujie and Feng, Tao and Pan, Yining and Zhang, Yingya and Liu, Ziwei and Albanie, Samuel and Ni, Dong},
  booktitle={arXiv preprint arXiv:2312.12490},
  year={2023}
}
@article{TFT2V,
 title={A Recipe for Scaling up Text-to-Video Generation with Text-free Videos},
 author={Wang, Xiang and Zhang, Shiwei and Yuan, Hangjie and Qing, Zhiwu and Gong, Biao and Zhang, Yingya and Shen, Yujun and Gao, Changxin and Sang, Nong},
 journal={arXiv preprint arXiv:2312.15770},
 year={2023}
}
```