## 🔥News!!!
- __[2024.03]__ HiGen's code and models are now publicly available! Additionally, the high-quality super-resolution code and models of our I2VGen-XL have also been released!
## Preparation
### Installation
```
conda create -n vgen python=3.8
conda activate vgen
pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
You also need to ensure that your system has installed the `ffmpeg` command. If it is not installed, you can install it using the following command:
```
sudo apt-get update && apt-get install ffmpeg libsm6 libxext6 -y
```
### Clone the code
```
git clone https://github.com/ali-vilab/VGen.git
cd VGen
```
### Download the checkpoints
Here, we are officially releasing the HiGen model and the super-resolution model. Other auxiliary models such as the CLIP text encoder and AutoEncoder can be referenced from Stable Diffusion. For convenience, they are also provided in this repository.
```
!pip install modelscope
from modelscope.hub.snapshot_download import snapshot_download
model_dir = snapshot_download('iic/HiGen', cache_dir='models/')
```
Then you might need the following command to move the checkpoints to the "models/" directory:
```
mv ./models/iic/HiGen/* ./models/
```
## Inference your prompts
You can perform text-to-video inference on the model using the following command.
```
python inference.py --cfg configs/higen_infer.yaml
```
Then you can execute the following command to perform super-resolution on the generated videos:
```
python inference.py --cfg configs/sr600_infer.yaml
```
Finally, you can find the videos you generated in the `workspace/experiments/text_list_for_t2v_share` directory. For specific configurations such as data, models, seed, etc., please refer to the `higen_infer.yaml` file.
Click HERE to view the generated video. |
Click HERE to view the generated video. |