---
title: DH_Live
canonical_url: "https://www.modelscope.cn/datasets/insummer/DH_Live"
md_url: "https://www.modelscope.cn/datasets/insummer/DH_Live.md"
repository: insummer/DH_Live
chinese_name: "开源数字人"
last_updated: 2024-12-09
license: "Apache License 2.0"
storage_size: "595 MB"
downloads: 609
stars: 3
---

# DH_Live

> DH_Live - insummer 在 ModelScope 开源的数据集。改自： https://github.com/kleinlee/DH_live

insummer/DH_Live 是 ModelScope 魔搭社区上的数据集，存储大小 595 MB，采用 Apache License 2.0 许可。

- **Repository**: insummer/DH_Live
- **License**: Apache License 2.0
- **Storage size**: 595 MB
- **Downloads**: 609
- **Stars**: 3
- **Last updated**: 2024-12-09

Source: https://www.modelscope.cn/datasets/insummer/DH_Live

---

# Real-time Live Streaming Digital Human
# 实时直播数字人  [bilibili video](https://www.bilibili.com/video/BV1Ppv1eEEgj/?vd_source=53601feee498369e726af7dbc2dae349)
### News
Audio Model training code released！Details can be found [here](https://github.com/kleinlee/DH_live/tree/master/train_audio).

## Training
Details on the render model training can be found [here](https://github.com/kleinlee/DH_live/tree/master/train).
### Video Example


https://github.com/user-attachments/assets/7e0b5bc2-067b-4048-9f88-961afed12478


## Overview
This project is a real-time live streaming digital human powered by few-shot learning. It is designed to run smoothly on all 30 and 40 series graphics cards, ensuring a seamless and interactive live streaming experience.

### Key Features
- **Real-time Performance**: The digital human can interact in real-time with 25+ fps for common NVIDIA 30 and 40 series GPUs
- **Few-shot Learning**: The system is capable of learning from a few examples to generate realistic responses.
## Usage

### Create Environment and Unzip the Model File 
First, navigate to the `checkpoint` directory and unzip the model file:
```bash
conda create -n dh_live python=3.12
conda activate dh_live
pip install torch --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
cd checkpoint
```
on Linux
```bash
cat render.pth.gz.001 render.pth.gz.002 > render.pth.gz
gzip -d -c render.pth.gz > render.pth
```
on Windows, use zip software such as 7zip/WinRAR to unzip checkpoint file.
### Prepare Your Video
Next, prepare your video using the data_preparation script. Replace YOUR_VIDEO_PATH with the path to your video:
```bash
python data_preparation.py YOUR_VIDEO_PATH
```
The result (video_info) will be stored in the ./video_data directory.
### Run with Audio File
Run the demo script with an audio file. Make sure the audio file is in .wav format with a sample rate of 16kHz and 16-bit single channel. Replace video_data/test with the path to your video_info file, video_data/audio0.wav with the path to your audio file, and 1.mp4 with the desired output video path:
```bash
python demo.py video_data/test video_data/audio0.wav 1.mp4
```
### Real-Time Run with Microphone
For real-time operation using a microphone, simply run the following command:
```bash
python demo_avatar.py
```

## Acknowledgements 
We would like to thank the contributors of [Wav2Lip](https://github.com/Rudrabha/Wav2Lip), [DINet](https://github.com/MRzzm/DINet), [LiveSpeechPortrait](https://github.com/YuanxunLu/LiveSpeechPortraits) repositories, for their open research and contributions.

## License
This project is licensed under the MIT License.

## Contact
For any questions or suggestions, please contact us at [kleinlee1@outlook.com].