---
title: MeanVC2
canonical_url: "https://www.modelscope.cn/models/aklwss/MeanVC2"
md_url: "https://www.modelscope.cn/models/aklwss/MeanVC2.md"
repository: aklwss/MeanVC2
chinese_name: "MeanVC2补全"
last_updated: 2026-08-08
license: apache-2.0
parameters: 35.4M
tensor_type:
- F32
library_name:
- safetensors
language:
- en
- zh
downloads: 56
stars: 0
---
# MeanVC2
> MeanVC2 - aklwss 在 ModelScope 开源的模型。MeanVC2，补全了wavlm_large_cfg和wavlm_large_finetune
aklwss/MeanVC2 是 ModelScope 魔搭社区上的 35.4M 参数机器学习模型，采用 apache-2.0 许可。
- **Repository**: aklwss/MeanVC2
- **License**: apache-2.0
- **Parameters**: 35.4M
- **Downloads**: 56
- **Stars**: 0
- **Last updated**: 2026-08-08
Source: https://www.modelscope.cn/models/aklwss/MeanVC2
---
# MeanVC2: Robust Low-Latency Streaming Zero-Shot Voice Conversion
## 模型调用说明
这个目录只存放 `MeanVC2` 的模型权重和配置文件。
如果别人单独下载了这个目录，需要再配合 `MeanVC2` 的运行时代码一起用；仅有这些 checkpoint 文件本身不能直接完成推理。
### 这个目录里的主要文件
- `meanvc2_120ms_40ms.safetensors`: 当前更推荐的模型，延迟稍高一点，但音质和稳定性更好
- `meanvc2_40ms_40ms.safetensors`: 更低延迟版本
- `config_120ms_40ms.json`: 120ms 版本配置
- `config_40ms_40ms.json`: 40ms 版本配置
- `vocos.pt`: 声码器
- `wavlm_large_cfg.pt` / `wavlm_large_finetune.pth`: 参考音色相关模型
- `fastu2pp_80ms.pt` / `fastu2pp_160ms.pt`: 运行时依赖的前端模型
### 最小代码调用示例
下面这段示例使用的是本仓库现成的 `MeanVC2Service` 封装，适合想直接把这套 checkpoint 跑起来的人：
```python
import numpy as np
import soundfile as sf
from app.services.meanvc2 import MeanVC2Service
service = MeanVC2Service(device="cuda", model_variant="120ms")
service.load(reference_audio_path="target.wav")
audio, sample_rate = sf.read("source.wav", dtype="float32")
audio = np.asarray(audio, dtype=np.float32)
if audio.ndim > 1:
audio = audio.mean(axis=1)
chunk_size = service.get_stream_chunk_samples()
outputs = []
for start in range(0, len(audio), chunk_size):
chunk = audio[start : start + chunk_size]
converted, out_sr = service.process_stream_chunk(chunk, sample_rate)
if converted.size:
outputs.append(converted)
tail, out_sr = service.flush_stream()
if tail.size:
outputs.append(tail)
if outputs:
merged = np.concatenate(outputs).astype(np.float32, copy=False)
sf.write("converted.wav", merged, out_sr)
```
### 也可以直接用官方脚本
如果你已经有完整的 `MeanVC2` 仓库，可以直接用它自带的脚本：
```bash
python src/infer/infer_e2e.py --model 120ms --source-wav source.wav --target-wav target.wav --output-wav output.wav
python runtime/run_rt.py --mode realtime --model 40ms
```
### 注意
- `MeanVC2` 是语音转语音模型，不是文本转语音
- 必须先提供参考音频，再处理源语音
- 更推荐 `120ms` 版本，质量通常更稳
- 不要随意修改这些文件名，运行时代码通常会按固定名字查找
[](https://arxiv.org/)
[](https://github.com/ASLP-lab/MeanVC2)
[](https://aslp-lab.github.io/MeanVC2/)
**MeanVC2** is a robust, low-latency streaming zero-shot voice conversion (VC) system built upon the diffusion-based conditional flow matching (CFM) framework. By introducing **Future-Receptive Chunking (FRC)** and a **Universal Timbre Token Encoder (UTTE)**, MeanVC2 achieves high-fidelity voice conversion with an end-to-end pipeline latency of only **110 ms** while maintaining superior speaker similarity and audio naturalness even with a **40 ms chunk size**.
## ✨ Key Features
- **🚀 Ultra-Low Latency Streaming**: 110 ms end-to-end first-packet latency with 40 ms chunk size; full pipeline RTF < 0.633 on single CPU core.
- **⚡ Single-Step Generation**: Mean flows + 1-NFE ODE solving for high-quality mel-spectrogram synthesis.
- **🎯 Zero-Shot Capability**: Convert to any unseen target speaker without re-training, robust under low-quality reference audio.
- **💾 Lightweight**: Only 18M parameters — far smaller than competing streaming VC systems.
- **🔊 High Fidelity**: Future-Receptive Chunking (FRC) with Universal Timbre Token Encoder (UTTE) for pronunciation-aware timbre modeling.
## 📁 Model Checkpoints
This repository hosts all pretrained models for MeanVC2:
### Voice Conversion Models
| File | Description |
|------|-------------|
| `meanvc2_120ms_40ms.safetensors` | 120ms chunk + 40ms future (recommended for quality) |
| `meanvc2_40ms_40ms.safetensors` | 40ms chunk + 40ms future (lower latency) |
### Vocoder (JIT)
| File | Description |
|------|-------------|
| `vocos.pt` | Vocos JIT-traced vocoder |
### ASR Encoder (Fast-U2++ JIT)
| File | Description |
|------|-------------|
| `fastu2pp_80ms.pt` | 80ms chunk JIT model (11-frame window, stride=8) |
| `fastu2pp_160ms.pt` | 160ms chunk JIT model (19-frame window, stride=16) |
## 💻 Quick Start
### 1. Clone and Install
```bash
git clone https://github.com/ASLP-lab/MeanVC2.git
cd MeanVC2
conda create -n meanvc2 python=3.11 -y
conda activate meanvc2
pip install torch==2.5.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
```
### 2. Download Pretrained Models
```bash
python initialization.py --task all
```
Or download selectively:
```bash
python initialization.py --task preprocess # BN + SpkEmb extraction only
python initialization.py --task train_120ms # preprocess + 120ms VC + vocoder
python initialization.py --task train_40ms # preprocess + 40ms VC + vocoder
```
### 3. End-to-End Voice Conversion
No pre-extracted features needed — input two wavs, output converted audio:
```bash
# 120ms+40ms model (recommended for quality)
python src/infer/infer_e2e.py \
--source-wav /path/to/source.wav \
--target-wav /path/to/target.wav \
--ckpt-path ckpts/pretrained_models/meanvc2_120ms_40ms.safetensors \
--model-config src/config/config_120ms_40ms.json \
--vocoder-ckpt-path ckpts/vocos/vocos.pt \
--chunk-size 12 --steps 3 \
--output-wav output.wav
# 40ms+40ms model (lower latency)
python src/infer/infer_e2e.py \
--source-wav /path/to/source.wav \
--target-wav /path/to/target.wav \
--ckpt-path ckpts/pretrained_models/meanvc2_40ms_40ms.safetensors \
--model-config src/config/config_40ms_40ms.json \
--vocoder-ckpt-path ckpts/vocos/vocos.pt \
--chunk-size 4 --steps 3 \
--output-wav output.wav
```
### 4. Real-Time Streaming
```bash
cd runtime
# File mode
python run_rt.py --mode file --input in.wav --output out.wav --model 120ms
# Microphone mode
python run_rt.py --mode realtime --model 40ms
```
## 🏗️ Model Architecture
| Component | Description |
|-----------|-------------|
| **Streaming ASR Encoder** | Fast-U2++ (WeNet) extracts bottleneck features (BNFs) from source waveform |
| **Speaker Encoder** | ECAPA-TDNN + WavLM upstream extracts global speaker embedding from reference audio |
| **Universal Timbre Token Encoder (UTTE)** | Transforms speaker embedding into K key-value UTT pairs; BNFs serve as queries in cross-attention for fine-grained, pronunciation-aware timbre cues |
| **DiT-based CFM Decoder** | 4-layer DiT (hidden dim 512, 2 heads) with Future-Receptive Chunking (FRC); trained with mean flows objective for 1-NFE mel-spectrogram generation |
| **Vocoder** | Vocos converts mel-spectrograms to 16 kHz high-fidelity speech waveforms |
**Total parameters**: ~18M
## 📜 License & Disclaimer
MeanVC2 is released under the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0). This open-source license allows you to freely use, modify, and distribute the model, as long as you include the appropriate copyright notice and disclaimer.
MeanVC2 is designed for research and legitimate applications in voice conversion technology. Users must obtain proper consent from individuals whose voices are being converted or used as references. We strongly discourage malicious use including impersonation, fraud, or creating misleading audio content. Users are solely responsible for ensuring compliance with ethical standards and legal requirements.
## 📄 Citation
If you find our work helpful, please cite:
```bibtex
@article{ma2026meanvc2,
title={MeanVC2: Robust Low-Latency Streaming Zero-Shot Voice Conversion},
author={Ma, Guobin and Xia, Yuxuan and Jiang, Yuepeng and Guo, Dake and Xie, Hanke and Hu, Jingbin and Wang, Yanbo and Xie, Lei and Zhu, Pengcheng},
journal={arXiv preprint arXiv:2606.09050},
year={2026}
}
```
## 📧 Contact
For questions or collaborations, please contact: guobin.ma@mail.nwpu.edu.cn
