---
title: MSVAE
canonical_url: "https://www.modelscope.cn/models/ASLP-lab/MSVAE"
md_url: "https://www.modelscope.cn/models/ASLP-lab/MSVAE.md"
repository: ASLP-lab/MSVAE
last_updated: 2026-10-08
license: apache-2.0
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 0
stars: 0
---

# MSVAE

> MSVAE - ASLP-lab 在 ModelScope 开源的模型。MSVAE: Rethinking Musical Semantics in Continuous Latent Spaces

ASLP-lab/MSVAE 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可。

- **Repository**: ASLP-lab/MSVAE
- **License**: apache-2.0
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-10-08

Source: https://www.modelscope.cn/models/ASLP-lab/MSVAE

---

# MSVAE: Rethinking Musical Semantics in Continuous Latent Spaces

MSVAE is a music variational autoencoder with a structured, continuous latent space. This repository contains the **Stage II reconstruction checkpoint**: it accepts stereo audio at 24 kHz and produces stereo audio at 48 kHz. The encoder runs at 24 kHz, while the decoder generates the 48 kHz waveform. This is an audio reconstruction model, not a standalone text-to-music generator.

The model combines a frozen music self-supervised encoder with an acoustic encoder through a gated residual posterior. During training, separate parts of the 64-dimensional latent are aligned with compositional and linguistic-vocal references. The released checkpoint is for inference; training-time alignment features are not required to reconstruct audio.

## Files

| File | Description |
| --- | --- |
| `model.pt` | Stage II MSVAE reconstruction weights. The checkpoint contains a `model_state_dict`; it does not bundle the pretrained MuQ encoder. |
| `config.yml` | Inference settings: 24 kHz encoder input, 48 kHz stereo output, and model dimensions. |
| `bigvgan_v2_48khz_25hz_base_vae64.json` | Decoder configuration, also included in the companion code repository. |

The Stage I 24 kHz-to-24 kHz checkpoint is **not** included here. The `model.pt` file is approximately 721 MiB.

## Requirements

Inference requires the accompanying MSVAE code and its Python environment. The checkpoint is not directly loadable through `transformers.AutoModel`. The model also uses [OpenMuQ/MuQ-large-msd-iter](https://huggingface.co/OpenMuQ/MuQ-large-msd-iter), which the inference code loads separately with `MuQ.from_pretrained(...)`; MuQ weights are not redistributed in this repository. Please follow the [MuQ model license](https://huggingface.co/OpenMuQ/MuQ-large-msd-iter) for that dependency.

The code repository, installation instructions, paper, and audio demo links will be added when their public releases are available.

## Reconstruction

From the root of the accompanying MSVAE code repository, after installing its runtime dependencies:

```bash
CKPT_PATH=/absolute/path/to/model.pt \
bash scripts/infer.sh /absolute/path/to/input.wav ./reconstructions 0
```

The third positional argument selects GPU `0`. The script resamples input audio to 24 kHz before encoding and writes a 48 kHz stereo WAV file under `./reconstructions`. Mono input is duplicated to stereo. By default, it samples the VAE latent with seed 42. For deterministic decoding of the posterior mean, append `--latent-mode mean`:

```bash
CKPT_PATH=/absolute/path/to/model.pt \
bash scripts/infer.sh /absolute/path/to/input.wav ./reconstructions 0 --latent-mode mean
```

The default inference configuration in the code repository, `src/config/svae/example_new.yml`, has the same model settings as `config.yml` here. Its `DAC.bigvgan_conf` path is resolved relative to the code repository root, where the same decoder JSON is provided.

## Intended use and limitations

This checkpoint is intended for music reconstruction research and listening comparisons. The 48 kHz output is generated from a 24 kHz input; high-frequency detail is inferred by the decoder rather than recovered from the original waveform. Performance on other audio domains has not been established here. Use only input audio you are entitled to process and publish.

## License and citation

The MSVAE checkpoint license and paper citation have not yet been specified. Add the chosen license and public paper reference before publishing this model repository. The external MuQ weights have their own [CC BY-NC 4.0 license](https://huggingface.co/OpenMuQ/MuQ-large-msd-iter).