---
title: VoxCPM2-LiteRT
canonical_url: "https://www.modelscope.cn/models/aufklarer/VoxCPM2-LiteRT"
md_url: "https://www.modelscope.cn/models/aufklarer/VoxCPM2-LiteRT.md"
repository: aufklarer/VoxCPM2-LiteRT
last_updated: 2026-06-16
license: apache-2.0
pipeline_tag: text-to-speech
tasks:
  - text-to-speech
base_model:
  - openbmb/VoxCPM2
base_model_relation: finetune
library_name:
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
  - id
  - ja
  - ko
  - multilingual
downloads: 9
stars: 0
tags:
  - tts
  - text-to-speech
  - voxcpm
  - voice-cloning
  - voice-design
  - litert
  - tflite
  - android
---

# VoxCPM2-LiteRT

> VoxCPM2-LiteRT - aufklarer 在 ModelScope 开源的模型。LiteRT / TensorFlow Lite port of openbmb/VoxCPM2 — a 2B-parameter multilingual diffusion-autoregressive TTS model with 48 kHz studio-quality output, voice cloning, and instruction-driven voice design.

aufklarer/VoxCPM2-LiteRT 是 ModelScope 魔搭社区上的text-to-speech模型，采用 apache-2.0 许可，基于 openbmb/VoxCPM2 构建。

- **Repository**: aufklarer/VoxCPM2-LiteRT
- **License**: apache-2.0
- **Tasks**: text-to-speech
- **Base model**: openbmb/VoxCPM2
- **Tags**: tts, text-to-speech, voxcpm, voice-cloning, voice-design, litert, tflite, android
- **Downloads**: 9
- **Stars**: 0
- **Last updated**: 2026-06-16

Source: https://www.modelscope.cn/models/aufklarer/VoxCPM2-LiteRT

---

# VoxCPM2 LiteRT

LiteRT / TensorFlow Lite port of [openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) — a 2B-parameter multilingual diffusion-autoregressive TTS model with **48 kHz** studio-quality output, voice cloning, and instruction-driven voice design.

This bundle ships the model as **four separate LiteRT graphs** plus a manifest. The on-device worker is expected to orchestrate the loop:

```
text_prefill ─► token_step (×N) ─► audio_decode
```

Reference audio is encoded once via `audio_encoder`. The K/V cache for the LM and residual decoder is owned by the host worker (not mutated inside the graph), which lets the runtime own retry / idempotency semantics.

Part of [soniqo.audio](https://soniqo.audio) — an on-device speech toolkit. Consumed by the Android SDK at [`speech-android`](https://github.com/soniqo/speech-android).

> **Status:** experimental. The `token_step` graph depends on `litert_torch` static K/V-cache lowering; integrators should validate numerical parity end-to-end before relying on this bundle in production.

## Capabilities

- **30 languages** including English, Chinese, Indonesian, Japanese, Korean
- **48 kHz output**
- **Zero-shot synthesis** — generate speech from text alone
- **Voice cloning** — clone a target speaker from a single reference clip
- **Voice design** — natural-language style control (e.g. *"young female voice, warm and gentle"*)
- **Ultimate cloning** — reference audio + transcript for prosody-preserving cloning

## Files

| File | Variant | Role |
|------|---------|------|
| `voxcpm2-text-prefill.tflite` | FP16 weights / FP32 compute | Encode text + (optional) reference-audio prefix into LM hidden states, residual hidden, prefix feature conditioning, and the initial K/V caches. |
| `voxcpm2-token-step.tflite`   | FP16 weights / FP32 compute, **LocDiT diffusion kept FP32** | One AR step. Takes current LM / residual hidden, conditioning, K/V cache and position id. Emits the next predicted feature, stop logits, updated hidden states, and updated K/V cache. |
| `voxcpm2-audio-encoder.tflite` | FP32 | Encode a reference clip (16 kHz PCM) into the patch features that condition the prefill. |
| `voxcpm2-audio-decoder.tflite` | FP32 | Decode latent → 48 kHz PCM via the upstream AudioVAE. |
| `config.json` | — | Manifest: tensor signatures, sample rates, default CFG / step counts, file mapping. |
| `tokenizer.json` / `tokenizer_config.json` / `special_tokens_map.json` / `generation_config.json` | — | HF tokenizer + generation defaults. |
| `tokenization_voxcpm2.py` | — | Upstream tokenizer source (kept for parity with the HF model). |

The encoder and decoder are kept FP32 (the AudioVAE is sensitive to quantisation). The LM graphs are FP16-weight / FP32-compute **except the LocDiT diffusion estimator, which is held at FP32** — sub-FP32 precision there turns the noise-like texture of consonants into tonal structure, producing sandy/metallic sibilants on cloned voices. This `selective` recipe matches full-FP32 cloning quality at roughly half the size (~8.7 GB vs ~17 GB).

## Default decoding parameters

| Parameter | Default |
|-----------|---------|
| `max_text_tokens` (context) | 512 |
| `max_generated_tokens` | 2048 |
| `inference_timesteps` (CFM) | 10 |
| `cfg_value` | 2.0 |
| Sample rate (output) | 48 000 Hz |
| Sample rate (audio conditioning) | 16 000 Hz |

These mirror the host-side defaults exposed in `config.json`; runtimes are free to override them.

## Token-step cache contract

The token-step graph takes and returns the LM and residual K/V cache as explicit inputs/outputs. Cache layout:

```
[2, layers, batch, kv_heads, max_cache_length, head_dim]
```

- Axis 0: `[K, V]`
- Axis 4 is sized to `max_text_tokens + max_generated_tokens` and pre-allocated by the worker.
- The graph does not mutate the cache buffers in place — it produces updated tensors which the worker copies / swaps.

This contract is what makes parallel decoding, mid-generation cancellation, and deterministic replay possible from the C++ side.

## Source

Converted from the upstream PyTorch weights at [openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) using `litert_torch` + `ai_edge_quantizer` float-casting (FP16 weights, FP32 compute), with the LocDiT diffusion estimator held at FP32 — the `selective` recipe (`convert_litert.py --variant selective`).

## Links

- [speech-android](https://github.com/soniqo/speech-android) — Android SDK
- [soniqo.audio](https://soniqo.audio) — website
- [blog](https://soniqo.audio/blog) — blog

## License

Apache 2.0 (inherited from upstream openbmb/VoxCPM2).

## Responsible use

Voice cloning capability is included. Users are responsible for obtaining consent for any voice that is cloned and for not using the model to impersonate individuals without their permission, generate disinformation, or commit fraud.