---
title: moss_nano_tts_onnx_100M_INT8
canonical_url: "https://www.modelscope.cn/models/hzp2000itb/moss_nano_tts_onnx_100M_INT8"
md_url: "https://www.modelscope.cn/models/hzp2000itb/moss_nano_tts_onnx_100M_INT8.md"
repository: hzp2000itb/moss_nano_tts_onnx_100M_INT8
chinese_name: moss_nano_tts_onnx_100M_INT8
last_updated: 2026-05-28
pipeline_tag: text-to-speech
tasks:
  - text-to-speech
base_model_relation: finetune
library_name:
  - onnx
  - other
frameworks:
  - other
downloads: 27
stars: 3
---

# moss_nano_tts_onnx_100M_INT8

> moss_nano_tts_onnx_100M_INT8 - hzp2000itb 在 ModelScope 开源的模型。全面优化 https://modelscope.cn/models/openmoss/MOSS-TTS-Nano-100M 的onnx 格式模型、进行int8精度量化、并实现onnxruntime cpu版本的加速推理，保证在cpu核上做到实时流式RTF指标低于1。

- **Repository**: hzp2000itb/moss_nano_tts_onnx_100M_INT8
- **Tasks**: text-to-speech
- **Downloads**: 27
- **Stars**: 3
- **Last updated**: 2026-05-28

Source: https://www.modelscope.cn/models/hzp2000itb/moss_nano_tts_onnx_100M_INT8

---

# MOSS-TTS-Nano ONNX

中文 | English

## 中文说明

### 简介

这是一个基于 MOSS-TTS-Nano 的本地 ONNX 推理仓库，面向 CPU 部署场景，支持：

- 文本转语音（TTS）
- 基于参考声纹的 voice clone
- LLM token 流到句子级语音的流式合成
- INT8 KV-Cache 推理
- 通道跳过与动态填充优化

本仓库模型基于 ModelScope 上的 [openmoss/MOSS-TTS-Nano-100M](https://modelscope.cn/models/openmoss/MOSS-TTS-Nano-100M) 进行 INT8 量化，并导出为 ONNX 格式。

原始模型虽然仅有 0.1B 规模，但在 CPU 上即使经过常规推理加速，RTF 仍然难以稳定达到小于 1 的水平，因此在实时流式场景下仍难以达到可实用的工程标准。

本仓库中的量化与推理优化策略由作者独立设计实现，重点面向低资源 CPU 场景下的时延与吞吐优化，目标是进一步降低 CPU 推理成本并提升实时流式语音合成的可用性。

### 特性

- 主模型拆分为 `global_prefill` / `global_step` / `local_prefill` / `local_step`
- 音频 tokenizer 编码器与解码器采用 ONNX 部署
- 支持声纹缓存复用，减少重复编码开销
- 支持流式文本输入触发句子级增量语音生成
- 支持非关键通道跳过与中位数/动态填充两种优化策略

### 目录结构

```text
.
├── moss_tts_handler.py
├── simulate_llm_stream_tts.py
├── tts_inference_with_cache_onnx_optimized.py
├── onnx
│   ├── audio_tokenizer
│   │   ├── config.json
│   │   ├── encoder_int8.onnx
│   │   └── decoder_int8.onnx
│   ├── tts_nano
│   │   ├── config.json
│   │   ├── tokenizer.model
│   │   ├── tokenizer_config.json
│   │   ├── special_tokens_map.json
│   │   └── quantized
│   │       ├── global_prefill_int8.onnx
│   │       ├── global_step_int8.onnx
│   │       ├── local_prefill_int8.onnx
│   │       ├── local_step_int8.onnx
│   │       ├── lm_heads.onnx
│   │       └── embeddings.npz
│   └── voice_prints
│       ├── male_voice_kv.npz
│       └── male_voice_kv_median_tokens.json
├── requirements.txt
├── LICENSE
└── README.md
```

### 环境要求

- Python 3.10+
- Linux / macOS / Windows
- CPU 环境可运行
- 推荐使用 `onnxruntime` CPU provider

### 安装依赖

```bash
pip install -r requirements.txt
```

### 快速开始

#### 1. 使用已有声纹缓存做单次合成

```bash
python tts_inference_with_cache_onnx_optimized.py \
  --text "你好，这是一个 ONNX 量化语音合成测试。" \
  --voice_name male_voice_kv
```

#### 2. 当声纹缓存不存在时，用你自己的参考音频编码声纹

```bash
python tts_inference_with_cache_onnx_optimized.py \
  --text "你好，这是一个 ONNX 量化语音合成测试。" \
  --voice_name my_voice \
  --ref_audio /path/to/your_reference.wav \
  --force_encode
```

#### 3. 模拟 LLM 流式输出并做流式语音生成

```bash
python simulate_llm_stream_tts.py \
  --text "你好，我正在模拟大语言模型逐步输出文本，并实时触发语音合成。" \
  --chunk-size 1 \
  --delay 0.1 \
  --voice-print-name male_voice_kv
```

#### 4. 保存每段流式音频分片

```bash
python simulate_llm_stream_tts.py \
  --text "这是一个流式 TTS 示例。" \
  --save-chunks
```

### Python 调用示例

```python
import asyncio
from moss_tts_handler import MossTTSConfigModel, MossTTSHandler

async def main():
    handler = MossTTSHandler()
    config = MossTTSConfigModel(
        enabled=True,
        voice_print_name="male_voice_kv",
    )
    handler.load(config)
    audio = await handler.synthesize("你好，这是一个测试。")
    handler.close()

asyncio.run(main())
```

如果目标声纹缓存不存在，则必须显式提供 `ref_audio`。

### 核心文件说明

- `moss_tts_handler.py`：面向应用集成的主处理器接口
- `simulate_llm_stream_tts.py`：模拟 LLM 文本流并做流式合成
- `tts_inference_with_cache_onnx_optimized.py`：单次推理与性能优化示例
- `onnx/tts_nano/config.json`：主模型结构与 token 配置
- `onnx/audio_tokenizer/config.json`：音频 tokenizer 配置
- `onnx/voice_prints/*.npz`：声纹缓存文件

### 量化与推理优化

本仓库包含以下优化方向：

- INT8 ONNX 导出与部署
- Global / Local 双层推理路径拆分
- KV-Cache 推理缓存
- 非关键音频通道跳过
- 中位数填充与动态填充策略
- 可调 ONNX Runtime 线程参数

这些策略的目标是降低 CPU 推理耗时，并在音质与速度之间取得工程上可用的平衡。

### 优化动机 / 方法概述 / 效果目标

优化动机在于：原始 0.1B TTS 模型即使模型规模不大，在 CPU 场景下的主要瓶颈仍然集中在自回归 step 路径、多音频通道逐帧采样、局部 Transformer 的重复计算，以及未针对小核低并发推理负载细化的 ONNX Runtime 执行配置。这些问题会直接推高单句合成时延与流式输出间隔，使实时流式场景难以达到稳定可用的工程标准。

方法上，本仓库围绕推理链路做了面向 CPU 的结构化优化：首先将主模型导出为 INT8 ONNX，并拆分为 `global_prefill`、`global_step`、`local_prefill`、`local_step` 四条执行路径，以降低不同阶段的计算与缓存开销；其次在自回归生成阶段引入 KV-Cache 与局部 attention cache，减少重复状态构建；同时基于音频通道重要性分析，对非关键通道采用跳过生成策略，并结合中位数填充或动态填充，在尽量控制音质退化的前提下显著减少 `local_step` 调用次数；此外还针对 prefill、step、decoder 分别暴露独立线程参数，使 ONNX Runtime 的线程模型更贴合 CPU 推理实际负载。

效果目标上，这套方法并不追求单纯的模型压缩，而是面向实时流式语音合成的整体工程可用性优化：在保持可接受音质与声纹一致性的前提下，尽可能压低 CPU 侧合成时延、改善吞吐与 RTF 表现，并推动系统更接近实时流式交互所需的实用部署区间。

### 使用限制与免责声明

- 本仓库仅用于研究、学习、评估与合规场景下的语音合成应用开发
- 使用 voice clone 功能时，你必须拥有参考音频的合法授权
- 禁止将本项目用于身份冒充、诈骗、侵权、伪造证据或其他违法违规用途
- 仓库中公开的声纹缓存不代表授权你克隆任意真实人物声音
- 使用者需自行承担合规与法律责任

### License

本项目代码与仓库发布内容采用 `Apache-2.0`。

请注意：如果你后续添加第三方样本、参考音频或外部权重，需要自行确认这些资产的再分发许可。

---

## English

### Overview

This repository provides a local ONNX inference setup for MOSS-TTS-Nano, optimized for CPU deployment scenarios. It supports:

- text-to-speech (TTS)
- voice cloning with reference voice prints
- streaming synthesis from LLM text token streams
- INT8 KV-cache inference
- channel skipping and dynamic filling optimizations

The model in this repository is based on [openmoss/MOSS-TTS-Nano-100M](https://modelscope.cn/models/openmoss/MOSS-TTS-Nano-100M) from ModelScope, quantized to INT8 precision, and exported to ONNX format.

Although the original model is only 0.1B in size, its real-time factor (RTF) on CPU is still difficult to keep consistently below 1 even with conventional inference acceleration, which makes it hard to reach a practical engineering level for real-time streaming scenarios.

The quantization and inference optimization strategy in this repository is independently designed and implemented by the author, focusing on low-resource CPU inference efficiency and aiming to further improve the practicality of real-time streaming speech synthesis.

### Features

- split inference path with `global_prefill`, `global_step`, `local_prefill`, and `local_step`
- ONNX-based audio tokenizer encoder and decoder
- reusable cached voice prints
- sentence-level incremental synthesis for streaming text input
- non-critical channel skipping with median or dynamic fill strategies

### Requirements

- Python 3.10+
- Linux / macOS / Windows
- CPU runtime supported
- recommended runtime: `onnxruntime` CPU provider

### Installation

```bash
pip install -r requirements.txt
```

### Quick Start

#### 1. Synthesize with an existing cached voice print

```bash
python tts_inference_with_cache_onnx_optimized.py \
  --text "Hello, this is a test for ONNX quantized speech synthesis." \
  --voice_name male_voice_kv
```

#### 2. Encode your own reference audio when the cache does not exist

```bash
python tts_inference_with_cache_onnx_optimized.py \
  --text "Hello, this is a test for ONNX quantized speech synthesis." \
  --voice_name my_voice \
  --ref_audio /path/to/your_reference.wav \
  --force_encode
```

#### 3. Simulate LLM streaming text and generate streaming audio

```bash
python simulate_llm_stream_tts.py \
  --text "Hello, I am simulating incremental LLM output and triggering streaming speech synthesis in real time." \
  --chunk-size 1 \
  --delay 0.1 \
  --voice-print-name male_voice_kv
```

### File Guide

- `moss_tts_handler.py`: main application-facing TTS handler
- `simulate_llm_stream_tts.py`: streaming demo driven by simulated LLM text output
- `tts_inference_with_cache_onnx_optimized.py`: one-shot inference and optimization demo
- `onnx/tts_nano/config.json`: main model configuration
- `onnx/audio_tokenizer/config.json`: audio tokenizer configuration
- `onnx/voice_prints/*.npz`: cached voice print assets

### Optimization Notes

This repository includes:

- INT8 ONNX deployment
- global/local dual-path inference
- KV-cache acceleration
- non-critical audio channel skipping
- median fill and dynamic fill strategies
- tunable ONNX Runtime threading parameters

These optimizations are intended to reduce CPU inference latency while preserving practical synthesis quality.

### Motivation / Method / Goal

The optimization motivation is straightforward: although the original 0.1B TTS model is relatively small, the main CPU bottlenecks still come from the autoregressive step path, frame-by-frame multi-channel audio token sampling, repeated local Transformer computation, and generic ONNX Runtime settings that are not tuned for low-concurrency CPU inference workloads. In practice, these factors increase sentence-level latency and streaming gaps, making real-time deployment difficult.

Methodologically, this repository applies a structured CPU-oriented optimization pipeline. The model is first quantized to INT8 and exported to ONNX, then split into `global_prefill`, `global_step`, `local_prefill`, and `local_step` execution paths to reduce stage-specific compute and cache overhead. During autoregressive generation, KV-cache and local attention cache are used to avoid redundant state construction. In addition, non-critical audio channels are skipped based on channel importance analysis, with median-fill or dynamic-fill strategies used to trade a limited quality loss for a significant reduction in `local_step` calls. The repository also exposes separate threading controls for prefill, step, and decoder stages so that ONNX Runtime can better match real CPU inference behavior.

The target is not merely model compression, but practical deployment quality for real-time streaming TTS. Under acceptable synthesis quality and voice consistency, the goal is to reduce CPU-side latency, improve throughput and RTF behavior, and move the system closer to a practically usable operating range for interactive streaming applications.

### Distribution Notes

For Hugging Face and ModelScope publishing:

- large files should be managed with Git LFS
- `*.onnx`, `*.npz`, and `*.wav` are covered by `.gitattributes`
- only cached voice print files are intended for publication; raw reference audio is not included by default

### Responsible Use

- This repository is intended for research, evaluation, education, and compliant application development
- You must have legal authorization for any reference audio used in voice cloning
- Do not use this project for impersonation, fraud, infringement, fabricated evidence, or unlawful activity
- Published cached voice prints do not grant permission to clone arbitrary real individuals
- The user is responsible for legal and policy compliance

### License

This repository is released under `Apache-2.0`.

If you later add third-party samples, reference audio, or external weights, please verify their redistribution terms separately.
