---
title: Phonological-Tokenizer
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/Phonological-Tokenizer"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/Phonological-Tokenizer.md"
repository: AI-ModelScope/Phonological-Tokenizer
last_updated: 2026-04-23
license: cc-by-sa-3.0
library_name:
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 3
stars: 0
---

# Phonological-Tokenizer

> Phonological-Tokenizer - AI-ModelScope 在 ModelScope 开源的模型。Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means

AI-ModelScope/Phonological-Tokenizer 是 ModelScope 魔搭社区上的机器学习模型，采用 cc-by-sa-3.0 许可。

- **Repository**: AI-ModelScope/Phonological-Tokenizer
- **License**: cc-by-sa-3.0
- **Downloads**: 3
- **Stars**: 0
- **Last updated**: 2026-04-23

Source: https://www.modelscope.cn/models/AI-ModelScope/Phonological-Tokenizer

---

## Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means

[[paper](https://arxiv.org/abs/2601.19781)] [[demo](https://ondatk68.github.io/onda-demo/projects/phonological-tokenizer/)]

![arch](./arch.png)

Phonological Tokenizer is a single-codebook speech tokenizer that encodes linguistic and prosodic information. The tokenizer has intermediate properties between phonetic tokens and acoustic tokens.

This tokenizer is obtained by fine-tuning the phonetic tokens derived from an SSL model (wavlm-large) using differentiable k-means  in a multi-task manner with ASR and speech reconstruction. In this repository, we release the fine-tuned SSL model and cluster centroids, along with simple inference code.

For more details, please refer to [our paper](https://arxiv.org/abs/2601.19781).

### Usage
```
git clone https://huggingface.co/Sony/Phonological-Tokenizer
cd Phonological-Tokenizer

pip install -r requirements.txt

python inference.py [audio file path]
```

### License
This model is licensed under CC BY-SA 3.0. See the [LICENSE file](./LICENSE) for details.

### Citation
```
@inproceedings{onda2026phonological,
  title={Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means},
  author={Onda, Kentaro and Futami, Hayato and Kashiwagi, Yosuke and Tsunoo, Emiru and Watanabe, Shinji},
  booktitle={ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={17817-17821},
  year={2026},
  organization={IEEE},
  doi={10.1109/ICASSP55912.2026.11464405}
}
```

### Reference
- Original SSL model: [WavLM-large](https://huggingface.co/microsoft/wavlm-large) (CC BY-SA 3.0)
- Training data: 
  - [VCTK](https://datashare.ed.ac.uk/handle/10283/3443) (CC BY 4.0)
  - [LibriSpeech](https://www.openslr.org/12) (CC BY 4.0; used a 30h random subset of train-clean-100 for centroid initialization)


### Contact
ondakentaro[at]gavo.t.u-tokyo.ac.jp; hayato.Futami[at]sony.com; Yosuke.Kashiwagi[at]sony.com