--- title: zipformer-medium canonical_url: "https://www.modelscope.cn/models/pkufool/zipformer-medium" md_url: "https://www.modelscope.cn/models/pkufool/zipformer-medium.md" repository: pkufool/zipformer-medium last_updated: 2026-06-18 license: "Apache License 2.0" pipeline_tag: auto-speech-recognition tasks: - auto-speech-recognition library_name: - onnx - pytorch frameworks: - Pytorch downloads: 1 stars: 0 --- # zipformer-medium > zipformer-medium - pkufool 在 ModelScope 开源的模型。This is a medium zipformer model developed by Xiaomi AI Lab Next-gen-Kaldi team. The model was trained on around 20,0000 hours of open-sourced Chinese and English datasets. The number of parameters is around 68M… pkufool/zipformer-medium 是 ModelScope 魔搭社区上的auto-speech-recognition模型，采用 Apache License 2.0 许可。 - **Repository**: pkufool/zipformer-medium - **License**: Apache License 2.0 - **Tasks**: auto-speech-recognition - **Downloads**: 1 - **Stars**: 0 - **Last updated**: 2026-06-18 Source: https://www.modelscope.cn/models/pkufool/zipformer-medium --- ## Introduction This is a medium [zipformer](https://arxiv.org/pdf/2310.11230) model developed by Xiaomi AI Lab Next-gen-Kaldi team. The model was trained on around 20,0000 hours of open-sourced Chinese and English datasets. The number of parameters is around 68M (for ctc head), 73M (for transducer head). The performance on some popular test sets (CER for Chinese, WER for English). | Head | aishell test 1 / 2 | wenetspeech test-net/meetting | Common Voice zh | kespeech test | librispeech test-clean / other | gigaspeech test | Common voice en | tedium test | | -- | -- | -- | -- | -- | -- | -- | -- | -- | | CTC | 3.08 / 3.98 | 7.08 / 7.62 | 9.2 | 11.23| 3.01 / 6.06 | 11.22 | 15.28 | 10.38 | | Transducer | 2.67 / 3.67 | 6.79 / 7.33 | 8.97 | 10.67| 2.61 / 5.36 | 10.56 | 12.94 | 10.06 | Please refer to [zipformer in github](https://github.com/pkufool/zipformer) for model details. > Training set list: Librispeech, Gigaspeech, Commonvoice-2022(zh + en), Libriheavy, Emilia (zh+en), AIshell 2, Wenetspeech, Wenetspeech4tts, Kespeech, AIshell, aidatatang, aishell4, alimeeting, magicdata, primewords, stcmds, thchs30. ## Download model ### By SDK ``` # install modelscope pip install modelscope # download the model from modelscope import snapshot_download model_dir = snapshot_download('pkufool/zipformer-medium') ``` ### By git ``` git clone https://www.modelscope.cn/pkufool/zipformer-medium.git ``` ## Inference ### Install zipformer ``` pip install zipformer ``` ### Infer with CTC head #### Examples (with downloaded model) > You have to download the model by youself and provide the model path. ``` # jit script model zipformer inference \ --model zipformer-medium/jit_model.pt \ --ctc 1 \ --model-type jit \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx model zipformer inference \ --model zipformer-medium/ctc.onnx \ --ctc 1 \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --model zipformer-medium/ctc.fp16.onnx \ --ctc 1 \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav ``` #### Example (with model name in modelscope) > You don't have to download the model, the model will be downloaded automatically from modelscope. ``` # jit script model zipformer inference \ --ms-model pkufool/zipformer-medium \ --ctc 1 \ --model-type jit \ data/en.wav data/zh.wav # onnx model zipformer inference \ --ms-model pkufool/zipformer-medium \ --ctc 1 \ --model-type onnx \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --ms-model pkufool/zipformer-medium \ --ctc 1 \ --dtype fp16 \ --model-type onnx \ data/en.wav data/zh.wav ``` ### Infer with Transducer head #### Examples (with downloaded model) > You have to download the model by youself and provide the model path. ``` # jit script model zipformer inference \ --model zipformer-medium/jit_model.pt \ --model-type jit \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx model zipformer inference \ --encoder zipformer-meidum/encoder.onnx \ --decoder zipformer-medium/decoder.onnx \ --joiner zipformer-medium/joiner.onnx \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --encoder zipformer-medium/encoder.fp16.onnx \ --decoder zipformer-medium/decoder.onnx \ --joiner zipformer-medium/joiner.fp16.onnx \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav ``` #### Example (with model name in modelscope) > You don't have to download the model, the model will be downloaded automatically from modelscope. ``` # jit script model zipformer inference \ --ms-model pkufool/zipformer-medium \ --model-type jit \ data/en.wav data/zh.wav # onnx model zipformer inference \ --ms-model pkufool/zipformer-medium \ --model-type onnx \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --ms-model pkufool/zipformer-medium \ --dtype fp16 \ --model-type onnx \ data/en.wav data/zh.wav ``` ## Training ### Install zipformer ``` pip install zipformer ``` ``` export CUDA_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" zipformer train \ --world-size 8 \ --exp-dir zipformer/exp_medium \ --num-encoder-layers 2,2,3,4,3,2 \ --feedforward-dim 512,768,1024,1536,1024,768 \ --encoder-dim 192,256,384,512,384,256 \ --encoder-unmasked-dim 192,192,256,256,256,192 \ --bpe-model zh-en-8776 \ --training-sets data/training_set.lst --num-epochs 20 \ --use-fp16 1 \ --start-epoch 1 \ --use-cr-ctc 1 \ --use-ctc 1 \ --base-lr 0.045 \ --use-transducer 1 \ --use-attention-decoder 0 \ --enable-spec-aug 0 \ --ctc-loss-scale 0.2 \ --cr-loss-scale 0.02 \ --time-mask-ratio 2.5 \ --lr-hours 50000 \ --num-workers 2 \ --max-duration 600 ``` ## Export model ### Install zipformer ``` pip install zipformer ``` ``` zipformer export \ --use-ctc 1 \ --use-transducer 1 \ --num-encoder-layers 2,2,3,4,3,2 \ --feedforward-dim 512,768,1024,1536,1024,768 \ --encoder-dim 192,256,384,512,384,256 \ --encoder-unmasked-dim 192,192,256,256,256,192 \ --exp-dir zipformer/exp_medium \ --bpe-model zh-en-8776 \ --iter ITER \ --avg AVG ``` ## Citation ``` @inproceedings{yao2024zipformer, title={Zipformer: A faster and better encoder for automatic speech recognition}, author={Yao, Zengwei and Guo, Liyong and Yang, Xiaoyu and Kang, Wei and Kuang, Fangjun and Yang, Yifan and Jin, Zengrui and Lin, Long and Povey, Daniel}, booktitle={International Conference on Learning Representations}, volume={2024}, pages={44440--44455}, year={2024} } @inproceedings{yao2025cr, title={Cr-ctc: Consistency regularization on ctc for improved speech recognition}, author={Yao, Zengwei and Kang, Wei and Yang, Xiaoyu and Kuang, Fangjun and Guo, Liyong and Zhu, Han and Jin, Zengrui and Li, Zhaoqing and Lin, Long and Povey, Daniel}, booktitle={International Conference on Learning Representations}, volume={2025}, pages={26850--26868}, year={2025} } ```