--- title: zipformer-large canonical_url: "https://www.modelscope.cn/models/pkufool/zipformer-large" md_url: "https://www.modelscope.cn/models/pkufool/zipformer-large.md" repository: pkufool/zipformer-large last_updated: 2026-06-18 license: "Apache License 2.0" pipeline_tag: auto-speech-recognition tasks: - auto-speech-recognition library_name: - onnx - pytorch frameworks: - Pytorch downloads: 1 stars: 0 --- # zipformer-large > zipformer-large - pkufool 在 ModelScope 开源的模型。This is a large zipformer model developed by Xiaomi AI Lab Next-gen-Kaldi team. The model was trained on around 20,0000 hours of open-sourced Chinese and English datasets. The number of parameters is around 150M. pkufool/zipformer-large 是 ModelScope 魔搭社区上的auto-speech-recognition模型，采用 Apache License 2.0 许可。 - **Repository**: pkufool/zipformer-large - **License**: Apache License 2.0 - **Tasks**: auto-speech-recognition - **Downloads**: 1 - **Stars**: 0 - **Last updated**: 2026-06-18 Source: https://www.modelscope.cn/models/pkufool/zipformer-large --- ## Introduction This is a large [zipformer](https://arxiv.org/pdf/2310.11230) model developed by Xiaomi AI Lab Next-gen-Kaldi team. The model was trained on around 20,0000 hours of open-sourced Chinese and English datasets. The number of parameters is around 150M. The performance on some popular test sets (CER for Chinese, WER for English). | Head | aishell test 1 / 2 | wenetspeech test-net/meetting | Common Voice zh | kespeech test | librispeech test-clean / other | gigaspeech test | Common voice en | tedium test | | -- | -- | -- | -- | -- | -- | -- | -- | -- | | CTC | 2.51 / 3.51 | 6.23 / 6.67 | 7.96 | 8.95 | 2.62 / 5.17 | 10.73 | 12.99 | 10.11 | | Transducer | 2.42 / 3.55 | 6.7 / 7.81 | 7.92 | 8.88 | 2.27 / 4.64 | 10.08 | 11.27 | 9.82 | Please refer to [zipformer in github](https://github.com/pkufool/zipformer) for model details. > Training set list: Librispeech, Gigaspeech, Commonvoice-2022(zh + en), Libriheavy, Emilia (zh+en), AIshell 2, Wenetspeech, Wenetspeech4tts, Kespeech, AIshell, aidatatang, aishell4, alimeeting, magicdata, primewords, stcmds, thchs30. ## Download model ### By SDK ``` # install modelscope pip install modelscope # download the model from modelscope import snapshot_download model_dir = snapshot_download('pkufool/zipformer-large') ``` ### By git ``` git clone https://www.modelscope.cn/pkufool/zipformer-large.git ``` ## Inference ### Install zipformer ``` pip install zipformer ``` ### Infer with CTC head #### Examples (with downloaded model) > You have to download the model by youself and provide the model path. ``` # jit script model zipformer inference \ --model zipformer-large/jit_model.pt \ --ctc 1 \ --model-type jit \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx model zipformer inference \ --model zipformer-large/ctc.onnx \ --ctc 1 \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --model zipformer-large/ctc.fp16.onnx \ --ctc 1 \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav ``` #### Example (with model name in modelscope) > You don't have to download the model, the model will be downloaded automatically from modelscope. ``` # jit script model zipformer inference \ --ms-model pkufool/zipformer-large \ --ctc 1 \ --model-type jit \ data/en.wav data/zh.wav # onnx model zipformer inference \ --ms-model pkufool/zipformer-large \ --ctc 1 \ --model-type onnx \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --ms-model pkufool/zipformer-large \ --ctc 1 \ --dtype fp16 \ --model-type onnx \ data/en.wav data/zh.wav ``` ### Infer with Transducer head #### Examples (with downloaded model) > You have to download the model by youself and provide the model path. ``` # jit script model zipformer inference \ --model zipformer-large/jit_model.pt \ --model-type jit \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx model zipformer inference \ --encoder zipformer-large/encoder.onnx \ --decoder zipformer-large/decoder.onnx \ --joiner zipformer-large/joiner.onnx \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --encoder zipformer-large/encoder.fp16.onnx \ --decoder zipformer-large/decoder.onnx \ --joiner zipformer-large/joiner.fp16.onnx \ --model-type onnx \ --tokens data/tokens.txt \ data/en.wav data/zh.wav ``` #### Example (with model name in modelscope) > You don't have to download the model, the model will be downloaded automatically from modelscope. ``` # jit script model zipformer inference \ --ms-model pkufool/zipformer-large \ --model-type jit \ data/en.wav data/zh.wav # onnx model zipformer inference \ --ms-model pkufool/zipformer-large \ --model-type onnx \ data/en.wav data/zh.wav # onnx fp16 model zipformer inference \ --ms-model pkufool/zipformer-large \ --dtype fp16 \ --model-type onnx \ data/en.wav data/zh.wav ``` ## Training ### Install zipformer ``` pip install zipformer ``` ``` export CUDA_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" zipformer train \ --world-size 8 \ --exp-dir zipformer/exp_large \ --num-encoder-layers 2,2,4,5,4,2 \ --feedforward-dim 512,768,1536,2048,1536,768 \ --encoder-dim 192,256,512,768,512,256 \ --encoder-unmasked-dim 192,192,256,320,256,192 \ --bpe-model zh-en-8776 \ --training-sets data/training_set.lst --num-epochs 20 \ --use-fp16 1 \ --start-epoch 1 \ --use-cr-ctc 1 \ --use-ctc 1 \ --base-lr 0.045 \ --use-transducer 1 \ --use-attention-decoder 0 \ --enable-spec-aug 0 \ --ctc-loss-scale 0.2 \ --cr-loss-scale 0.02 \ --time-mask-ratio 2.5 \ --lr-hours 50000 \ --num-workers 2 \ --max-duration 600 ``` ## Export model ### Install zipformer ``` pip install zipformer ``` ``` zipformer export \ --use-ctc 1 \ --use-transducer 1 \ --num-encoder-layers 2,2,4,5,4,2 \ --feedforward-dim 512,768,1536,2048,1536,768 \ --encoder-dim 192,256,512,768,512,256 \ --encoder-unmasked-dim 192,192,256,320,256,192 \ --exp-dir zipformer/exp_large \ --bpe-model zh-en-8776 \ --iter ITER \ --avg AVG ``` ## Citation ``` @inproceedings{yao2024zipformer, title={Zipformer: A faster and better encoder for automatic speech recognition}, author={Yao, Zengwei and Guo, Liyong and Yang, Xiaoyu and Kang, Wei and Kuang, Fangjun and Yang, Yifan and Jin, Zengrui and Lin, Long and Povey, Daniel}, booktitle={International Conference on Learning Representations}, volume={2024}, pages={44440--44455}, year={2024} } @inproceedings{yao2025cr, title={Cr-ctc: Consistency regularization on ctc for improved speech recognition}, author={Yao, Zengwei and Kang, Wei and Yang, Xiaoyu and Kuang, Fangjun and Guo, Liyong and Zhu, Han and Jin, Zengrui and Li, Zhaoqing and Lin, Long and Povey, Daniel}, booktitle={International Conference on Learning Representations}, volume={2025}, pages={26850--26868}, year={2025} } ```