---
title: speech_whisper-large_lid_multilingual_pytorch
canonical_url: "https://www.modelscope.cn/models/iic/speech_whisper-large_lid_multilingual_pytorch"
md_url: "https://www.modelscope.cn/models/iic/speech_whisper-large_lid_multilingual_pytorch.md"
repository: iic/speech_whisper-large_lid_multilingual_pytorch
chinese_name: "Whisper-Large多语种识别模型"
last_updated: 2024-03-25
license: "Apache License 2.0"
pipeline_tag: speech-language-recognition
tasks:
  - speech-language-recognition
library_name:
  - pytorch
frameworks:
  - Pytorch
downloads: 29474
stars: 9
---

# speech_whisper-large_lid_multilingual_pytorch

> speech_whisper-large_lid_multilingual_pytorch - iic 在 ModelScope 开源的模型。Whisper-LID模型介绍 Highlights 支持语种识别，包括中文、英文、日语、韩语等多个语种。 基于语种识别结果，配合Whisper模型在未知语种场景下进行语音识别。

iic/speech_whisper-large_lid_multilingual_pytorch 是 ModelScope 魔搭社区上的speech-language-recognition模型，采用 Apache License 2.0 许可。

- **Repository**: iic/speech_whisper-large_lid_multilingual_pytorch
- **License**: Apache License 2.0
- **Tasks**: speech-language-recognition
- **Downloads**: 29474
- **Stars**: 9
- **Last updated**: 2024-03-25

Source: https://www.modelscope.cn/models/iic/speech_whisper-large_lid_multilingual_pytorch

---

# Whisper-LID模型介绍
## Highlights
- 支持语种识别，包括中文、英文、日语、韩语等多个语种。
- 基于语种识别结果，配合Whisper模型在未知语种场景下进行语音识别。

## <strong>[FunASR开源项目介绍](https://github.com/alibaba-damo-academy/FunASR)</strong>
<strong>[FunASR](https://github.com/alibaba-damo-academy/FunASR)</strong>希望在语音识别的学术研究和工业应用之间架起一座桥梁。通过发布工业级语音识别模型的训练和微调，研究人员和开发人员可以更方便地进行语音识别模型的研究和生产，并推动语音识别生态的发展。让语音识别更有趣！

[**github仓库**](https://github.com/alibaba-damo-academy/FunASR)
| [**最新动态**](https://github.com/alibaba-damo-academy/FunASR#whats-new) 
| [**环境安装**](https://github.com/alibaba-damo-academy/FunASR#installation)
| [**服务部署**](https://www.funasr.com)
| [**模型库**](https://github.com/alibaba-damo-academy/FunASR/tree/main/model_zoo)
| [**联系我们**](https://github.com/alibaba-damo-academy/FunASR#contact)

## 模型原理介绍
Whisper是OpenAI发布的支持多语种语音识别的语音系统，其在许多benchmark上都取得了不错的性能。在此，我们基于Whisper编码器得到的语音特征，使用EResNet作为语种识别模块，提供比原始Whisper更为准确的语种识别能力。

## 基于ModelScope进行推理
```python
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

multilingual_wavs=[
    "https://www.modelscope.cn/api/v1/models/iic/speech_whisper-large_lid_multilingual_pytorch/repo?Revision=master&FilePath=examples/example_zh-CN.mp3",
    "https://www.modelscope.cn/api/v1/models/iic/speech_whisper-large_lid_multilingual_pytorch/repo?Revision=master&FilePath=examples/example_en.mp3",
    "https://www.modelscope.cn/api/v1/models/iic/speech_whisper-large_lid_multilingual_pytorch/repo?Revision=master&FilePath=examples/example_ja.mp3",
    "https://www.modelscope.cn/api/v1/models/iic/speech_whisper-large_lid_multilingual_pytorch/repo?Revision=master&FilePath=examples/example_ko.mp3",
]

inference_pipeline = pipeline(
    task=Tasks.auto_speech_recognition,
    model='iic/speech_whisper-large_lid_multilingual_pytorch', model_revision="v2.0.4")

for wav in multilingual_wavs:
    rec_result = inference_pipeline(input=wav)
    print(rec_result)
```

## 基于FunASR进行推理
```python
from funasr import AutoModel

multilingual_wavs = [
    "example_zh-CN.mp3",
    "example_en.mp3",
    "example_ja.mp3",
    "example_ko.mp3",
]

model = AutoModel(model="iic/speech_whisper-large_lid_multilingual_pytorch", model_revision="v2.0.4")
for wav_id in multilingual_wavs:
    wav_file = f"{model.model_path}/examples/{wav_id}"
    res = model.generate(input=wav_file, data_type="sound")
    print("detect sample {}: {}".format(wav_id, res))
```

## 相关论文以及引用信息

```BibTeX
@inproceedings{radford2023robust,
  title={Robust speech recognition via large-scale weak supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  booktitle={International Conference on Machine Learning},
  pages={28492--28518},
  year={2023},
  organization={PMLR}
}
```