---
title: "Qwen3-Omni Technical Report"
canonical_url: "https://www.modelscope.cn/papers/190355"
md_url: "https://www.modelscope.cn/papers/190355.md"
arxiv_id: 2509.17765
published: 2025-09-22
last_updated: 2025-09-22
authors:
  - "Jin Xu"
  - "Zhifang Guo"
  - "Hangrui Hu"
  - "Yunfei Chu"
  - "Xiong Wang"
  - "Jinzheng He"
  - "Yuxuan Wang"
  - "Xian Shi"
  - "Ting He"
  - "Xinfa Zhu"
  - "Yuanjun Lv"
  - "Yongqi Wang"
  - "Dake Guo"
  - "He Wang"
  - "Linhan Ma"
  - "Pei Zhang"
  - "Xinyu Zhang"
  - "Hongkun Hao"
  - "Zishan Guo"
  - "Baosong Yang"
  - "Bin Zhang"
  - "Ziyang Ma"
  - "Xipin Wei"
  - "Shuai Bai"
  - "Keqin Chen"
  - "Xuejing Liu"
  - "Peng Wang"
  - "Mingkun Yang"
  - "Dayiheng Liu"
  - "Xingzhang Ren"
  - "Bo Zheng"
  - "Rui Men"
  - "Fan Zhou"
  - "Bowen Yu"
  - "Jianxin Yang"
  - "Le Yu"
  - "Jingren Zhou"
  - "Junyang Lin"
model_name: Qwen3-Omni
model_developer: "Qwen Team"
domain:
  - "多模态大模型"
  - "语音识别与合成"
  - "计算机视觉"
  - "自然语言处理"
  - "流式推理"
type:
  - "多模态大模型"
  - "语音识别与合成"
  - "计算机视觉"
  - "自然语言处理"
  - "流式推理"
  - "Computation and Language"
  - "Artificial Intelligence"
  - "Computer Vision and Pattern Recognition"
  - "Audio and Speech Processing"
arxiv_url: "https://arxiv.org/abs/2509.17765"
pdf_url: "https://arxiv.org/pdf/2509.17765"
code_link: "https://github.com/QwenLM/Qwen3-Omni"
---

# Qwen3-Omni Technical Report

> We present Qwen3-Omni, a single multimodal model that, for the first time, maintains state-of-the-art performance across text, image, audio, and video without any degradation relative to single-modal counterparts. Qwen3-Omni matches the performance of…

「Qwen3-Omni Technical Report」是 ModelScope 魔搭社区收录的论文，arXiv 2509.17765，作者为 Jin Xu, Zhifang Guo, Hangrui Hu et al.，发表于 2025-09-22，属于 多模态大模型、语音识别与合成、计算机视觉 领域。

- **ArXiv**: 2509.17765
- **Published**: 2025-09-22
- **Authors**: Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo Zheng, Rui Men, Fan Zhou, Bowen Yu, Jianxin Yang, Le Yu, Jingren Zhou, Junyang Lin
- **Model**: Qwen3-Omni
- **Developer**: Qwen Team
- **Domain**: 多模态大模型, 语音识别与合成, 计算机视觉, 自然语言处理, 流式推理
- **ArXiv URL**: https://arxiv.org/abs/2509.17765
- **PDF**: https://arxiv.org/pdf/2509.17765
- **Code**: https://github.com/QwenLM/Qwen3-Omni

Source: https://www.modelscope.cn/papers/190355

---

> Qwen3-Omni：首个全模态无损SOTA的端到端多模态大模型

## 摘要

本文提出了Qwen3-Omni，一个在文本、图像、音频和视频多种模态上均达到最先进性能且无性能退化的统一多模态模型。该模型基于Thinker-Talker架构，并引入混合专家（MoE）设计，分别用于推理（Thinker）和实时语音生成（Talker）。通过自研的Audio Transformer（AuT）编码器和多码本语音生成机制，Qwen3-Omni在36项音频与音视频基准测试中取得32项开源SOTA和22项整体SOTA，超越Gemini-2.5-Pro、GPT-4o-Transcribe等闭源模型。系统支持119种语言文本交互、19种语言语音理解与10种语言语音生成，并可处理长达40分钟的音频输入。为实现低延迟流式合成，模型采用因果卷积网络（ConvNet）替代扩散模型，理论端到端首包延迟低至234毫秒。此外，团队还发布了专用于音频描述的Qwen3-Omni-Captioner模型，显著提升音频内容的语义理解能力。

## Abstract

We present Qwen3-Omni, a single multimodal model that, for the first time, maintains state-of-the-art performance across text, image, audio, and video without any degradation relative to single-modal counterparts. Qwen3-Omni matches the performance of same-sized single-modal models within the Qwen series and excels particularly on audio tasks. Across 36 audio and audio-visual benchmarks, Qwen3-Omni achieves open-source SOTA on 32 benchmarks and overall SOTA on 22, outperforming strong closed-source models such as Gemini-2.5-Pro, Seed-ASR, and GPT-4o-Transcribe. Qwen3-Omni adopts a Thinker-Talker MoE architecture that unifies perception and generation across text, images, audio, and video, yielding fluent text and natural real-time speech. It supports text interaction in 119 languages, speech understanding in 19 languages, and speech generation in 10 languages. To reduce first-packet latency in streaming synthesis, Talker autoregressively predicts discrete speech codecs using a multi-codebook scheme. Leveraging the representational capacity of these codebooks, we replace computationally intensive block-wise diffusion with a lightweight causal ConvNet, enabling streaming from the first codec frame. In cold-start settings, Qwen3-Omni achieves a theoretical end-to-end first-packet latency of 234 ms. To further strengthen multimodal reasoning, we introduce a Thinking model that explicitly reasons over inputs from any modality. Since the research community currently lacks a general-purpose audio captioning model, we fine-tuned Qwen3-Omni-30B-A3B to obtain Qwen3-Omni-30B-A3B-Captioner, which produces detailed, low-hallucination captions for arbitrary audio inputs. Qwen3-Omni-30B-A3B, Qwen3-Omni-30B-A3B-Thinking, and Qwen3-Omni-30B-A3B-Captioner are publicly released under the Apache 2.0 license.