---
title: "Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models"
canonical_url: "https://www.modelscope.cn/papers/126955"
md_url: "https://www.modelscope.cn/papers/126955.md"
arxiv_id: 2503.11224
published: 2025-03-14
last_updated: 2025-03-14
authors:
  - "Xingtai Lv"
  - "Youbang Sun"
  - "Kaiyan Zhang"
  - "Shang Qu"
  - "Xuekai Zhu"
  - "Yuchen Fan"
  - "Yi Wu"
  - "Ermo Hua"
  - "Xinwei Long"
  - "Ning Ding"
  - "Bowen Zhou"
model_name: "State Space Models (SSMs)"
model_developer: "清华大学电子工程系，上海人工智能实验室，卡内基梅隆大学机器人研究所"
domain:
  - "自然语言处理"
  - "深度学习"
  - "机器学习"
  - "计算机视觉"
  - "语音技术"
type:
  - "自然语言处理"
  - "深度学习"
  - "机器学习"
  - "计算机视觉"
  - "语音技术"
  - "Machine Learning (cs.LG)"
  - "Artificial Intelligence (cs.AI)"
  - "Computation and Language (cs.CL)"
arxiv_url: "https://arxiv.org/abs/2503.11224"
pdf_url: "https://arxiv.org/pdf/2503.11224.pdf"
---

# Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models

> State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts, demonstrating…

「Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models」是 ModelScope 魔搭社区收录的论文，arXiv 2503.11224，作者为 Xingtai Lv, Youbang Sun, Kaiyan Zhang et al.，发表于 2025-03-14，属于 自然语言处理、深度学习、机器学习 领域。

- **ArXiv**: 2503.11224
- **Published**: 2025-03-14
- **Authors**: Xingtai Lv, Youbang Sun, Kaiyan Zhang, Shang Qu, Xuekai Zhu, Yuchen Fan, Yi Wu, Ermo Hua, Xinwei Long, Ning Ding, Bowen Zhou
- **Model**: State Space Models (SSMs)
- **Developer**: 清华大学电子工程系，上海人工智能实验室，卡内基梅隆大学机器人研究所
- **Domain**: 自然语言处理, 深度学习, 机器学习, 计算机视觉, 语音技术
- **ArXiv URL**: https://arxiv.org/abs/2503.11224
- **PDF**: https://arxiv.org/pdf/2503.11224.pdf

Source: https://www.modelscope.cn/papers/126955

---

> 状态空间模型：从连续到离散的革新之旅

## 摘要

本文围绕状态空间模型（SSMs）展开研究，探讨其在序列数据处理中的有效性和效率。与Transformer架构相比，SSMs在处理长文本或序列数据时表现出显著的性能优势和更高的计算效率。论文将SSM的发展分为三个阶段：原始SSM、结构化SSM（如S4）和选择性SSM（如Mamba）。第一阶段的原始SSM通过离散化连续时间模型，能够处理离散序列数据；第二阶段的S4通过引入卷积表达形式降低计算复杂度，解决了长期依赖捕捉问题；第三阶段的Mamba通过优化硬件适配和计算公式，实现了接近Transformer的性能。此外，论文详细介绍了多种关键技术，包括欧拉方法、零阶保持法和双线性变换等离散化技术，以及LegS、HiPPO等提升模型性能的技术。这些技术使SSMs在视频处理、分子建模、语音分析等领域展现出广泛应用前景。总体而言，本文为研究人员提供了关于SSMs理论基础和实际应用的全面介绍。

## Abstract

State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts, demonstrating comparable performances with significant efficiency gains. In this survey, we provide a coherent and systematic overview for SSMs, including their theoretical motivations, mathematical formulations, comparison with existing model classes, and various applications. We divide the SSM series into three main sections, providing a detailed introduction to the original SSM, the structured SSM represented by S4, and the selective SSM typified by Mamba. We put an emphasis on technicality, and highlight the various key techniques introduced to address the effectiveness and efficiency of SSMs. We hope this manuscript serves as an introduction for researchers to explore the theoretical foundations of SSMs.