---
title: "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion"
canonical_url: "https://www.modelscope.cn/papers/127088"
md_url: "https://www.modelscope.cn/papers/127088.md"
arxiv_id: 2503.11576
published: 2025-03-14
last_updated: 2025-03-14
authors:
  - "Ahmed Nassar"
  - "Andres Marafioti"
  - "Matteo Omenetti"
  - "Maksym Lysak"
  - "Nikolaos Livathinos"
  - "Christoph Auer"
  - "Lucas Morin"
  - "Rafael Teixeira de Lima"
  - "Yusik Kim"
  - "A. Said Gurbuz"
  - "Michele Dolfi"
  - "Miquel Farré"
  - "Peter W. J. Staar"
model_name: SmolDocling
model_developer: "IBM研究院, HuggingFace"
domain:
  - "计算机视觉"
  - "自然语言处理"
  - "深度学习"
type:
  - "计算机视觉"
  - "自然语言处理"
  - "深度学习"
  - "Computer Vision and Pattern Recognition (cs.CV)"
arxiv_url: "https://arxiv.org/abs/2503.11576"
pdf_url: "https://arxiv.org/pdf/2503.11576.pdf"
---

# SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

> We introduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full…

「SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion」是 ModelScope 魔搭社区收录的论文，arXiv 2503.11576，作者为 Ahmed Nassar, Andres Marafioti, Matteo Omenetti et al.，发表于 2025-03-14，属于 计算机视觉、自然语言处理、深度学习 领域。

- **ArXiv**: 2503.11576
- **Published**: 2025-03-14
- **Authors**: Ahmed Nassar, Andres Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos, Christoph Auer, Lucas Morin, Rafael Teixeira de Lima, Yusik Kim, A. Said Gurbuz, Michele Dolfi, Miquel Farré, Peter W. J. Staar
- **Model**: SmolDocling
- **Developer**: IBM研究院, HuggingFace
- **Domain**: 计算机视觉, 自然语言处理, 深度学习
- **ArXiv URL**: https://arxiv.org/abs/2503.11576
- **PDF**: https://arxiv.org/pdf/2503.11576.pdf

Source: https://www.modelscope.cn/papers/127088

---

> SmolDocling：用超紧凑视觉-语言模型实现文档转换的革命性突破

## 摘要

本文的研究背景在于复杂文档转换为结构化、可机读格式的技术挑战，尤其是PDF格式的不透明性和多样化布局风格带来的问题。现有方法包括复杂的流水线系统和大型多模态模型，但前者难以调整和泛化，后者计算资源消耗大且可能产生幻觉问题。为此，本文提出了一种超紧凑的视觉-语言模型SmolDocling，用于端到端的多模态文档转换。SmolDocling基于Hugging Face的SmolVLM-256M架构，参数量仅为2.5亿，远小于其他类似模型，同时能够准确捕捉文档内容、结构和空间位置。该模型通过生成DocTags（一种新的通用标记格式）来表示整个页面的所有元素及其上下文信息。此外，作者还贡献了新的公开数据集，涵盖图表、表格、公式和代码识别任务，并扩展了现有数据集的特征注释。实验结果表明，SmolDocling在多个任务上的表现与比其大27倍的模型相当，同时显著降低了计算需求。SmolDocling的独特之处在于它能够统一表示文档的内容、结构和空间位置，适用于多种类型的文档，包括商业文件、学术论文、技术报告、专利和表单等。

## Abstract

We introduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms -- significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon.