---
title: olmOCR-mix-0225
canonical_url: "https://www.modelscope.cn/datasets/AI-ModelScope/olmOCR-mix-0225"
md_url: "https://www.modelscope.cn/datasets/AI-ModelScope/olmOCR-mix-0225.md"
repository: AI-ModelScope/olmOCR-mix-0225
last_updated: 2025-02-28
license: "Apache License 2.0"
storage_size: "52 GB"
downloads: 1759
stars: 4
---

# olmOCR-mix-0225

> olmOCR-mix-0225 - AI-ModelScope 在 ModelScope 开源的数据集。olmOCR-mix-0225 olmOCR-mix-0225 is a dataset of 250,000 PDF pages which have been OCRed into plain-text in a natural reading order using gpt-4o-2024-08-06 and a special prompting strategy that preserves any…

AI-ModelScope/olmOCR-mix-0225 是 ModelScope 魔搭社区上的数据集，存储大小 52 GB，采用 Apache License 2.0 许可。

- **Repository**: AI-ModelScope/olmOCR-mix-0225
- **License**: Apache License 2.0
- **Storage size**: 52 GB
- **Downloads**: 1759
- **Stars**: 4
- **Last updated**: 2025-02-28

Source: https://www.modelscope.cn/datasets/AI-ModelScope/olmOCR-mix-0225

---

# olmOCR-mix-0225
olmOCR-mix-0225 is a dataset of ~250,000 PDF pages which have been OCRed into plain-text in a natural reading order using gpt-4o-2024-08-06 and a special
prompting strategy that preserves any born-digital content from each page.

This dataset can be used to train, fine-tune, or evaluate your own OCR document pipeline.

Quick links:
- 📃 [Paper](https://olmocr.allenai.org/papers/olmocr.pdf)
- 🤗 [Model](https://huggingface.co/allenai/olmOCR-7B-0225-preview)
- 🛠️ [Code](https://github.com/allenai/olmocr)
- 🎮 [Demo](https://olmocr.allenai.org/)

## Data Mix

## Table 1: Training set composition by source
| Source | Unique docs | Total pages |
|--------|-------------|-------------|
| Web crawled PDFs | 99,903 | 249,332 |
| Internet Archive books | 5,601 | 16,803 |
| **Total** | **105,504** | **266,135** |

Web crawled PDFs are sampled from a set of over 240 million documents crawled from public websites. Books in the Internet Archive set are in the public domain.

## Table 2: Web PDFs breakdown by document type
| Document type | Fraction |
|---------------|----------|
| Academic | 60% |
| Brochure | 12% |
| Legal | 11% |
| Table | 6% |
| Diagram | 5% |
| Slideshow | 2% |
| Other | 4% |

Distribution is estimating by sampling 707 pages, which are classified using *gpt-4o-2024-11-20*.

## Data Format

Each row in the dataset corresponds to a single page, extracted at random, from a source PDF and transformed into plain text. 

No source PDF has had more than 3 random pages extracted from it.

Each extracted page is available as a standalone .pdf file, under the `pdf_tarballs/` directory.

### Features:
```python
{
    'url': string,         # Original URL of the PDF document
    'page_number': int,    # Page number within the document, 1-indexed
    'id': string,          # ID into /pdfs files folder
    'response': {          # OCRed Page information as JSON blob
        'primary_language': string,
        'is_rotation_valid': bool,
        'rotation_correction': int,
        'is_table': bool,
        'is_diagram': bool,
        'natural_text': str   # The actual text of the PDF is here
    }
}
```

## License
This dataset is licensed under ODC-BY-1.0. It is intended for research and educational use in accordance with AI2's [Responsible Use Guidelines](https://allenai.org/responsible-use).
The responses were generated from GPT-4o and GPT-4o is subject to OpenAI's [terms of use](https://openai.com/policies/row-terms-of-use).