---
title: OCR-KIE
canonical_url: "https://www.modelscope.cn/datasets/megemini/OCR-KIE"
md_url: "https://www.modelscope.cn/datasets/megemini/OCR-KIE.md"
repository: megemini/OCR-KIE
chinese_name: OCR-KIE
last_updated: 2026-07-18
license: CC-BY-4.0
storage_size: "1.9 GB"
domain:
  - image
tasks:
  - image-to-text
size_scale:
  - 100-10k
downloads: 2305
stars: 0
---

# OCR-KIE

> OCR-KIE - megemini 在 ModelScope 开源的数据集。OCR-KIE

megemini/OCR-KIE 是 ModelScope 魔搭社区上的image-to-text数据集，涉及 image 领域，存储大小 1.9 GB，采用 CC-BY-4.0 许可。

- **Repository**: megemini/OCR-KIE
- **License**: CC-BY-4.0
- **Tasks**: image-to-text
- **Domain**: image
- **Storage size**: 1.9 GB
- **Downloads**: 2305
- **Stars**: 0
- **Last updated**: 2026-07-18

Source: https://www.modelscope.cn/datasets/megemini/OCR-KIE

---

# OCR Key Information Extraction Dataset

> Full dataset can be downloaded: https://modelscope.cn/datasets/megemini/OCR-KIE

This project combines 7 different OCR datasets into a unified format for training and evaluating document information extraction models.

![](https://ai-studio-static-online.cdn.bcebos.com/03cd12faa9f94b95b6c1d89d2f6f83d933ee2a2d717248f494c3de5cded14c84)

## Dataset Categories

| Category | Datasets | Records | Description |
|----------|----------|---------|-------------|
| **Invoices** | FATURA | 2,500 | Invoice documents with billing information |
| **Receipts** | SROIE, wildreceipt, CORD-v2 | 3,712 | Various receipt types (retail, restaurant, commercial) |
| **Forms** | xfund, SIBR | 2,393 | Structured forms with key-value pairs |
| **Labels** | nutritional-data-poie-1 | 483 | Product and nutritional information labels |

**Note:** All images have been resized to max 2048px (width or height) to reduce VRAM usage while maintaining OCR readability.

**Total: 8,889 records** (after removing records with no extractable fields)

## Dataset Overview

| Dataset | Category | Description | Records |
|---------|----------|-------------|---------|
| FATURA | Invoices | Invoice documents (50 templates × 50 instances) | 2,500 |
| SROIE | Receipts | Receipt information extraction | 973 |
| wildreceipt | Receipts | Wild receipt recognition | 1,739 |
| CORD-v2 | Receipts | Commercial receipts | 1,000 |
| xfund | Forms | Multi-language form understanding (7 languages) | 1,393 |
| SIBR | Forms | Structured information in documents | 1,000 |
| nutritional-data-poie-1 | Labels | Nutritional information labels | 483 |

## Modifications

The following modifications have been made to the original datasets:

### 1. Information Extraction
- Extracted key-value pairs from all 7 datasets into a unified JSONL format
- Preserved original field names from annotations
- Structured data as JSON strings for easy parsing

### 2. FATURA Dataset Trimming
- Original: 50 templates × 200 instances = 10,000 records
- Modified: 50 templates × 50 instances = 2,500 records
- Kept instances 0-49 for each template

### 3. Image Processing
- **PNG to JPG Conversion**: Converted 1,485 PNG images to JPG format
  - Reduced cord-v2 size: 1,397.8 MB → 286.5 MB (79% reduction)
  - Reduced nutritional size: 274.3 MB → 77.6 MB (72% reduction)
- **Image Resizing**: All images resized to max 2048px (width or height)
  - Reduced VRAM usage while maintaining OCR readability
- **xFund Smart Resize**: Applied adaptive scaling to all 1,393 xFund images
  - Hybrid algorithm (density + SWT) for training set; simple proportional for validation set
  - zh/ja median scale 0.400; other languages median scale 0.581

### 4. Header Normalization & Translation
- `header` fields normalized to `\n`-joined strings (was list in xfund/SIBR, string elsewhere)
- Per-language header translations applied at extraction time:

  | Language | Translation |
  |----------|-------------|
  | Chinese (zh) | `标题` |
  | German (de) | `Kopfzeile` |
  | Spanish (es) | `Encabezado` |
  | French (fr) | `En-tête` |
  | Italian (it) | `Intestazione` |
  | Japanese (ja) | `ヘッダー` |
  | Portuguese (pt) | `Cabeçalho` |

- Header deduplication via `--remove_header`: `header` removed when translation exists

### 5. Train/Test Split
- Applied 9:1 fixed ratio across all datasets
- Train: 8,000 records (90%)
- Test: 889 records (10%)

### 6. Dataset Expansion
- Generated additional training records with random field subsets
- Added structured mask records preserving full field structure
- Total expanded training set: 48,000 records

## Output Format

Each record in the JSONL files follows this format:

```json
{
  "image_info": [
    {"matched_text_index": 0, "image_url": "path/to/image.jpg"}
  ],
  "text_info": [
    {"text": "OCR:{}", "tag": "mask"},
    {"text": "{\"key\": \"value\", ...}", "tag": "no_mask"}
  ]
}
```

- `image_info`: Contains the image reference
- `text_info[0]`: OCR placeholder with "mask" tag
- `text_info[1]`: Extracted information as JSON string with "no_mask" tag

## Dataset Details

### Invoices (2,500 records)

#### FATURA
- **Description**: Invoice documents with various templates
- **Fields**: Invoice number, date, due date, seller/buyer info, items, totals
- **Languages**: English
- **Split**: 9:1 random split
- **Templates**: 50 different invoice layouts

### Receipts (3,712 records)

#### SROIE (973 records)
- **Description**: Receipt information extraction
- **Fields**: Company name, date, address, total
- **Languages**: English
- **Split**: Original train/test

#### wildreceipt (1,739 records)
- **Description**: Wild receipt recognition
- **Fields**: Store info, items, prices, quantities, totals
- **Languages**: English
- **Split**: Original train/test

#### CORD-v2 (1,000 records)
- **Description**: Commercial receipts
- **Fields**: Menu items, subtotals, totals
- **Languages**: Various (Korean, English)
- **Split**: Train/test (valid skipped)

### Forms (2,393 records)

#### xfund (1,393 records)
- **Description**: Multi-lingual form understanding dataset
- **Languages**: Chinese (zh), German (de), Spanish (es), French (fr), Italian (it), Japanese (ja), Portuguese (pt)
- **Fields**: Headers, questions, answers
- **Split**: Train/val from original dataset

#### SIBR (1,000 records)
- **Description**: Structured information in documents
- **Fields**: Headers, key-value pairs
- **Languages**: Chinese, English
- **Split**: Original train/test

### Labels (483 records)

#### nutritional-data-poie-1
- **Description**: Nutritional information labels
- **Fields**: Calories, fat, protein, carbs, sodium, etc.
- **Languages**: English
- **Split**: Train/test (valid skipped)
- **Split**: Original train/test

## Scripts

### Individual Dataset Scripts

Each dataset has its own extraction script:

| Dataset | Script | Usage |
|---------|--------|-------|
| FATURA | `FATURA/extract_fatura.py` | `python extract_fatura.py --images_dir <dir> --pseudo_path` |
| xfund | `xfund/extract_xfund.py` | `python extract_xfund.py --dataset_dir <dir> --pseudo_path` |
| SROIE | `SROIE_2019_with_labels/extract_sroie.py` | `python extract_sroie.py --data_dir <dir> --pseudo_path` |
| wildreceipt | `wildreceipt/extract_wildreceipt.py` | `python extract_wildreceipt.py --data_dir <dir> --pseudo_path` |
| CORD-v2 | `cord-v2/extract_cord.py` | `python extract_cord.py --data_dir <dir> --pseudo_path` |
| nutritional | `nutritional-data-poie-1/extract_nutritional.py` | `python extract_nutritional.py --data_dir <dir> --pseudo_path` |
| SIBR | `SIBR/extract_sibr.py` | `python extract_sibr.py --data_dir <dir> --pseudo_path` |

### Unified Extraction Script

```bash
# Reproduce current dataset (9:1 split, header dedup)
python unified_extract.py --fixed_ratio 0.9 --remove_header

# Prefix image URLs (for S3 or relocated images)
python unified_extract.py --fixed_ratio 0.9 --prefix "/data/datasets/"

# Remove 'other' field
python unified_extract.py --fixed_ratio 0.9 --remove_header --remove_other
```

**Parameters:**
- `--data_dir`: Base directory containing all datasets (default: current directory)
- `--pseudo_path`: Use pseudo image paths (`path/to/image.jpg`) instead of real paths
- `--output_dir`: Output directory for train.jsonl and test.jsonl (default: current directory)
- `--fixed_ratio`: Force fixed train/test ratio (e.g., `0.9` for 9:1, `0.8` for 8:2)
- `--prefix`: Prefix to prepend to image URLs (e.g., `/data/datasets/` or `s3://bucket/images/`)
- `--remove_header`: Remove original `header` field when translation exists
- `--remove_other`: Remove `other` field from records

### Dataset Expansion Script

Expand training data by generating additional records with random field subsets.

```bash
# Generate 4 extended records per original record (seed fixed at 42)
python expand_dataset.py --input train.jsonl --output train_expanded.jsonl --extend 4

# Generate 4 extended records + 1 structured mask record per original
python expand_dataset.py --input train.jsonl --output train_expanded.jsonl --extend 4 --structured
```

**Parameters:**
- `-i, --input`: Input JSONL file (e.g., `train.jsonl`)
- `-o, --output`: Output JSONL file (e.g., `train_expanded.jsonl`)
- `-e, --extend`: Number of additional entries per record (default: 3)
- `--structured`: Add one structured mask record per original (preserves full structure)

**Note:** Random seed is fixed at 42 to ensure reproducibility.

**How it works:**
- Original record: `OCR:{}` with all fields in no_mask
- Extended records: `OCR:{"field1": "", "field2": []}` with only selected fields
- Structured mask records: preserves full structure with type-aware placeholders

**Mask format examples:**
- String value: `"field": ""`
- List of strings: `"field": []`
- List of dicts: `"field": [{"key": ""}]` (preserves structure of first element)
- Nested dict: `"field": {"k1": "", "k2": ""}`

### Image Processing Scripts

#### Convert PNG to JPG

Convert all PNG images to JPG format to reduce file size.

```bash
python convert_to_jpg.py
```

**Note:** All PNG images have been converted to JPG format. This reduced:
- cord-v2: 1,397.8 MB → 286.5 MB (79% reduction)
- nutritional: 274.3 MB → 77.6 MB (72% reduction)

#### Resize Images (simple)

Resize images that exceed a maximum dimension threshold to reduce VRAM usage.

```bash
# Resize images exceeding 2048px (default, overwrites original)
python resize_images.py --input train.jsonl

# Resize with custom max dimension
python resize_images.py --input train.jsonl --max_dim 1024
```

**Parameters:**
- `-i, --input`: Input JSONL file
- `--images_dir`: Base directory for images (default: current directory)
- `--max_dim`: Maximum width or height in pixels (default: 2048)

**Note:** All images have been resized with max_dim=2048 to reduce VRAM usage while maintaining OCR readability.

#### Smart Resize xFund Images (adaptive scaling)

`xfund_resize.py` uses content-aware scaling algorithms to intelligently resize xFund images by analysing text density and stroke width. Unlike simple max-dim resize, this adapts per-image: dense documents are shrunk more aggressively, sparse documents are treated conservatively.

```bash
# Process both train and val splits (overwrites originals)
python xfund_resize.py

# Override max dimension cap
python xfund_resize.py --max_dim 2048

# Preview without modifying
python xfund_resize.py --dry_run
```

**Parameters:**
- `--max_dim`: Maximum width or height after scaling (default: 2048)
- `--dry_run`: Print what would be done without modifying images

**Processing strategy:**
- **Train split**: hybrid algorithm (density + SWT + bbox protection) — per-image adaptive
- **Val split**: simple proportional resize using per-language median scale computed from training set

## Adaptive Image Scaling Algorithms

`xfund_resize.py` implements three algorithms for content-aware image downscaling. This section explains each algorithm, its performance characteristics, and when to use it.

### Algorithm 1: Information Density Scaling

**Core idea** (original algorithm by user): Count foreground (text) pixels, divide by the number of characters in the document to get average pixels-per-character, then compute the maximum safe downscale ratio that keeps each character above a minimum pixel threshold.

**Steps:**
1. Binarise the image with Otsu thresholding to separate text from background
2. Count foreground pixels (dark text on light background)
3. Divide by ground-truth character count to get `avg_pixels_per_char`
4. Apply a dynamic margin factor to discount non-text foreground noise (table lines, borders, noise):
   - Text ratio > 30%: margin = 1.2× (dense text, less noise)
   - Text ratio 10–30%: margin = 1.5× (medium density)
   - Text ratio < 10%: margin = 2.1× (sparse text, more noise)
5. `scale = sqrt(min_pixels_per_char / effective_pixels_per_char)`
6. Clamp with `min_scale = 0.4` (prevents over-shrinking on sparse documents)
7. Apply bbox floor to protect thin characters (punctuation, narrow strokes):
   - Ensures that characters ≤ 3px wide don't shrink below 1px width
   - Ensures that characters ≤ 8px tall don't shrink below 4px height

**Performance:**
- ~0.02s per image (on 2480×3508 xFund documents)
- Average downscale: ~64%
- Best for: clean scanned documents with available ground-truth character counts

**Effect on xFund (7 languages, 1393 images):**

| Language | Median scale | Typical output | Avg reduction |
|----------|-------------|----------------|--------------|
| zh | 0.400 | 992×1376 | 84% |
| de | 0.581 | 1440×2048 | 66% |
| es | 0.581 | 1440×2048 | 66% |
| fr | 0.581 | 1440×2048 | 66% |
| it | 0.581 | 1440×2048 | 66% |
| ja | 0.400 | 992×1376 | 84% |
| pt | 0.581 | 1440×2048 | 66% |

**When to use:**
- Documents with clear text/background contrast
- Ground-truth character counts are available (or can be estimated via connected components)
- Speed is important (fastest algorithm)
- Text is mostly uniform size and font

**When to avoid:**
- Natural scene text with complex backgrounds
- Images with heavy noise or watermarks that inflate foreground pixel counts
- Documents with extremely sparse text (density algorithm may over-shrink; `min_scale` mitigates this)

### Algorithm 2: Stroke Width Transform (SWT) Scaling

**Core idea:** Measure the physical stroke width of text characters using distance transform (as a fast approximation to full Stroke Width Transform). The distance from each foreground pixel to the nearest background pixel approximates half the stroke width. Then compute the safe scale that keeps stroke widths above a target threshold (default: 4 pixels).

**Steps:**
1. Binarise and apply morphological opening to remove isolated noise
2. Compute distance transform on the binary image
3. For each foreground pixel, stroke width ≈ 2 × distance value
4. Compute 10th and 50th percentiles of stroke width distribution
5. `scale = min(target_stroke_width / p50, target_stroke_width / p10, 1.0)`
6. Clamp with `min_scale = 0.4`

**Why distance transform instead of full SWT:**
- Full Stroke Width Transform (Epshtein et al., CVPR 2010) requires per-pixel ray casting along gradient directions, which is O(n × ray_length) and very slow on high-resolution images (~0.5–2s per 2480×3508 image)
- Distance transform achieves similar signal (stroke width estimation) in O(n) with ~0.07s per image
- For clean document scans, distance transform closely approximates true stroke width

**Performance:**
- ~0.10s per image
- Conservative for high-resolution scanned documents (median stroke width 2–4px ≈ target 4px)
- Average downscale: ~7% (mostly near 1.0 on xFund; rarely triggers)

**When to use:**
- Natural scene text (street signs, billboards, product packaging)
- Complex backgrounds with noise, stamps, handwriting
- Multi-font documents where stroke width varies significantly
- As a safety net alongside density algorithm (in hybrid mode)

**When to avoid:**
- High-resolution scanned documents with adequate stroke widths (will not trigger)
- Speed-critical batch processing (slower than density)

### Algorithm 3: Hybrid (Density + SWT + Bbox)

**Core idea:** Combine all three safeguards into one robust algorithm:
1. Compute density scale (per-image text density analysis)
2. Compute SWT scale (per-image stroke width analysis)
3. Take the more conservative (smaller) of the two scales
4. Apply bbox floor (protect thin punctuation from disappearing)
5. Apply `min_scale = 0.4` (prevent over-shrinking)
6. Apply `max_dim` cap (default: ensure both dimensions ≤ 2048px)

This is the **default algorithm for the training set** in `xfund_resize.py`. Density drives the scaling decision; SWT provides a safety veto when unusually thin strokes are detected.

**Performance:**
- ~0.13s per image (includes both density and SWT computation)
- Average downscale: ~65% on xFund (with max_dim=2048 cap)
- Successfully protects thin punctuation (zh: 2px colons preserved at 1px+)
- Prevents over-shrinking of sparse documents (ja: from 544×384 → 1376×992)

### Summary: Which Algorithm to Use

| Scenario | Algorithm | Rationale |
|----------|-----------|-----------|
| Monochrome docs with GT char count | **Density** | Fastest, most accurate for this case |
| Monochrome docs with no char count available | **Density** (auto-estimate) | Connected components provide reasonable estimate |
| Natural scene text, variable fonts | **SWT** | Measures stroke width directly, ignores background |
| Mixed document types, production | **Hybrid** | Best safety, combines both approaches |
| Validation / test sets | **Simple proportional** | Use median scale from training set |

### File Size Reduction Examples (xFund)

| Image | Original | After hybrid resize | Reduction |
|-------|----------|-------------------|-----------|
| zh_train_0 (dense form) | 2480×3508 (1785 KB) | 1216×1760 (451 KB) | 75% |
| de_train_0 (dense form) | 2480×3508 (2634 KB) | 1440×2048 (708 KB) | 73% |
| ja_train_0 (sparse form, 75 chars) | 3508×2480 (614 KB) | 1376×992 (106 KB) | 83% |
| it_train_1 (dense form, 4051 chars) | 2480×3508 (884 KB) | 1440×2048 (839 KB) | 5% |

## Output Statistics

### Without Fixed Ratio (Original splits)
- Train: 6,924 records
- Test: 1,968 records

### With Fixed 9:1 Ratio (--fixed_ratio 0.9 --remove_header)
- Train: 8,000 records (90%)
- Test: 889 records (10%)
- Header deduplication via `--remove_header`: `header` removed when translation exists
- Image extensions auto-fixed (PNG→JPG mismatch resolved)
- Test records use structured mask: `OCR:{"field1": "", "field2": []}`

### Expanded Dataset (with --extend 4 --structured)
- train_expanded.jsonl: 48,000 records
  - 8,000 original records (mask: `OCR:{}`)
  - 32,000 extended records (random field subsets)
  - 8,000 structured mask records (full structure preserved)

## Data Processing Notes

1. **Key Preservation**: Original field names from annotations are preserved
2. **JSON String Format**: Extracted data is stored as JSON strings for easy parsing
3. **Pseudo Paths**: Use `--pseudo_path` to generate generic image paths
4. **Multi-value Handling**: Multiple values for the same field are stored as lists
5. **Nested Structures**: Hierarchical data is preserved (e.g., menu items with quantities and prices)

## File Structure

```
ALL_DATA/
├── unified_extract.py          # Main extraction script
├── expand_dataset.py           # Dataset expansion script
├── resize_images.py            # Simple max-dim image resizer
├── convert_to_jpg.py           # PNG→JPG converter
├── xfund_resize.py             # Adaptive xFund image scaling
├── AGENTS.md                   # Developer guide for this repo
├── README.md                   # This file
├── LICENSE                     # CC BY 4.0 license
├── train.jsonl                 # Combined training data (8,000 records)
├── test.jsonl                  # Combined test data (889 records)
├── train_expanded.jsonl        # Expanded training data (48,000 records)
├── FATURA/                     # Invoice dataset
├── xfund/                      # Multi-language forms
├── SROIE_2019_with_labels/     # Receipt extraction
├── wildreceipt/                # Wild receipts
├── cord-v2/                    # Commercial receipts
├── nutritional-data-poie-1/    # Nutritional labels
└── SIBR/                       # Structured documents
```

## Citations

If you use this dataset collection, please cite the original datasets:

### FATURA
```bibtex
@article{fatura2023,
  title={FATURA: A Dataset for Invoice Information Extraction},
  year={2023}
}
```

### XFUND
```bibtex
@article{xu2021layoutxlm,
  title={LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding},
  author={Xu, Yiheng and Lv, Tengchao and Cui, Lei and Wang, Guoxin and Lu, Yijuan and Florencio, Dinei and Zhang, Cha and Wei, Furu},
  journal={arXiv preprint arXiv:2104.08836},
  year={2021}
}
```

### SROIE
```bibtex
@inproceedings{mishra2019icdar,
  title={ICDAR 2019 Competition on Scanned Receipt OCR and Information Extraction (SROIE)},
  author={Mishra, Ravi and Shetty, Santosh and Alahari, Srikar},
  booktitle={International Conference on Document Analysis and Recognition (ICDAR)},
  year={2019}
}
```

### WildReceipt
```bibtex
@inproceedings{li2021wildreceipt,
  title={WildReceipt: A Large-scale Receipt Dataset for Document Information Extraction},
  author={Li, Yudi and Liu, Yongkui and Wang, Jiaqi},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  year={2021}
}
```

### CORD-v2
```bibtex
@inproceedings{park2019cord,
  title={CORD: A Consolidated Receipt Dataset for Post-OCR Parsing},
  author={Park, Seungyeon and Kim, Sangdoo and Lee, Seungju and Sohn, Jooyoung and Kim, Joonho and Ha, Seungwon},
  booktitle={Workshop on Document Intelligence at NeurIPS},
  year={2019}
}
```

### Nutritional Data POIE-1
```bibtex
@article{nguyen2021nutritional,
  title={Nutritional Data POIE: A Dataset for Post-OCR Information Extraction from Nutritional Labels},
  author={Nguyen, Anh and others},
  year={2021}
}
```

### SIBR
```bibtex
@article{sibr2023,
  title={SIBR: A Dataset for Structured Information Extraction from Business Receipts},
  year={2023}
}
```

## License

This dataset collection is licensed under the **Creative Commons Attribution 4.0 International License (CC BY 4.0)**.

You are free to:
- **Share**: copy and redistribute the material in any medium or format
- **Adapt**: remix, transform, and build upon the material for any purpose, even commercially

Under the following terms:
- **Attribution**: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

For more details, see the [LICENSE](LICENSE) file or visit https://creativecommons.org/licenses/by/4.0/

### Original Dataset Licenses

| Dataset | Original License | Citation Required |
|---------|------------------|-------------------|
| FATURA | CC BY 4.0 | Yes |
| XFUND | MIT License | Yes |
| SROIE | CC BY 4.0 | Yes |
| WildReceipt | CC BY 4.0 | Yes |
| CORD-v2 | CC BY 4.0 | Yes |
| Nutritional Data POIE-1 | CC BY 4.0 | Yes |
| SIBR | CC BY 4.0 | Yes |

**Note**: When using this dataset collection, please cite both this dataset and the original dataset papers as specified in the Citations section.