---
title: MaritimeOCRBench
canonical_url: "https://www.modelscope.cn/datasets/HiDolphin/MaritimeOCRBench"
md_url: "https://www.modelscope.cn/datasets/HiDolphin/MaritimeOCRBench.md"
repository: HiDolphin/MaritimeOCRBench
last_updated: 2026-06-27
license: "Apache License 2.0"
storage_size: "518 MB"
domain:
  - image
tasks:
  - image-to-text
size_scale:
  - 100-10k
downloads: 105387
stars: 2
---

# MaritimeOCRBench

> MaritimeOCRBench - HiDolphin 在 ModelScope 开源的数据集。Maritime-OCR-Bench 是一个面向海事任务设计的 OCR 与文档理解基准。本数据集中与海事相关的内容主要包括集装箱箱体编号与箱门标识、提单和运输单证、港口与货运相关单据，以及船舶或船机设备说明书等材料。除此之外，数据中也包含多种真实世界文档、扫描页、表格、票据、发票、报告以及场景化文本图像，用于覆盖更广泛的 OCR 与文档理解场景。

HiDolphin/MaritimeOCRBench 是 ModelScope 魔搭社区上的image-to-text数据集，涉及 image 领域，存储大小 518 MB，采用 Apache License 2.0 许可。

- **Repository**: HiDolphin/MaritimeOCRBench
- **License**: Apache License 2.0
- **Tasks**: image-to-text
- **Domain**: image
- **Storage size**: 518 MB
- **Downloads**: 105387
- **Stars**: 2
- **Last updated**: 2026-06-27

Source: https://www.modelscope.cn/datasets/HiDolphin/MaritimeOCRBench

---

# Maritime-OCR-Bench

[简体中文](README.zh-CN.md)

## Overview

Maritime-OCR-Bench is an OCR and document understanding benchmark built around maritime-oriented tasks. In the maritime portion of the dataset, the samples mainly include container identification markings, bills of lading and shipping forms, port and cargo documents, and ship or engine manuals. Beyond these materials, the dataset also includes a broader mix of real-world documents, scanned pages, tables, receipts, invoices, reports, and scene-style text images.

Maritime-OCR-Bench is organized as an instruction-following multimodal conversation dataset. Each sample includes image paths, a user instruction, a reference answer, and a task type. The task design covers both free-form text understanding and strictly formatted outputs: VQA and Parsing emphasize semantic understanding, reading order, and long-text reconstruction, while IE and Spotting place more emphasis on JSON validity, field coverage, layout coordinates, and output-structure stability.

![Maritime-OCR-Bench overview](assets/ocrbench.png)

## Task Types

This dataset includes 5 task types and 1888 samples.

| Task | task_type in dataset | Samples | Description |
| --- | --- | ---: | --- |
| IE | `IE` | 471 | Strict information extraction with JSON-formatted answers. |
| VQA | `vqa` | 471 | Visual question answering over cross-domain documents, tables, forms, receipts, and scene-style images. |
| Parsing | `parsing` | 472 | OCR parsing / full-text recognition, including long-form structured text. |
| JSON Spotting v1 | `json1` | 237 | Text spotting with image-grouped JSON output for systems that consume results by page or image. |
| JSON Spotting v2 | `json2` | 237 | Text spotting with unified `images`-based JSON output for systems that require a general downstream schema. |

Task distribution in this dataset:

![Maritime-OCR-Bench task distribution](assets/task_distribution.png)

## Data Format

The annotation file of this dataset uses JSONL format. Each sample is stored as one JSON object per line, and the top-level fields are:

- `images`: relative image path list.
- `messages`: multi-turn conversation containing the user instruction and the reference answer.
- `task_type`: task identifier.

Example:

```json
{
  "messages": [
    {
      "role": "user",
      "content": "<image>Please extract all key information and return it in JSON format."
    },
    {
      "role": "assistant",
      "content": "{\n  \"field_a\": \"value\"\n}"
    }
  ],
  "images": [
    "images/example_image.jpg"
  ],
  "task_type": "IE"
}
```

## File Structure

```text
MaritimeOCRBench/
├── README.md
├── README.zh-CN.md
├── assets/
├── maritime_ocr_bench.jsonl
└── images/
```

The repository is organized into the following parts:

- `README.md`: English overview, task definitions, evaluation protocol, and reference results.
- `README.zh-CN.md`: Chinese overview, task definitions, evaluation protocol, and reference results.
- `assets/`: figures used in the README and supplementary documentation.
- `maritime_ocr_bench.jsonl`: the main annotation file of this dataset.
- `images/`: image assets referenced by the dataset annotation file.

## Evaluation

Evaluation uses task-specific metrics:

- VQA / Parsing: a composite text score is used. Before scoring, the text is normalized and compared from four views: normalized text, compact text, loose text, and table-aware text. The final score is a weighted sum where edit-distance similarity, character-level F1, LCS F1, loose edit-distance similarity, and table-aware edit-distance similarity have weights of 0.25, 0.20, 0.15, 0.20, and 0.20 respectively. `text_score = 0.25 * edit_sim_v1 + 0.20 * char_f1_v2 + 0.15 * lcs_f1_v1 + 0.20 * loose_edit_sim + 0.20 * table_aware_sim`.
- IE: a combined score of text coverage and strict JSON validity is used. Text coverage measures how much of the reference-answer text is covered by the model output, without extra penalty for additional completion text. At the same time, the completion must be strict JSON text, with no extra explanation or Markdown code fences. `ie_score = 0.5 * text_coverage_score + 0.5 * json_strict_score`.
- Spotting Json1 / Json2: layout localization score and composite text score are computed separately and then aggregated with weights. `overall_score = 0.7 * diou_score + 0.3 * text_score`.

## evalscope Integration

You can evaluate `maritime_ocr_bench` with the official `modelscope/evalscope` repository.

Example setup:

```bash
git clone https://github.com/modelscope/evalscope.git
cd evalscope
```

Then run the benchmark with:

```python
from evalscope import TaskConfig, run_task

task_cfg = TaskConfig(
  model='qwen-vl-max',
  api_url='https://dashscope.aliyuncs.com/compatible-mode/v1',
  api_key='',
  datasets=['maritime_ocr_bench'],
  eval_batch_size=20,
  generation_config={
    'retries': 1,
    'retry_interval': 1,
    'timeout': 300,
  },
  ignore_errors=True,
  limit=None,
  debug=False,
)

run_task(task_cfg=task_cfg)
```

## Reference Results

### Full-set Results (1888 samples)

| Model | Samples | Overall | IE | VQA | Parsing | Spotting_Json1 (2D array output) | Spotting_Json2 (1D array output) |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Hi-Dolphin-OCR | 1888 | 0.7941 | 0.7795 | 0.7817 | 0.8260 | 0.7700 | 0.8161 |

![Maritime-OCR-Bench radar chart](assets/radar.png)