---
title: OvisOCR2-GGUF
canonical_url: "https://www.modelscope.cn/models/Abiray/OvisOCR2-GGUF"
md_url: "https://www.modelscope.cn/models/Abiray/OvisOCR2-GGUF.md"
repository: Abiray/OvisOCR2-GGUF
last_updated: 2026-07-14
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
base_model:
  - ATH-MaaS/OvisOCR2
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 1620
stars: 7
tags:
  - ocr
  - document-parsing
  - multimodal
  - markdown
  - tables
  - formulas
  - gguf
  - llama-cpp
---

# OvisOCR2-GGUF

> OvisOCR2-GGUF - Abiray 在 ModelScope 开源的模型。OvisOCR2 - GGUF Quantizations

Abiray/OvisOCR2-GGUF 是 ModelScope 魔搭社区上的image-text-to-text模型，采用 apache-2.0 许可，基于 ATH-MaaS/OvisOCR2 构建。

- **Repository**: Abiray/OvisOCR2-GGUF
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Base model**: ATH-MaaS/OvisOCR2
- **Tags**: ocr, document-parsing, multimodal, markdown, tables, formulas, gguf, llama-cpp
- **Downloads**: 1620
- **Stars**: 7
- **Last updated**: 2026-07-14

Source: https://www.modelscope.cn/models/Abiray/OvisOCR2-GGUF

---

# OvisOCR2 - GGUF Quantizations

<p align="center">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/658a8a837959448ef5500ce5/vRCIu5QD8VuIJolkC_ZHQ.png" alt="Ovis" width="30%" />
</p>

This repository contains GGUF format quantizations of **OvisOCR2**, a compact 0.8B end-to-end model for page-level document parsing. The original model was developed by **ATH-MaaS** by post-training `Qwen3.5-0.8B` to parse full document pages directly into clean Markdown (including LaTeX formulas, HTML tables, and layout components).

OvisOCR2 establishes a new state-of-the-art for compact document understanding, scoring **96.58** on OmniDocBench v1.6 and outperforming traditional, multi-stage layout analysis pipelines.

---

## Available Files

### Main Text Models

| File Name | Precision / Quantization | File Size | Description |
| :--- | :--- | :--- | :--- |
| `OvisOCR2-F16.gguf` | 16-bit Float | 1.52 GB | Baseline unquantized model |
| `OvisOCR2-BF16.gguf` | 16-bit Brain Float | 1.52 GB | Native weight precision |
| `OvisOCR2-Q8_0.gguf` | 8-bit | 812 MB | Near-identical precision to F16 |
| `OvisOCR2-Q6_K.gguf` | 6-bit | 630 MB | Excellent balance of size and accuracy |
| `OvisOCR2-Q5_K_M.gguf` | 5-bit (Medium) | 578 MB | Recommended for low-resource deployment |
| `OvisOCR2-Q5_K_S.gguf` | 5-bit (Small) | 564 MB | Highly optimized 5-bit layout |
| `OvisOCR2-Q4_K_M.gguf` | 4-bit (Medium) | 529 MB | Standard 4-bit quantization |
| `OvisOCR2-Q4_K_S.gguf` | 4-bit (Small) | 505 MB | Lightweight 4-bit footprint |
| `OvisOCR2-Q3_K_M.gguf` | 3-bit (Medium) | 466 MB | Maximum compression ratio |

### Multimodal Projectors (`mmproj`)
*Note: Because OvisOCR2 is a vision-language model, you **must** download one of these image processing units alongside your choice of the text models listed above.*

* `mmproj-F32.gguf` (402 MB) - Unquantized full precision projector.
* `mmproj-F16.gguf` (205 MB) - Recommended standard performance/size option.
* `mmproj-BF16.gguf` (207 MB) - Target alternative precision layout.

---

## Inference Guide (`llama.cpp`)

To run multimodal OCR tasks using these GGUF files, you need to use the `llama-minicpmv-cli` or `llama-llava-cli` tool (depending on your build version of `llama.cpp`) to handle simultaneous image and text tokens. 

### Basic Command Line Example

```bash
# Run parsing via llama.cpp cli tools
./llama-minicpmv-cli \
  -m OvisOCR2-Q5_K_M.gguf \
  --mmproj mmproj-F16.gguf \
  --image /path/to/your/document_page.jpg \
  -p "<|im_start|>user\nExtract all readable content from the image in natural human reading order and output the result as a single Markdown document. Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Preserve the original text without translation.<|im_end|>\n<|im_start|>assistant\n" \
  -n 4096 \
  --temp 0.0