FireRed-OCR

### TL;DR

FireRed-OCR is a specialized framework developed by FireRedTeam to transform general Large Vision-Language Models (LVLMs) into high-performance, pixel-precise structural document parsing experts. It addresses the issue of "Structural Hallucination" in general VLMs by shifting the paradigm from "impressionist" text generation to "structural engineering," achieving state-of-the-art results on benchmarks like OmniDocBench v1.5.

Key Insights & Metrics

Key Features

• SOTA Performance: Achieves 92.94% overall score on OmniDocBench v1.5, outperforming models like DeepSeek-OCR 2 and OCRVerse.
• Structural Integrity: Utilizes Format-Constrained GRPO (Group Relative Policy Optimization) to enforce strict syntactic validity, eliminating common errors like unclosed tables or invalid LaTeX formulas.
• "Geometry + Semantics" Data Factory: Employs a novel data engine that uses geometric feature clustering and multi-dimensional tagging to synthesize balanced datasets, effectively handling long-tail layouts.
• Progressive Training Pipeline: Consists of multi-task pre-alignment, specialized supervised fine-tuning, and format-constrained GRPO for self-correction via reinforcement learning.
• In-the-Wild Robustness: Demonstrates superior resilience on complex, non-standard layouts compared to traditional pipeline systems like PaddleOCR.

Discussion