Hugging Peers

PaddleOCR-VL-Receipt

☆ Save
by megemini Apache License 2.0
359 downloads 1 likes on ModelScope updated

Model card

PaddleOCR-VL-Receipt

This model is based on WildReceipt data and fine-tuned from PaddleOCR-VL to modify its output for different prompts, enabling it to extract information from receipts and documents.

Model Inference

Use the fine-tuned model for inference by modifying the prompt parameter to control the model's output with PaddleOCR-VL-REC.

Input Image Complete Information Extraction Specific Information Extraction
Receipt Full Recognition Partial Recognition

Model Description

Basic Model Information

PaddleOCR-VL is a Vision-Language Model (VLM) specifically designed for document understanding. It can complete various document understanding tasks through prompts, including text recognition (OCR), table recognition, formula recognition, and chart recognition. This project extends its application scenarios to enable structured information extraction tasks.

Model Architecture and Training

  • Base Model: PaddleOCR-VL-0.9B
  • Fine-tuning Method: Supervised Fine-Tuning (SFT) using the ERNIE framework
  • Core Innovation: Extends model functionality through custom prompt templates, enabling the model to output structured JSON format data based on different field requirements

Applicable Scenarios

  • Complete document information extraction
  • Partial document information extraction

This model is optimized specifically for WildReceipt. The generalization capability for other types of documents requires further verification.

Expected Model Usage and Applicable Scope

Core Features

The fine-tuned PaddleOCR-VL model supports two main usage methods:

  1. Complete Information Extraction: Input OCR:{} prompt, the model outputs all structured information (JSON format) from the document
  2. Specific Field Extraction: By specifying field names, the model outputs values for specific fields (JSON format)

Applicable Scope

  • Support for string fields: OCR:{"field_name":""}
  • Support for dictionary/object fields: OCR:{"field_name":{}}
  • Support for list fields: OCR:{"field_name":[]}

Fine-tuning Usage

For fine-tuning the model based on your own data, please refer to:

Model Limitations and Potential Biases

Known Limitations

  1. Data Volume Limitations: The current fine-tuning uses a relatively small amount of data, and there is still room for improvement in model performance
  2. Document Type Limitations: Primarily optimized for receipt documents; generalization capability for other document types requires verification
  3. Information Completeness: The model may perform poorly on complex tables or multi-column layout documents

Potential Biases

  1. Recognition Accuracy: The model's recognition results depend on the input image quality
  2. Field Format: For non-standard format field values, the model may fail to parse correctly
  3. Multi-page Documents: Currently primarily optimized for single-page documents

Usage Recommendations

  • Ensure input images are clear and complete
  • Prompt templates should be consistent with training data format
  • For new document types, targeted fine-tuning is recommended
  • Comprehensive validation and testing are recommended before production use

Related Resources

References

Related Tools

  • ERNIE Framework: https://github.com/PaddlePaddle/ERNIE
  • PaddleOCR-VL Model: https://github.com/PaddlePaddle/PaddleOCR