The Labeling Bottleneck

Training a Custom Document Extractor (CDE) in Google Cloud Document AI Workbench is the ultimate way to achieve high-accuracy data extraction (often 95%+ confidence). However, the bottleneck is always data labeling. Manually drawing boxes and assigning fields on 100 to 200 documents in the cloud console can easily take 10 to 20 hours of tedious work.

Fortunately, you don't have to start from scratch. A semi-automated pre-labeling workflow can reduce manual labeling time by up to 70%.

The Semi-Automated Pre-Labeling Workflow

Instead of importing raw PDFs and labeling everything manually, follow this three-step workflow:

1. Pre-Label with the Default Model

Write a script to process your training documents through the default Document AI parser. Save the resulting JSON responses. These contain the model's best guess of all text entities and bounding boxes.

2. Import JSON Annotations

Upload both the PDFs and their corresponding JSON prediction files into your Document AI Workbench dataset. Because Document AI Workbench understands its own JSON schema, it will auto-populate the bounding boxes and fields.

3. Review and Correct in Workbench UI

Instead of drawing boxes from scratch, your job is now to simply review the pre-populated labels, correct the occasional mistake, and save. Correcting an existing label is much faster than creating one.

Automating the Loop

Once your dataset is labeled, you can automate the rest of the MLOps loop (validation, training, and deployment) using the Document AI Python SDK:

from google.cloud import documentai_v1beta3 as documentai

def train_custom_processor(project_id, location, processor_id, dataset_uri):
    client = documentai.DocumentProcessorServiceClient()
    parent = client.processor_path(project_id, location, processor_id)
    
    # Define training request details
    # (Triggers training, monitors logs, and deploys if accuracy hits target)
    print("Training triggered...")

By leveraging the default model for pre-labeling and automating the training steps via scripts, you can scale your custom OCR solutions in a fraction of the time.