pedro castanheiraPT
← All work

Production · education company

Document triage before OCR

Built a two-stage visual validator with ConvNeXt-Tiny that discards illegible or wrong-type documents before extraction with Document AI.

Role
Model training, serving and integration
Status
In production
Stack
  • Python
  • FastAPI
  • PyTorch
  • ConvNeXt-Tiny
  • OpenCV
  • NestJS
  • GCP Document AI
  • GCS
document(PDF / image)Document AI billsper processed pagepositives per type+ negativesquality gate(blank, blur)ConvNeXt-Tinywhich type?binary validatorfor the typetrainingTTA only when unsureinvalid →returned with reasonvalidDocument AI(OCR)validatedfields~45% lower OCR costclassifier: 98.3% on the test split

Problem

OCR and extraction with Document AI are billed per processed page. Without triage, blank, blurry or wrong-type documents pay for OCR and only fail later, at field validation.

Available data

Images and PDFs of personal and academic documents of several types. Training uses labeled positives per type and out-of-scope negatives, with manifests and datasets isolated per type.

Architecture decisions

Classify before reading

The documents API calls the validator before OCR. When a document fails, it returns with a reason and Document AI is never called.

Two stages: type and validity

A ConvNeXt-Tiny classifier predicts the type — skipped when the type is already known — and a specialized binary validator decides whether that document is valid.

Spend inference only when needed

A quality gate drops blank, blurry or low-resolution pages before the model. Four-view TTA only kicks in for intermediate confidence, with per-model thresholds.

Serving built for cold starts

Models are cached and warmed at boot, with concurrency capped by a semaphore. On the API side, a 90 s timeout and a retry on cold start.

Implementation

  • Training pipeline split into corpus, manifests, sampling, training and evaluation, with versioned metrics per model.
  • A registry that selects version and threshold per document type.
  • Multi-page documents: up to 8 pages, valid if any page passes, with early accept above 0.97.
  • Integration in the NestJS API: validation and OCR timed separately; OCR receives the GCS URI.

Result

  • ~45%lower OCR cost on Document AI (GCP)
  • 98.3%type-classifier accuracy on the test split (1,500 images)

Invalid documents are discarded before OCR: roughly 45% lower Document AI cost.

Limits and lessons

  • Accuracy comes from a versioned test split, not a guarantee on real traffic.
  • The savings percentage is approximate.
  • Detailed document types and volumes stay out for confidentiality.
Next caseCast Review →