Problem
Today benchmark/spatial.py computes IoU + centroid error against a single ground-truth oracle: raw Tesseract OCR at 150 DPI. Tesseract is a bad oracle on complex forms — on IRS 1040 it extracts "Treasiny—Intemal" for "Treasury—Internal", which tanks spdf's measured F1 to 15.5% even though its output is visually correct ("Department of the Treasury—Internal").
We need a second, independent oracle so the benchmark reports where two engines agree, not where tesseract's hallucinations live.
Proposal
Add Poppler pdftotext -bbox-layout as a second oracle. It emits per-word bounding boxes directly from the PDF content stream (no rasterisation), which is exactly the ground truth we need for digitally-born PDFs.
Design
benchmark/spatial.py:
- Keep the existing tesseract-TSV pipeline for raster-only fixtures (
example-1.jpg, test-ocr.pdf).
- For PDF fixtures, run
pdftotext -bbox-layout <pdf> - and parse the <word xMin yMin xMax yMax> XHTML output.
- Report three columns per engine: F1 vs tesseract, F1 vs pdftotext, F1 vs agreement (words where the two oracles also agree).
- CLI flag
--oracle {tesseract,pdftotext,both} (default both on PDFs).
- Update
README.md benchmark table: headline number becomes F1 against oracle agreement, which is a more defensible claim.
Acceptance
Problem
Today
benchmark/spatial.pycomputes IoU + centroid error against a single ground-truth oracle: raw Tesseract OCR at 150 DPI. Tesseract is a bad oracle on complex forms — on IRS 1040 it extracts"Treasiny—Intemal"for"Treasury—Internal", which tanks spdf's measured F1 to 15.5% even though its output is visually correct ("Department of the Treasury—Internal").We need a second, independent oracle so the benchmark reports where two engines agree, not where tesseract's hallucinations live.
Proposal
Add Poppler
pdftotext -bbox-layoutas a second oracle. It emits per-word bounding boxes directly from the PDF content stream (no rasterisation), which is exactly the ground truth we need for digitally-born PDFs.Design
benchmark/spatial.py:example-1.jpg,test-ocr.pdf).pdftotext -bbox-layout <pdf> -and parse the<word xMin yMin xMax yMax>XHTML output.--oracle {tesseract,pdftotext,both}(defaultbothon PDFs).README.mdbenchmark table: headline number becomes F1 against oracle agreement, which is a more defensible claim.Acceptance
benchmark/spatial.pysupports both oracles.benchmark/results/spatial.{md,json}includes both columns.