fix(detect): Filter bright rectangles from TOC leak bbox matching - #218
fix(detect): Filter bright rectangles from TOC leak bbox matching#218mlissner wants to merge 2 commits into
Conversation
Colored highlight annotations (pink, yellow, gray) pass get_good_rectangles but aren't redactions. Their bboxes were polluting the TOC leak matcher, causing false positives on PDFs with highlighted text and TOC entries. Now only dark rectangles (checked via _is_dark_color on fill color) contribute bboxes for TOC matching. This preserves detection of applied redactions (text removed, black bar left) which have dark fill colors even though no text survives the pipeline. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ives Academic paper with colored highlight annotations and TOC entries. Verifies that highlighted text doesn't trigger false TOC leak detections. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
LGTM — targeted fix for false TOC leak detections from colored highlight rectangles, with clear test coverage.
Extended reasoning...
Overview
This PR fixes a false-positive bug in TOC bookmark leak detection: get_bad_redactions was adding bboxes of all opaque rectangles (including colored highlights used as markup in academic papers) to the redaction_bboxes list used by get_toc_leaks, causing bookmarks near highlighted (non-redacted) text to be wrongly flagged as leaking redacted content. The fix filters these bboxes to only dark-colored rectangles via the existing _is_dark_color helper, before they are added to redaction_bboxes. It deliberately leaves the unfiltered good_rectangles list untouched for the actual text-under-rectangle detection path, and preserves TOC-matching for genuinely applied redactions (black bar, no text) by checking fill color directly rather than gating on the pixmap filter.
Security risks
None. This is pure PDF-content heuristic logic with no user input handling, network calls, or auth/permissions surface.
Level of scrutiny
Low-to-moderate. The change is a small, localized addition (an if filtering which bboxes get appended) inside a well-tested detection pipeline. The two candidate concerns raised by the automated bug hunt (bbox/redaction desync, dropping legit applied-redaction bars) were both examined and are addressed by the PR's design rationale (dark rectangles, including bar-only applied redactions with no text, are still included) — I independently confirmed rect.fill is already normalized RGB elsewhere in this file (e.g. get_dark_highlight_annotations), so the int(c * 255) conversion is consistent with existing usage.
Other factors
New test test_highlight_annotations_toc_no_results uses a real academic-paper PDF per the project's testing rules, and the full test suite (42 passed, 1 expected-failure) passes locally, including all TOC-leak and highlight-related tests. CHANGES.md is updated per project convention. No outstanding review comments in the PR timeline.
Summary
Fix false TOC leak detections on PDFs with colored highlight annotations (pink, yellow, gray highlights over text in academic papers, etc.).
Root cause:
get_good_rectangles()returns all opaque filled rectangles, including colored highlights. Their bboxes were being added toredaction_bboxesbefore any color filtering, polluting the TOC leak matcher.Fix: Filter rectangle bboxes by
_is_dark_coloron fill color before adding toredaction_bboxes. This excludes bright highlights while keeping dark rectangles (including applied redactions with no text underneath).Note: The initial suggestion was to move bbox collection after the pixmap filter, but that would have broken TOC leak detection for properly applied redactions (text removed, black bar left) since those have no text and don't survive the pixmap pipeline.
Test plan
test_highlight_annotations_toc_no_resultswith real academic paper PDFtest_toc_leak(7-page synthetic) still passes — applied redaction on page 1 still detectedtest_toc_leak_real(JOSH MERRITT) still passes🤖 Generated with Claude Code