Skip to content

feat: Add DocumentProfile for document-type-specific extraction - #4

Merged
rookslog merged 3 commits into
mainfrom
feature/document-profiles
Dec 26, 2025
Merged

feat: Add DocumentProfile for document-type-specific extraction#4
rookslog merged 3 commits into
mainfrom
feature/document-profiles

Conversation

@rookslog

Copy link
Copy Markdown
Owner

Summary

  • Implements document profiles system for configuring CascadingExtractor based on document type
  • Adds 5 standard profiles: BOOK, ARTICLE, ESSAY, REPORT, DEFAULT (generic)
  • Adds auto-detection via get_profile() using existing estimate_document_type()
  • CascadingExtractor now accepts optional profile parameter
  • Backward compatible: existing code without profiles continues to work

Changes

  • Add DocumentProfile frozen dataclass with extraction settings
  • Add for_profile() and for_document() classmethods to CascadingExtractor
  • Add profile_used field to StructureResult for tracking
  • Update module exports in extractors/__init__.py
  • Update ROADMAP.md to mark milestones 1.4/1.5 complete

Test plan

  • 19 unit tests for DocumentProfile dataclass and get_profile()
  • 9 tests for CascadingExtractor profile integration
  • 3 integration tests with real philosophy PDFs
  • All 288 tests pass

The plan command now requires defining specific test cases (Given/When/Then)
before breaking down tasks. This:
- Forces concrete thinking about behavior during planning
- Creates clear 'done' criteria
- Makes plan → implement handoff smoother
- Validates plans are specific enough
Implements document profiles system for configuring CascadingExtractor
based on document type (book, article, essay, report, generic).

Changes:
- Add DocumentProfile frozen dataclass with extraction settings
- Add 5 standard profiles: BOOK, ARTICLE, ESSAY, REPORT, DEFAULT
- Add get_profile() for auto-detection using estimate_document_type()
- Modify CascadingExtractor to accept profile parameter
- Add for_profile() and for_document() classmethods
- Add profile_used field to StructureResult
- Update module exports in extractors/__init__.py

This completes ROADMAP milestones 1.4 (Structure Extraction) and
1.5 (Document Profiles).

Tests: 288 passing (28 new profile tests + 3 integration tests)
@rookslog

Copy link
Copy Markdown
Owner Author

Code Review

Status: ✅ Approved

Strengths

  • Clean Design: DocumentProfile as a frozen dataclass is the right choice - immutable, type-safe, simple to use
  • Backward Compatible: Existing code without profiles continues to work unchanged
  • Good Test Coverage: 31 new tests covering all standard profiles, auto-detection, immutability, and real PDF integration
  • Follows Project Patterns: Uses existing estimate_document_type() instead of reinventing
  • Proper Exports: All new symbols exported in extractors/__init__.py with __all__

Minor Observations (Non-blocking)

  1. pytest.mark.slow warning: Two tests use unregistered mark at tests/integration/test_convert.py:283, 301. Can fix later by adding to pyproject.toml:

    [tool.pytest.ini_options]
    markers = ["slow: marks tests as slow"]
  2. min_content validator: ESSAY_PROFILE references it but MinimumContentValidator may not be fully implemented yet. Code handles this gracefully (skips unknown validators).

Verification

  • ✅ All 288 tests pass
  • ✅ Lint passes
  • ✅ CI passes (Python 3.11/3.12)

LGTM - ready to merge.

Resolves merge conflict in .claude/commands/plan.md by accepting
the more comprehensive main branch version.

Brings in workflow improvements from PRs #5, #6, #7:
- Autonomous development workflow with review gates
- Enhanced commands (auto, debug, diagnose, review-pr, etc.)
- Session logging and signal capture
- Parallelization guide
@rookslog
rookslog merged commit fae85af into main Dec 26, 2025
2 checks passed
@rookslog
rookslog deleted the feature/document-profiles branch December 26, 2025 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant