Skip to content

Chore/evaluator check cleanups#62

Open
lszhuhaichao wants to merge 6 commits into
mainfrom
chore/evaluator-check-cleanups
Open

Chore/evaluator check cleanups#62
lszhuhaichao wants to merge 6 commits into
mainfrom
chore/evaluator-check-cleanups

Conversation

@lszhuhaichao

Copy link
Copy Markdown
Collaborator

Summary

This change continues the evaluator refactor by consolidating repeated path, baseline, substring, and regex-checking logic into shared helpers under evaluator/shared/check_utils.py, then applying those helpers across multiple evaluator checks.

What changed

  • Extended shared evaluator utilities with reusable helpers for:

    • multi-root file scanning
    • regex pattern matching/counting
    • substring presence/counting
    • relative-path allowlist / top-level path validation
    • baseline path classification:
      • missing in submission
      • missing in baseline
      • created in submission
      • deleted from submission
      • modified relative to baseline
    • shared brace-matching extraction used by both class-body and function-body parsing
  • Refactored multiple evaluator checks to use shared helpers instead of local ad hoc logic:

    • regex scanning loops
    • substring presence checks
    • file existence checks
    • path-boundary checks
    • baseline-protected file checks
    • Path.cwd() / CASE_REL style root resolution
    • local wrapper utilities like collect_code_files(...)

Key shared helpers introduced / expanded

  • scan_files(..., suffixes=...)
  • find_matching_patterns(...)
  • find_missing_patterns(...)
  • has_any_pattern(...)
  • count_matching_patterns(...)
  • has_any_substring(...)
  • has_all_substrings(...)
  • count_matching_substrings(...)

@lszhuhaichao lszhuhaichao requested a review from qzhang-ucr June 13, 2026 00:54
@lszhuhaichao lszhuhaichao self-assigned this Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant