Skip to content

Add missing_context to QualityEvaluation for incomplete data tracking#693

Open
iamnbutler wants to merge 1 commit intomainfrom
tasks/gh-iamnbutler-tasks-issue-509--07796e9c
Open

Add missing_context to QualityEvaluation for incomplete data tracking#693
iamnbutler wants to merge 1 commit intomainfrom
tasks/gh-iamnbutler-tasks-issue-509--07796e9c

Conversation

@iamnbutler
Copy link
Copy Markdown
Owner

Summary

  • Adds missing_context: Vec<String> field to QualityEvaluation so humans can distinguish evaluations made with full context from those made with partial data
  • Tracks "pr_diff" and "linked_issue" as missing context when GitHub fetches fail in ClaudeOrchestrator::evaluate()
  • Surfaces missing context in PR evaluation comments (blockquote note)
  • Uses #[serde(default)] for backward-compatible deserialization

Test plan

  • All 53 orchestrator tests pass
  • Full cargo check succeeds
  • Verify PR comments show missing context note when diff/issue fetch fails

Closes #509

🤖 Generated with Claude Code

…acking

When fetching PR diffs or linked issues fails, the orchestrator now records
what was missing in a `missing_context: Vec<String>` field on QualityEvaluation.
This lets humans distinguish "evaluated with full context" from "evaluated blind."
The PR comment also surfaces this info when present.

Closes #509

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orchestrator evaluation doesn't flag when data was incomplete

1 participant