feat(respond-pr-feedback): add emoji reactions to bot comments for quality tracking #40
Description
Context
We have a PR quality tracking script (amelia/scripts/pr_quality.py) that measures CodeRabbit comment density over time. To distinguish valuable comments from noise, we use GitHub emoji reactions as a triage signal:
- 👍 (+1) = valuable — the comment caught a real issue or led to a fix
- 👎 (-1) = noise — false positive, nitpick, or not actionable
These reactions are already queryable via the GitHub API (returned inline on every comment object), making them ideal for automated analysis.
Problem
/beagle:respond-pr-feedback already evaluates each bot comment and categorizes the response type (fixed, won't fix, incorrect, lacking context), but it doesn't leave a reaction. This means we have to manually go back and react to every comment for quality tracking.
Proposed Change
After posting each reply in step 4, also add an emoji reaction to the original bot comment based on the response type:
| Response Type | Reaction | Rationale |
|---|---|---|
| Valid and fixed | 👍 (+1) | Comment was valuable — it led to a fix |
| Valid but won't fix | 👍 (+1) | Comment identified a real tradeoff, still valuable |
| Incorrect feedback | 👎 (-1) | False positive, not actionable |
| Feedback lacking context | 👎 (-1) | Comment didn't account for design decisions |
API call
gh api --method POST \
repos/{owner}/{repo}/pulls/comments/{comment_id}/reactions \
-f content="+1" # or "-1"Behavior
- Reactions should be added automatically (no extra flag needed) since the command already knows the response type
- If a reaction already exists from the same user, the API is a no-op (idempotent)
- Add the reaction to the summary table output (new column showing the reaction added)
Acceptance Criteria
- Each replied-to bot comment gets a +1 or -1 reaction based on response type mapping above
- Summary table includes a column showing which reaction was added
- Existing behavior (text replies, thread resolution) is unchanged