Skip to content

PR state synthesis can resurface stale review blockers after follow-up commits #42

Description

@keyoku-bot

Summary

Keyoku can surface outdated PR review state even after the PR branch has moved forward and the original blocker has already been fixed. This produces misleading summaries like "still blocked on X" when later commits on the branch have already addressed X.

Concrete example

On keyoku PR #35, an earlier valid blocker continued to surface after later commits had already addressed it and CI had re-run green. This suggests older PR conclusions are not being invalidated cleanly against the latest head SHA and branch state.

Actual behavior

Keyoku reports stale blocker context from prior review state even when:

  • the PR branch has newer commits
  • the flagged issue is already fixed in branch
  • CI has re-run successfully on the updated branch

Expected behavior

Once a PR receives new commits that address earlier review concerns, PR tracking/synthesis should:

  • treat earlier blocker summaries as potentially stale
  • re-evaluate against the latest head SHA / latest diff
  • avoid presenting prior review concerns as current unless they still reproduce on the updated branch

Impact

This causes:

  • misleading maintainer summaries
  • duplicate or outdated blocker reports
  • wasted review attention
  • lower trust in PR-state synthesis

Suspected cause

Likely one or more of:

  • stale memory being recalled without commit-aware invalidation
  • PR summaries being generated from historical review comments without checking the latest head SHA
  • insufficient preference for fresh hook events / latest branch state over older stored conclusions
  • missing "superseded by newer commit" logic in PR status synthesis

Suggested fix direction

  • attach PR review findings to the specific head SHA they were observed on
  • invalidate or down-rank older review conclusions after new commits land
  • when summarizing PR state, prefer:
    1. latest head SHA
    2. latest CI on that SHA
    3. latest diff state
    4. only then historical review comments
  • if an old review is still open but the code changed since that review, say something like:
    • "older changes-requested review exists, but the flagged issues may already be addressed by newer commits"

Repro shape

  1. Open a PR
  2. Record a valid blocker
  3. Push a follow-up commit fixing the blocker
  4. Re-run CI successfully
  5. Ask Keyoku for current PR status
  6. Observe that the old blocker is still surfaced as if unresolved

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions