Conversation
vLLM-Omni runs 19 roadmap/tracking issues from 15 authors. One is current, by hand: #7074 took 88 body edits in six days. #6494 tracks 129 numbers, was last edited 2026-09-05, and five of its PRs have merged since. Measured over 300 merged PRs (2026-08-17..2026-09-11, ~12/day), deriving the roadmap from PR traffic fails: 2 of 25 curated PRs cite the roadmap they belong to (8% recall), title keywords give 15% precision, and a human selected 8 of 300. The taxonomy is judgment and is not recoverable from the artifacts, because it was never written into them. Status is. So the agent maintains and extracts, never invents: enrollment by an author-placed marker, bootstrap gated on verbatim quotes of the source body, status refreshed every poll cycle with no model call, and attachment left as a proposal a maintainer accepts in the management bot service. The recurring lane writes a comment the bot solely authors, not the issue body. GitHub offers no conditional update for an issue body, so a read-check-write against a body edited 24 times a day is a real TOCTOU race: it can destroy a maintainer's edit, and a splice from a cached body can restore an enrollment marker someone just deleted, defeating revocation. Only bootstrap and attachment promotion touch the body, both human-triggered and spliced into a fresh read; the residual race is recorded in Honest limits rather than claimed away. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MmTLpGtoZHBz81xj1dCo2r
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fcd65c8dda
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 4. **The HTTP surface takes a proposal id and a verdict, never roadmap | ||
| content.** It can only apply proposals the agent already computed and | ||
| quote-validated, and every apply re-runs the quote gate before writing. A | ||
| session compromise buys an attacker "accept a proposal that was already on | ||
| offer" — not arbitrary writes into upstream issues with the bot's token. |
There was a problem hiding this comment.
Define a quote gate that can admit untracked PRs
When a lane-two candidate is absent from the current roadmap body—the normal reason to propose attaching it—§3's quote gate rejects the attachment because its supporting span is not present in that body. Re-running that same gate during apply therefore prevents useful proposals from ever being accepted. Define a proposal-specific gate whose allowed evidence includes the candidate PR while independently validating the destination feature against the roadmap.
Useful? React with 👍 / 👎.
| What the same signals are good for is **prefiltering**: the title keyword carries | ||
| 88% recall (22 of the 25 curated PRs) while cutting the candidate set to 40 of | ||
| 300. Useless as a classifier, exactly right as a gate in front of one. |
There was a problem hiding this comment.
Recompute recall from the in-window cohort
The table reports that the title keyword finds 6 of the 8 curated PRs in the measured window, which is 75% recall, not 88%. The 22/25 figure uses the broader curated set while the claimed reduction to 40/300 uses the in-window set, so combining them does not establish an 88%-recall prefilter for the daily workload; use one matched cohort or provide the corresponding candidate count for all 25 PRs.
Useful? React with 👍 / 👎.
Enriches #152 ("[RFC] Auto Roadmap Generation") into a full RFC. Design only — nothing implemented.
What the measurements changed about the ask
#152 asks the agent to "auto dive the relationship of feature/rfcs/prs". Measured against
vllm-project/vllm-omniover 2026-08-17 → 2026-09-11 (300 merged PRs, ~12/day), that reading does not survive:world modeldiffusionA human selected 8 of 300 merged PRs as roadmap-relevant. The taxonomy is judgment and was never written into the artifacts, so it cannot be recovered from them. What is derivable, completely and without a model, is status — and that is where the entire cost sits:
So the agent maintains and extracts; it never invents.
Design
partialby judgment although its PR merged.pushpermission; the HTTP surface takes a proposal id and a verdict, never roadmap content.RFC-knowledge-intake.mdwith one named deviation: bootstrap runs as an IMC task kind (L2, confirm-gated) rather than SDK functions the bot orchestrates, because it is one-shot, judgment-heavy and writes into someone else's RFC.Rollout is ordered so increment one carries 100% of the measured cost while needing no model, no bootstrap and no authentication — the increment that puts OAuth on a public host must not gate it.
Review notes
The first draft specified marker-scoped writes into the issue body guarded by a hash of the human-owned regions. The automated review gate rejected it correctly: GitHub exposes no
If-MatchforPATCH /issues/:n, so that guarantee cannot be delivered. §2 now moves the unattended lane off the body entirely, and the residual race on the two supervised body writes is recorded in Honest limits rather than claimed away.Doc gates green:
check_doc_links,check_doc_citations,check_spec_freshness.Closes #152 only if the design is accepted as written; otherwise the issue stays open for the parts that change.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MmTLpGtoZHBz81xj1dCo2r