feat: make GitHub Cohort collection explicit - #84
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Issue #83 looked like an alphabetical five-topic cap, but there was no limit in boast. The GitHub Provider already attempted every topic declared by the repository.
The real limit was GitHub Search capacity. One Cohort rank needs two Search requests:
GitHub normally allows 10 Search requests per minute without authentication and 30 with
GITHUB_TOKEN. A repository with 20 topics can therefore require 40 requests. In the reported case,UPHL-BioNGS/Cecret, boast produced 5 ranks anonymously or 15 with a token. Failed searches were omitted, the command exited successfully, and nothing in the Snapshot or Report said that later Cohorts were missing. The output consequently looked complete when it was not.The new default behaviour
The default remains best-effort because Cohort ranks are secondary to the repository's core GitHub metrics. Boast still attempts every declared topic without waiting. If Search fails part-way through, it now:
ValuesOutcome, not a transient failure of the whole GitHub Provider;Provider Notes are part of Snapshot schema v2, so they survive offline rendering in terminal and Markdown Reports. Schema v1 Snapshots without the new field remain readable. The notes are deliberately excluded from grant-writing prose and Snapshot diffs because they describe collection, not a Metric's meaning.
For the 20-topic example, the default anonymous result is therefore still five ranks, but it explicitly says
5 of 20and names the other fifteen. SupplyingGITHUB_TOKENusually raises that to fifteen ranks, but the note and documentation do not claim that a token guarantees completeness.New CLI controls
Select exact Cohorts
Repeat
--topicto rank exactly the supplied topics, in command-line order:Explicit topics do not need to be declared by the repository. This preserves the previous single-topic override behaviour while making it repeatable. Use this mode when only specific Cohorts belong in the Report or when the selected set must fit in the available Search allowance.
Prioritise declared Cohorts
Repeat
--priority-topicto spend the available Search requests on important declared topics first, then attempt every remaining declared topic:A priority topic must actually be declared by the repository. An undeclared value exits 2 and directs the user to
--topicif an undeclared Cohort was intentional.--topicand--priority-topicare mutually exclusive.This is separate from exact selection because the two express different intent: exact selection narrows the requested set, whereas priority selection keeps the request for every declared Cohort but controls which ones are attempted before capacity runs out.
Wait across confirmed Search resets
Waiting is opt-in:
Accepted duration units include
s,mandh; zero and malformed durations are usage errors. The optional value requires=so it cannot consume a positional repository identifier.Boast waits only when GitHub identifies the exhausted resource as Search and supplies a reset or retry time. Each planned delay and cumulative progress is shown on stderr. It continues until all requested ranks are collected or the cumulative limit expires; expiry produces the same successful partial result and durable Provider Note. GitHub Search bypasses the generic retry layer, so unrelated 403s, server errors, malformed responses and transport errors do not cause hidden sleeps.
Manifest behaviour
Generated Manifests persist selection, but not waiting:
topicsandpriority_topicsare mutually exclusive. Legacytopic = "..."remains readable, and CLI topic options override a Manifest's selection.--wait-for-cohort-ranksstays runtime-only so a committed Manifest does not unexpectedly make CI sleep.Why these controls instead of
--max-topicsAn arbitrary numeric limit would preserve the misleading idea that boast intentionally selects an alphabetical prefix. It would not tell boast which Cohorts matter, and changing repository topic order could change the selected set. Exact selection and priority ordering make the user's intent explicit while preserving a safe, honest best-effort default. Bounded opt-in waiting covers the separate case where completeness matters more than run time.
Validation
Offline tests cover anonymous 10-request and authenticated 30-request exhaustion, preservation of successful metrics, stopped searches, exact and priority ordering, invalid priorities, argument conflicts, confirmed reset continuation, cumulative timeout, unrelated failures, Manifest compatibility, schema v1 reads and offline Provider Note rendering.
cargo test --allcargo clippy --all-targets -- -D warningscargo check --all-targetsmdbook build docs/sitegit diff --checkCloses #83