Skip to content

feat: make GitHub Cohort collection explicit - #84

Merged
mbhall88 merged 1 commit into
mainfrom
codex/github-cohort-collection
Aug 10, 2026
Merged

feat: make GitHub Cohort collection explicit#84
mbhall88 merged 1 commit into
mainfrom
codex/github-cohort-collection

Conversation

@mbhall88

@mbhall88 mbhall88 commented Aug 10, 2026

Copy link
Copy Markdown
Owner

The problem

Issue #83 looked like an alphabetical five-topic cap, but there was no limit in boast. The GitHub Provider already attempted every topic declared by the repository.

The real limit was GitHub Search capacity. One Cohort rank needs two Search requests:

  1. count every repository carrying the topic, to get the Cohort size;
  2. count repositories carrying the topic with more stars, to calculate this repository's rank.

GitHub normally allows 10 Search requests per minute without authentication and 30 with GITHUB_TOKEN. A repository with 20 topics can therefore require 40 requests. In the reported case, UPHL-BioNGS/Cecret, boast produced 5 ranks anonymously or 15 with a token. Failed searches were omitted, the command exited successfully, and nothing in the Snapshot or Report said that later Cohorts were missing. The output consequently looked complete when it was not.

The new default behaviour

The default remains best-effort because Cohort ranks are secondary to the repository's core GitHub metrics. Boast still attempts every declared topic without waiting. If Search fails part-way through, it now:

  • retains stars, forks, watchers, contributors, release downloads and every completed Cohort rank;
  • stops issuing searches for later topics rather than continuing to spend requests after the first failure;
  • exits 0 because this is still a successful Values Outcome, not a transient failure of the whole GitHub Provider;
  • prints a short warning to stderr; and
  • stores a Provider Note containing the collected/requested counts, every skipped topic, the reason collection stopped and an appropriate remedy.

Provider Notes are part of Snapshot schema v2, so they survive offline rendering in terminal and Markdown Reports. Schema v1 Snapshots without the new field remain readable. The notes are deliberately excluded from grant-writing prose and Snapshot diffs because they describe collection, not a Metric's meaning.

For the 20-topic example, the default anonymous result is therefore still five ranks, but it explicitly says 5 of 20 and names the other fifteen. Supplying GITHUB_TOKEN usually raises that to fifteen ranks, but the note and documentation do not claim that a token guarantees completeness.

New CLI controls

Select exact Cohorts

Repeat --topic to rank exactly the supplied topics, in command-line order:

boast about --repo owner/repo \
  --topic rna-seq \
  --topic nextflow

Explicit topics do not need to be declared by the repository. This preserves the previous single-topic override behaviour while making it repeatable. Use this mode when only specific Cohorts belong in the Report or when the selected set must fit in the available Search allowance.

Prioritise declared Cohorts

Repeat --priority-topic to spend the available Search requests on important declared topics first, then attempt every remaining declared topic:

boast about --repo owner/repo \
  --priority-topic bioinformatics \
  --priority-topic nextflow

A priority topic must actually be declared by the repository. An undeclared value exits 2 and directs the user to --topic if an undeclared Cohort was intentional. --topic and --priority-topic are mutually exclusive.

This is separate from exact selection because the two express different intent: exact selection narrows the requested set, whereas priority selection keeps the request for every declared Cohort but controls which ones are attempted before capacity runs out.

Wait across confirmed Search resets

Waiting is opt-in:

# Up to five cumulative minutes
boast about --repo owner/repo --wait-for-cohort-ranks

# A custom positive cumulative limit; `=` is required
boast about --repo owner/repo --wait-for-cohort-ranks=30s

Accepted duration units include s, m and h; zero and malformed durations are usage errors. The optional value requires = so it cannot consume a positional repository identifier.

Boast waits only when GitHub identifies the exhausted resource as Search and supplies a reset or retry time. Each planned delay and cumulative progress is shown on stderr. It continues until all requested ranks are collected or the cumulative limit expires; expiry produces the same successful partial result and durable Provider Note. GitHub Search bypasses the generic retry layer, so unrelated 403s, server errors, malformed responses and transport errors do not cause hidden sleeps.

Manifest behaviour

Generated Manifests persist selection, but not waiting:

[[project]]
identities = ["github:owner/repo"]
topics = ["rna-seq", "nextflow"]
[[project]]
identities = ["github:owner/repo"]
priority_topics = ["bioinformatics", "nextflow"]

topics and priority_topics are mutually exclusive. Legacy topic = "..." remains readable, and CLI topic options override a Manifest's selection. --wait-for-cohort-ranks stays runtime-only so a committed Manifest does not unexpectedly make CI sleep.

Why these controls instead of --max-topics

An arbitrary numeric limit would preserve the misleading idea that boast intentionally selects an alphabetical prefix. It would not tell boast which Cohorts matter, and changing repository topic order could change the selected set. Exact selection and priority ordering make the user's intent explicit while preserving a safe, honest best-effort default. Bounded opt-in waiting covers the separate case where completeness matters more than run time.

Validation

Offline tests cover anonymous 10-request and authenticated 30-request exhaustion, preservation of successful metrics, stopped searches, exact and priority ordering, invalid priorities, argument conflicts, confirmed reset continuation, cumulative timeout, unrelated failures, Manifest compatibility, schema v1 reads and offline Provider Note rendering.

  • cargo test --all
  • cargo clippy --all-targets -- -D warnings
  • cargo check --all-targets
  • mdbook build docs/site
  • git diff --check

Closes #83

@mbhall88
mbhall88 marked this pull request as ready for review August 10, 2026 11:47
@mbhall88
mbhall88 merged commit 02cbcf0 into main Aug 10, 2026
12 checks passed
@mbhall88
mbhall88 deleted the codex/github-cohort-collection branch August 10, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature request: allow configurable limit for topic cohort ranks (currently capped at 5 tags alphabetically)

1 participant