diff --git a/skills/maka-github-research/SKILL.md b/skills/maka-github-research/SKILL.md new file mode 100644 index 0000000000..28176631a5 --- /dev/null +++ b/skills/maka-github-research/SKILL.md @@ -0,0 +1,81 @@ +--- +name: maka-github-research +description: Research public GitHub repositories — issues, pull requests, discussions, comments, and timelines — with an anonymous-first, evidence-backed workflow. Use when Maka must inspect public GitHub work (including its own apache/maka tracker) without local CLI credentials, HTML scraping, or several custom parsers. It resolves canonical repository redirects, prefers unauthenticated REST, paginates deterministically, records rate-limit evidence, and escalates to authentication only when the endpoint provably requires it. Do not use it to weaken the Host sandbox, to embed credentials in the model context, or to promote a forge adapter into core; it is a research recipe layer, not a new authority. +--- + + +# Maka GitHub Research + +Inspect public GitHub repositories predictably and cheaply. This skill replaces the fragile improvised chain that a real Maka session had to build for ordinary public research (`gh search` → `401` → anonymous REST → repository redirect discovery → REST issue fetch → GraphQL auth failure → HTML scraping → link extraction → per-page fetch → extra metadata calls). The goal is one dependable route, not ten unrelated ones. + +Tracks the "Structured public GitHub research" item of the harness friction umbrella (apache/maka#4267, item 4): *public repository research should not require local CLI credentials, HTML scraping, and several custom parsers.* + +## Core principle: anonymous-first, escalate on evidence + +1. **Assume no credentials.** Public repositories are fully researchable over unauthenticated REST. Start there so the workflow is reproducible on any machine and never blocks on a stale or scoped local token. +2. **Escalate only when an endpoint provably rejects anonymous access** (HTTP `401`/`403` from the endpoint itself, not a guess). Discussions search is the main case — see below. +3. **Do not scrape HTML** to recover data the REST API already returns as JSON (author, state, labels, comments, timeline, linked work). HTML scraping is the fallback of last resort, never the default. +4. **Record rate-limit evidence** (`X-RateLimit-Remaining` / `-Reset`) whenever you report results, so the user sees the real budget instead of a silent failure when it runs out. + +## Workflow + +1. **Resolve the canonical repository first.** Owners rename and redirect (the observed session hit `Maka-Agent/maka-agent` → `apache/maka`). Follow the redirect before any other call so every later request targets the real repository: + + ```bash + curl -sSL -o /dev/null -w '%{url_effective}\n' \ + https://api.github.com/repos/Maka-Agent/maka-agent + # -> https://api.github.com/repositories/1251460378 (id-canonical) + curl -sSL https://api.github.com/repos/Maka-Agent/maka-agent \ + | grep '"full_name"' + # -> "full_name": "apache/maka", + ``` + + Use the resolved `owner/repo` (or the numeric `repositories/{id}` form) for everything downstream. + +2. **Pick the narrowest endpoint for the question.** Prefer a direct object read over search; prefer search over listing; prefer any REST route over scraping. See `references/rest-recipes.md` for copy-paste recipes covering: single issue/PR, comments, timeline (labels, cross-references, linked PRs), issue/PR search, per-file PR diff, and discussions. + +3. **Paginate deterministically.** Use `per_page=100` and follow the `Link: rel="next"` header (or increment `page=` until an empty array). Never assume the first page is complete. + +4. **Collect metadata from the API, not the page.** Canonical URL (`html_url`), author (`user.login`), `state`, `labels[].name`, `comments`, and linked work (timeline `cross-referenced` / `connected` events) are all JSON fields. Read `references/rest-recipes.md#metadata` for the exact `jq` selectors. + +5. **Attach rate-limit evidence to the answer.** Report remaining budget and reset time; if a call fails, distinguish "rate-limited" from "not found" from "needs auth" using the response headers and status — do not report an empty result as "nothing found." + +## Discussions: the one endpoint that needs auth + +GitHub Discussions have **no anonymous REST list/search**; the data lives behind GraphQL, and unauthenticated GraphQL returns `403` (verified). Do **not** fall straight to HTML scraping. In order: + +1. Try the REST search API scoped to the repo (`/search/issues` covers issues and PRs anonymously) for adjacent context. +2. For Discussions specifically, escalate to a token per `references/rate-limits-and-escalation.md` and use the documented GraphQL query — this is the sanctioned exception, gated on the `403` evidence. +3. Only if authentication is genuinely unavailable, fetch the discussion `html_url` with WebFetch for the body, and state explicitly in the result that metadata (labels, answer state, comment authors) could not be recovered anonymously. + +## Escalation ladder + +Anonymous REST → repo-scoped REST search → authenticated REST/GraphQL (only on `401`/`403` from the endpoint) → WebFetch of the canonical `html_url` (content only) → HTML scraping (last resort, flagged as lossy). Never skip a rung to reach for scraping. See `references/rate-limits-and-escalation.md` for exactly when and how to add a token without leaking it into the model context. + +## Non-goals + +- Not a new capability, permission, or network authority; it runs entirely on tools Maka already exposes (`ShellRun`/`curl`, `gh`, `WebFetch`). +- Does not embed or print credentials. Tokens come from the environment/`gh` keyring; never echo them. +- Does not promote a forge adapter into core. If a recurring need justifies that, open a focused issue — this skill is the unauthenticated-first precursor the umbrella asks for. + +## References + +- `references/rest-recipes.md` — tested anonymous-first recipes for every common object and query. +- `references/rate-limits-and-escalation.md` — rate-limit budgets, evidence headers, the discussions/GraphQL exception, and safe token escalation. diff --git a/skills/maka-github-research/references/rate-limits-and-escalation.md b/skills/maka-github-research/references/rate-limits-and-escalation.md new file mode 100644 index 0000000000..dcc23e2805 --- /dev/null +++ b/skills/maka-github-research/references/rate-limits-and-escalation.md @@ -0,0 +1,78 @@ + + +# Rate limits, evidence, and safe escalation + +## Anonymous budget (measure, don't assume) + +Unauthenticated REST is limited **per source IP**. Read the real budget instead of guessing: + +```bash +curl -sSD - -o /dev/null https://api.github.com/rate_limit \ + | grep -i '^x-ratelimit-' +# X-RateLimit-Limit: 60 +# X-RateLimit-Remaining: 33 +# X-RateLimit-Reset: 1725446400 # unix seconds +``` + +Observed anonymous ceilings (subject to change — always confirm from headers): + +| Endpoint family | Anonymous limit | Notes | +|------------------------|------------------------|--------------------------------------------------| +| Core REST (`/repos/…`) | ~60 requests / hour | Shared across all core calls from the IP. | +| Search (`/search/…`) | ~10 requests / minute | Separate bucket from core. | +| GraphQL (`/graphql`) | **auth required** | Anonymous returns `403`. | + +Every REST response carries these headers — attach `Remaining`/`Reset` to any result you report so the user sees the budget, and back off (or escalate) *before* hitting zero rather than after. + +## Classify failures — never report an empty result as "nothing found" + +| Symptom | Meaning | Action | +|---------------------------------------------------|--------------------------------------|---------------------------------------------------------------| +| `403` + `X-RateLimit-Remaining: 0` | Rate-limited, not forbidden | Wait until `Reset`, or authenticate to raise the ceiling. | +| `403` on `/graphql` (Discussions) | Endpoint needs auth | Escalate to a token; do not scrape first. | +| `401` | A token was sent and is bad/expired | Fall back to **anonymous** (drop the token), then re-auth. | +| `404` on a known-public repo | Renamed/moved | Re-resolve the canonical repo (redirect step). | +| `301`/`302` | Repo redirect | Follow with `-L`; use the resolved `full_name`. | +| `200` + empty `items` | Genuinely no matches | Report as empty *with* the query used, so it can be widened. | + +The observed session's `gh search` `401` was this table's row 3: a local token was present but rejected. The fix is to drop to anonymous REST immediately, which is why this skill starts there. + +## Escalation ladder (in order; stop at the first that answers) + +1. **Anonymous REST** — direct object read, then repo-scoped search. +2. **Authenticated REST/GraphQL** — only when the endpoint itself returns `401`/`403` (Discussions, or you exhausted the anonymous budget). Raises limits to ~5,000 core req/hour and unlocks GraphQL. +3. **WebFetch of the canonical `html_url`** — for prose/rendering the API does not expose; content only, metadata flagged as unavailable. +4. **HTML scraping** — last resort, explicitly labelled lossy in the result. + +## Adding a token without leaking it + +- Prefer the existing `gh` keyring: `gh api ` and `gh api graphql` use it automatically. Confirm with `gh auth status` (do **not** print the token). +- If using raw `curl`, read the token from the environment, never a literal: + + ```bash + curl -sS -H "Authorization: Bearer $GITHUB_TOKEN" "$API/…" + ``` + +- Never echo, log, or paste a token into the transcript, a commit, or a tool argument that is rendered back. Redact it if a command would surface it. +- A read-only public-repo research task needs no scopes beyond default read; do not request write scopes for research. + +## When to stop using this skill and open an issue instead + +If a workflow repeatedly needs authenticated GraphQL (heavy Discussions research) or structured cross-repo aggregation, that is the signal to open a focused issue for a first-class forge research surface — the umbrella (apache/maka#4267) explicitly wants this unauthenticated-first skill *before* any adapter is promoted into core. Capture the concrete recurring need in that issue rather than growing this skill into a de-facto adapter. diff --git a/skills/maka-github-research/references/rest-recipes.md b/skills/maka-github-research/references/rest-recipes.md new file mode 100644 index 0000000000..95d1afbdad --- /dev/null +++ b/skills/maka-github-research/references/rest-recipes.md @@ -0,0 +1,141 @@ + + +# Anonymous-first REST recipes + +Every recipe below works **without authentication** against public repositories unless explicitly marked. All were exercised against `apache/maka`. Set a base once: + +```bash +API=https://api.github.com +H='-H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28"' +``` + +Prefer `gh api ` when a token is already present (it paginates and sets headers for you); the raw `curl` form is the credential-free fallback that always works. + +## 0. Resolve the canonical repository (always first) + +```bash +# Redirect resolution — renamed/moved repos answer with 301 to the id-canonical URL. +curl -sSL https://api.github.com/repos/Maka-Agent/maka-agent | jq -r .full_name +# -> apache/maka +``` + +Reuse the resolved `OWNER/REPO` below. + +## 1. A single issue or pull request + +A PR *is* an issue for metadata; use `/issues/{n}` for state/labels/author/comment-count, `/pulls/{n}` for diff/merge details. + +```bash +OWNER=apache REPO=maka N=4267 +curl -sS "$API/repos/$OWNER/$REPO/issues/$N" \ + | jq '{number, title, state, author: .user.login, labels: [.labels[].name], comments, html_url}' +``` + +## 2. Comments on an issue/PR (paginated) + +```bash +curl -sS "$API/repos/$OWNER/$REPO/issues/$N/comments?per_page=100" \ + | jq -r '.[] | "\(.user.login) @ \(.created_at):\n\(.body)\n---"' +``` + +If the response fills 100 items, follow the `Link` header: + +```bash +curl -sSD /tmp/h.txt "$API/repos/$OWNER/$REPO/issues/$N/comments?per_page=100&page=1" -o /tmp/c.json +grep -i '^link:' /tmp/h.txt # look for rel="next" +``` + +## 3. Timeline — labels, linked PRs, cross-references {#metadata} + +The timeline endpoint is how you recover "linked work" without scraping. It needs the preview `Accept` on older servers but works on github.com anonymously: + +```bash +curl -sS -H "Accept: application/vnd.github+json" \ + "$API/repos/$OWNER/$REPO/issues/$N/timeline?per_page=100" \ + | jq -r '.[] | select(.event | test("cross-referenced|connected|labeled|closed")) | .event' +``` + +Useful selectors: + +- `cross-referenced` → `.source.issue.html_url` (issue/PR that mentions this one) +- `connected` / `disconnected` → linked PR lifecycle +- `labeled` / `unlabeled` → `.label.name` +- `closed` → `.commit_id` (closing commit, if any) + +## 4. Search issues and PRs (anonymous, repo-scoped) + +The Search API returns `200` anonymously; scope with `repo:` to keep the query cheap and specific. + +```bash +Q='repo:apache/maka+harness+in:title+state:open' +curl -sS "$API/search/issues?q=$Q&per_page=20" \ + | jq -r '.items[] | "#\(.number) [\(.state)] \(.title) \(.html_url)"' +``` + +Qualifiers worth knowing: `is:issue` / `is:pr`, `state:open|closed`, `label:"enhancement"`, `author:`, `in:title,body`, `created:>2026-01-01`, `sort=updated`. + +## 5. Pull request specifics + +```bash +PR=4380 +# Files + patch per file (no scraping of the "Files changed" tab) +curl -sS "$API/repos/$OWNER/$REPO/pulls/$PR/files?per_page=100" \ + | jq -r '.[] | "\(.status)\t+\(.additions)-\(.deletions)\t\(.filename)"' +# Review state and mergeability +curl -sS "$API/repos/$OWNER/$REPO/pulls/$PR" \ + | jq '{state, merged, mergeable, draft, base: .base.ref, head: .head.ref}' +``` + +Full unified diff without cloning: + +```bash +curl -sS -H "Accept: application/vnd.github.v3.diff" \ + "$API/repos/$OWNER/$REPO/pulls/$PR" -o pr-$PR.diff +``` + +## 6. Repository facts (default branch, topics, redirect target) + +```bash +curl -sS "$API/repos/$OWNER/$REPO" \ + | jq '{full_name, default_branch, description, open_issues_count, topics}' +``` + +## 7. Discussions (needs a token — see rate-limits reference) + +There is **no anonymous Discussions list/search**. With a token: + +```bash +gh api graphql -f query=' + query($owner:String!,$name:String!){ + repository(owner:$owner,name:$name){ + discussions(first:20, orderBy:{field:UPDATED_AT, direction:DESC}){ + nodes{ number title url category{name} answerChosenAt + comments{totalCount} author{login} } + } + } + }' -F owner=$OWNER -F name=$REPO +``` + +Anonymous GraphQL returns `403`; do not retry it unauthenticated, and do not scrape — escalate per the rate-limits reference. + +## Notes + +- Every JSON field above is canonical API data; reach for WebFetch/HTML only when the API cannot answer (Discussions without a token, or prose rendering). +- `jq` and `curl` are assumed present; if `jq` is unavailable, `gh api --jq` uses the built-in filter, and `python3 -m json.tool` is a portable pretty-printer.