Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions skills/maka-github-research/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
name: maka-github-research
description: Research public GitHub repositories — issues, pull requests, discussions, comments, and timelines — with an anonymous-first, evidence-backed workflow. Use when Maka must inspect public GitHub work (including its own apache/maka tracker) without local CLI credentials, HTML scraping, or several custom parsers. It resolves canonical repository redirects, prefers unauthenticated REST, paginates deterministically, records rate-limit evidence, and escalates to authentication only when the endpoint provably requires it. Do not use it to weaken the Host sandbox, to embed credentials in the model context, or to promote a forge adapter into core; it is a research recipe layer, not a new authority.
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Maka GitHub Research

Inspect public GitHub repositories predictably and cheaply. This skill replaces the fragile improvised chain that a real Maka session had to build for ordinary public research (`gh search` → `401` → anonymous REST → repository redirect discovery → REST issue fetch → GraphQL auth failure → HTML scraping → link extraction → per-page fetch → extra metadata calls). The goal is one dependable route, not ten unrelated ones.

Tracks the "Structured public GitHub research" item of the harness friction umbrella (apache/maka#4267, item 4): *public repository research should not require local CLI credentials, HTML scraping, and several custom parsers.*

## Core principle: anonymous-first, escalate on evidence

1. **Assume no credentials.** Public repositories are fully researchable over unauthenticated REST. Start there so the workflow is reproducible on any machine and never blocks on a stale or scoped local token.
2. **Escalate only when an endpoint provably rejects anonymous access** (HTTP `401`/`403` from the endpoint itself, not a guess). Discussions search is the main case — see below.
3. **Do not scrape HTML** to recover data the REST API already returns as JSON (author, state, labels, comments, timeline, linked work). HTML scraping is the fallback of last resort, never the default.
4. **Record rate-limit evidence** (`X-RateLimit-Remaining` / `-Reset`) whenever you report results, so the user sees the real budget instead of a silent failure when it runs out.

## Workflow

1. **Resolve the canonical repository first.** Owners rename and redirect (the observed session hit `Maka-Agent/maka-agent` → `apache/maka`). Follow the redirect before any other call so every later request targets the real repository:

```bash
curl -sSL -o /dev/null -w '%{url_effective}\n' \
https://api.github.com/repos/Maka-Agent/maka-agent
# -> https://api.github.com/repositories/1251460378 (id-canonical)
curl -sSL https://api.github.com/repos/Maka-Agent/maka-agent \
| grep '"full_name"'
# -> "full_name": "apache/maka",
```

Use the resolved `owner/repo` (or the numeric `repositories/{id}` form) for everything downstream.

2. **Pick the narrowest endpoint for the question.** Prefer a direct object read over search; prefer search over listing; prefer any REST route over scraping. See `references/rest-recipes.md` for copy-paste recipes covering: single issue/PR, comments, timeline (labels, cross-references, linked PRs), issue/PR search, per-file PR diff, and discussions.

3. **Paginate deterministically.** Use `per_page=100` and follow the `Link: rel="next"` header (or increment `page=` until an empty array). Never assume the first page is complete.

4. **Collect metadata from the API, not the page.** Canonical URL (`html_url`), author (`user.login`), `state`, `labels[].name`, `comments`, and linked work (timeline `cross-referenced` / `connected` events) are all JSON fields. Read `references/rest-recipes.md#metadata` for the exact `jq` selectors.

5. **Attach rate-limit evidence to the answer.** Report remaining budget and reset time; if a call fails, distinguish "rate-limited" from "not found" from "needs auth" using the response headers and status — do not report an empty result as "nothing found."

## Discussions: the one endpoint that needs auth

GitHub Discussions have **no anonymous REST list/search**; the data lives behind GraphQL, and unauthenticated GraphQL returns `403` (verified). Do **not** fall straight to HTML scraping. In order:

1. Try the REST search API scoped to the repo (`/search/issues` covers issues and PRs anonymously) for adjacent context.
2. For Discussions specifically, escalate to a token per `references/rate-limits-and-escalation.md` and use the documented GraphQL query — this is the sanctioned exception, gated on the `403` evidence.
3. Only if authentication is genuinely unavailable, fetch the discussion `html_url` with WebFetch for the body, and state explicitly in the result that metadata (labels, answer state, comment authors) could not be recovered anonymously.

## Escalation ladder

Anonymous REST → repo-scoped REST search → authenticated REST/GraphQL (only on `401`/`403` from the endpoint) → WebFetch of the canonical `html_url` (content only) → HTML scraping (last resort, flagged as lossy). Never skip a rung to reach for scraping. See `references/rate-limits-and-escalation.md` for exactly when and how to add a token without leaking it into the model context.

## Non-goals

- Not a new capability, permission, or network authority; it runs entirely on tools Maka already exposes (`ShellRun`/`curl`, `gh`, `WebFetch`).
- Does not embed or print credentials. Tokens come from the environment/`gh` keyring; never echo them.
- Does not promote a forge adapter into core. If a recurring need justifies that, open a focused issue — this skill is the unauthenticated-first precursor the umbrella asks for.

## References

- `references/rest-recipes.md` — tested anonymous-first recipes for every common object and query.
- `references/rate-limits-and-escalation.md` — rate-limit budgets, evidence headers, the discussions/GraphQL exception, and safe token escalation.
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Rate limits, evidence, and safe escalation

## Anonymous budget (measure, don't assume)

Unauthenticated REST is limited **per source IP**. Read the real budget instead of guessing:

```bash
curl -sSD - -o /dev/null https://api.github.com/rate_limit \
| grep -i '^x-ratelimit-'
# X-RateLimit-Limit: 60
# X-RateLimit-Remaining: 33
# X-RateLimit-Reset: 1725446400 # unix seconds
```

Observed anonymous ceilings (subject to change — always confirm from headers):

| Endpoint family | Anonymous limit | Notes |
|------------------------|------------------------|--------------------------------------------------|
| Core REST (`/repos/…`) | ~60 requests / hour | Shared across all core calls from the IP. |
| Search (`/search/…`) | ~10 requests / minute | Separate bucket from core. |
| GraphQL (`/graphql`) | **auth required** | Anonymous returns `403`. |

Every REST response carries these headers — attach `Remaining`/`Reset` to any result you report so the user sees the budget, and back off (or escalate) *before* hitting zero rather than after.

## Classify failures — never report an empty result as "nothing found"

| Symptom | Meaning | Action |
|---------------------------------------------------|--------------------------------------|---------------------------------------------------------------|
| `403` + `X-RateLimit-Remaining: 0` | Rate-limited, not forbidden | Wait until `Reset`, or authenticate to raise the ceiling. |
| `403` on `/graphql` (Discussions) | Endpoint needs auth | Escalate to a token; do not scrape first. |
| `401` | A token was sent and is bad/expired | Fall back to **anonymous** (drop the token), then re-auth. |
| `404` on a known-public repo | Renamed/moved | Re-resolve the canonical repo (redirect step). |
| `301`/`302` | Repo redirect | Follow with `-L`; use the resolved `full_name`. |
| `200` + empty `items` | Genuinely no matches | Report as empty *with* the query used, so it can be widened. |

The observed session's `gh search` `401` was this table's row 3: a local token was present but rejected. The fix is to drop to anonymous REST immediately, which is why this skill starts there.

## Escalation ladder (in order; stop at the first that answers)

1. **Anonymous REST** — direct object read, then repo-scoped search.
2. **Authenticated REST/GraphQL** — only when the endpoint itself returns `401`/`403` (Discussions, or you exhausted the anonymous budget). Raises limits to ~5,000 core req/hour and unlocks GraphQL.
3. **WebFetch of the canonical `html_url`** — for prose/rendering the API does not expose; content only, metadata flagged as unavailable.
4. **HTML scraping** — last resort, explicitly labelled lossy in the result.

## Adding a token without leaking it

- Prefer the existing `gh` keyring: `gh api <path>` and `gh api graphql` use it automatically. Confirm with `gh auth status` (do **not** print the token).
- If using raw `curl`, read the token from the environment, never a literal:

```bash
curl -sS -H "Authorization: Bearer $GITHUB_TOKEN" "$API/…"
```

- Never echo, log, or paste a token into the transcript, a commit, or a tool argument that is rendered back. Redact it if a command would surface it.
- A read-only public-repo research task needs no scopes beyond default read; do not request write scopes for research.

## When to stop using this skill and open an issue instead

If a workflow repeatedly needs authenticated GraphQL (heavy Discussions research) or structured cross-repo aggregation, that is the signal to open a focused issue for a first-class forge research surface — the umbrella (apache/maka#4267) explicitly wants this unauthenticated-first skill *before* any adapter is promoted into core. Capture the concrete recurring need in that issue rather than growing this skill into a de-facto adapter.
141 changes: 141 additions & 0 deletions skills/maka-github-research/references/rest-recipes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Anonymous-first REST recipes

Every recipe below works **without authentication** against public repositories unless explicitly marked. All were exercised against `apache/maka`. Set a base once:

```bash
API=https://api.github.com
H='-H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28"'
```

Prefer `gh api <path>` when a token is already present (it paginates and sets headers for you); the raw `curl` form is the credential-free fallback that always works.

## 0. Resolve the canonical repository (always first)

```bash
# Redirect resolution — renamed/moved repos answer with 301 to the id-canonical URL.
curl -sSL https://api.github.com/repos/Maka-Agent/maka-agent | jq -r .full_name
# -> apache/maka
```

Reuse the resolved `OWNER/REPO` below.

## 1. A single issue or pull request

A PR *is* an issue for metadata; use `/issues/{n}` for state/labels/author/comment-count, `/pulls/{n}` for diff/merge details.

```bash
OWNER=apache REPO=maka N=4267
curl -sS "$API/repos/$OWNER/$REPO/issues/$N" \
| jq '{number, title, state, author: .user.login, labels: [.labels[].name], comments, html_url}'
```

## 2. Comments on an issue/PR (paginated)

```bash
curl -sS "$API/repos/$OWNER/$REPO/issues/$N/comments?per_page=100" \
| jq -r '.[] | "\(.user.login) @ \(.created_at):\n\(.body)\n---"'
```

If the response fills 100 items, follow the `Link` header:

```bash
curl -sSD /tmp/h.txt "$API/repos/$OWNER/$REPO/issues/$N/comments?per_page=100&page=1" -o /tmp/c.json
grep -i '^link:' /tmp/h.txt # look for rel="next"
```

## 3. Timeline — labels, linked PRs, cross-references {#metadata}

The timeline endpoint is how you recover "linked work" without scraping. It needs the preview `Accept` on older servers but works on github.com anonymously:

```bash
curl -sS -H "Accept: application/vnd.github+json" \
"$API/repos/$OWNER/$REPO/issues/$N/timeline?per_page=100" \
| jq -r '.[] | select(.event | test("cross-referenced|connected|labeled|closed")) | .event'
```

Useful selectors:

- `cross-referenced` → `.source.issue.html_url` (issue/PR that mentions this one)
- `connected` / `disconnected` → linked PR lifecycle
- `labeled` / `unlabeled` → `.label.name`
- `closed` → `.commit_id` (closing commit, if any)

## 4. Search issues and PRs (anonymous, repo-scoped)

The Search API returns `200` anonymously; scope with `repo:` to keep the query cheap and specific.

```bash
Q='repo:apache/maka+harness+in:title+state:open'
curl -sS "$API/search/issues?q=$Q&per_page=20" \
| jq -r '.items[] | "#\(.number) [\(.state)] \(.title) \(.html_url)"'
```

Qualifiers worth knowing: `is:issue` / `is:pr`, `state:open|closed`, `label:"enhancement"`, `author:<login>`, `in:title,body`, `created:>2026-01-01`, `sort=updated`.

## 5. Pull request specifics

```bash
PR=4380
# Files + patch per file (no scraping of the "Files changed" tab)
curl -sS "$API/repos/$OWNER/$REPO/pulls/$PR/files?per_page=100" \
| jq -r '.[] | "\(.status)\t+\(.additions)-\(.deletions)\t\(.filename)"'
# Review state and mergeability
curl -sS "$API/repos/$OWNER/$REPO/pulls/$PR" \
| jq '{state, merged, mergeable, draft, base: .base.ref, head: .head.ref}'
```

Full unified diff without cloning:

```bash
curl -sS -H "Accept: application/vnd.github.v3.diff" \
"$API/repos/$OWNER/$REPO/pulls/$PR" -o pr-$PR.diff
```

## 6. Repository facts (default branch, topics, redirect target)

```bash
curl -sS "$API/repos/$OWNER/$REPO" \
| jq '{full_name, default_branch, description, open_issues_count, topics}'
```

## 7. Discussions (needs a token — see rate-limits reference)

There is **no anonymous Discussions list/search**. With a token:

```bash
gh api graphql -f query='
query($owner:String!,$name:String!){
repository(owner:$owner,name:$name){
discussions(first:20, orderBy:{field:UPDATED_AT, direction:DESC}){
nodes{ number title url category{name} answerChosenAt
comments{totalCount} author{login} }
}
}
}' -F owner=$OWNER -F name=$REPO
```

Anonymous GraphQL returns `403`; do not retry it unauthenticated, and do not scrape — escalate per the rate-limits reference.

## Notes

- Every JSON field above is canonical API data; reach for WebFetch/HTML only when the API cannot answer (Discussions without a token, or prose rendering).
- `jq` and `curl` are assumed present; if `jq` is unavailable, `gh api --jq` uses the built-in filter, and `python3 -m json.tool` is a portable pretty-printer.