Skip to content

feat(triage): add entry point, prompt template, and TRIAGE.md example#137

Merged
keinstn merged 5 commits into
developfrom
agent/keinstn__baton-131
Jun 24, 2026
Merged

feat(triage): add entry point, prompt template, and TRIAGE.md example#137
keinstn merged 5 commits into
developfrom
agent/keinstn__baton-131

Conversation

@keinstn

@keinstn keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/triage/index.ts: loads config, fetches issues grouped by repo, runs the evaluator per group, and applies addLabel / postComment / skip based on each IssueDecision. Errors in one repo group are caught and logged without aborting other groups. Falls back to scripts/triage/prompt.md when the config file has no prompt body.
  • Adds scripts/triage/prompt.md: default LiquidJS evaluation prompt that instructs the LLM to assess spec completeness, dependency status, and relative priority, then return a JSON array of IssueDecision objects with no other output.
  • Adds TRIAGE.md at repo root: fully documented example config with all fields explained via YAML comments, empty prompt body (triggers default prompt fallback), and a cron example in comments.

Closes #131. Part of #126.

Test plan

  • npm run typecheck passes
  • npm run lint passes
  • npm test — all 323 tests pass (no new tests needed; entry point logic is covered by existing unit tests for each composed module)
  • node --experimental-strip-types scripts/triage/index.ts against a real board with GITHUB_TOKEN set completes without error

🤖 Generated with Claude Code

Wire scripts/triage/index.ts to load config, fetch issues by repo, evaluate
per group, and apply label/comment/skip actions. Add default LiquidJS prompt
in scripts/triage/prompt.md and a fully documented TRIAGE.md example at repo root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Reviewer] [must] Needs changes: restore the required defaults from #131 (TRIAGE.md at repo root and scripts/triage/prompt.md as the default prompt template).
Managed by Baton; do not edit the marker line manually.

@keinstn keinstn left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

インラインコメント(3件)に加え、本PRが依存する既存ファイルへの指摘を以下に記載します。


scripts/triage/parse.ts:17 — unchecked cast で実行時型エラーが検出されない

return parsed as IssueDecision[];

Array.isArray() チェックの後に as IssueDecision[] でキャストしているが、各要素のフィールド型は実行時に検証されない。LLM が "number": "42" (文字列) や "number": null を返した場合、TypeScript はコンパイル時にエラーを出さず addLabel(repo, decision.number, ...) にそのまま流れる。

最低限 number フィールドが数値であることを確認するガードを追加することを推奨:

const decisions = parsed as IssueDecision[];
for (const d of decisions) {
  if (typeof d.number !== "number") {
    throw new Error(
      `eval: decision.number must be a number, got ${JSON.stringify(d.number)}`,
    );
  }
}
return decisions;

scripts/triage/actions.ts:23,41repo.split('/') の結果が未検証

const [owner, name] = repo.split("/");

repo"owner/name" 形式でない場合(TRIAGE.md の設定ミスなど)、nameundefined になり、URL が /repos/owner/undefined/issues/… となって GitHub から 404 が返る。エラーメッセージから原因を特定しにくい。

const parts = repo.split("/");
if (parts.length !== 2 || !parts[0] || !parts[1]) {
  throw new Error(`invalid repo format (expected "owner/name"): ${repo}`);
}
const [owner, name] = parts as [string, string];

scripts/triage/claude-eval.ts:44permissionMode のデフォルト値が二重定義

const permissionMode = config.permissionMode ?? "bypassPermissions";

parseTriageConfigconfig.ts)がすでに permissionMode = permissionModeRaw ?? "bypassPermissions" としてデフォルトを解決しており、EvaluatorConfig.permissionMode は常に非 undefined の文字列として格納される。この ?? フォールバックは実行されないデッドコード。

EvaluatorConfig.permissionMode の型を string | undefined から string に変更するか、この行を const permissionMode = config.permissionMode; に修正することで意図を明確にできる。

Comment thread scripts/triage/index.ts Outdated
Comment thread scripts/triage/index.ts Outdated
Comment thread scripts/triage/index.ts Outdated
- parse.ts: add runtime validation that decision.number is a number
- actions.ts: validate repo "owner/name" format before split
- claude-eval.ts: remove redundant ?? fallback now that EvaluatorConfig.permissionMode is always set
- config.ts: tighten EvaluatorConfig.permissionMode to string (non-optional)
- index.ts: split totalErrors into eval_errors/action_errors; warn on missing decisions; use || for comment fallback with a log warning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Implementer] Addressed all three findings from the review body:

  • parse.ts:17 — Added a per-element runtime check: if typeof d.number !== "number" an error is thrown immediately with the serialized value, catching strings, nulls, etc. before they reach addLabel.
  • actions.ts:23,41 — Extracted a splitRepo helper that validates the owner/name format and throws a descriptive error if the split does not yield exactly two non-empty parts. Both addLabel and postComment now call it.
  • claude-eval.ts:44 — Removed the redundant ?? "bypassPermissions" fallback. Tightened EvaluatorConfig.permissionMode to string (non-optional) in config.ts; parseTriageConfig already guarantees a value, so the ?? was dead code.

@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Reviewer] [must] scripts/triage/index.ts applies actions to every decision returned by the LLM without verifying that decision.number belongs to the fetched issues for that repo group. If the model returns an extra or hallucinated issue number, triage can add labels or post comments to an unintended issue in the same repository.

Please constrain actions to the fetched issue-number set for that repo (and warn on unexpected extras), so evaluator output cannot mutate issues outside the current batch.

…po group

Adds a `validDecisions` filter after `parseDecisions` returns: any decision
whose issue number is not in the fetched set for the current repo group is
dropped with a `warn` log, preventing the evaluator from mutating unintended
issues via hallucinated or out-of-scope numbers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Implementer] Fixed. Added a validDecisions filter immediately after missing is computed: any decision whose number is not in fetchedNumbers (the set of issue numbers returned by the fetcher for this repo group) is dropped with a warn log that includes the unknown issue_number. Only validDecisions are passed to the addLabel/postComment/skip loop, so hallucinated or out-of-scope numbers cannot mutate unintended issues. Pushed in 34836b8.

@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Implementer] All findings addressed. The blocking finding (decision membership validation) has been fixed in 34836b8 — decisions whose issue numbers are not in the fetched repo group are now rejected before any label or comment action is applied.

@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Reviewer] [must] TRIAGE.md should not place documentation comments after the closing ---. Those lines are treated as the prompt body, so the example file overrides scripts/triage/prompt.md with documentation prose instead of the triage instructions.

Move the comments above the front matter or remove them from the body so the example can fall back to the default prompt template.

…rride

Documentation comments explaining the optional prompt-body template were
placed at the end of the YAML front matter, right above the closing ---.
That position made them look like they document (or belong to) the prompt
body area, creating a risk that users copying the example would leave the
prose in the body and send it to the LLM instead of the built-in default.

Move the cron example and template explanation to the top of the YAML
block (as a file-level header), and remove them from the end of the
front matter so the closing --- is followed only by the empty prompt
body that correctly triggers the default-prompt fallback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Implementer] Fixed. The documentation comments about the optional prompt-body template were sitting at the very end of the YAML front matter (just before the closing ---), making them look like they described content that belonged after the delimiter. Moved them to the top of the YAML block as a file-level header comment (cron example + template explanation appear right after the opening ---, before tracker:). The closing --- is now followed only by the empty body that correctly triggers the built-in default fallback. Pushed in 72d4a7c.

@keinstn keinstn left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scripts/triage/prompt.md fallback pattern diverges from how WORKFLOW.md works: config + prompt are always one file. Three changes to align with that convention:

  1. Delete scripts/triage/prompt.md.
  2. Move TRIAGE.mdexamples/TRIAGE.md and append the prompt body inline (mirrors examples/WORKFLOW.md).
  3. In index.ts, remove DEFAULT_PROMPT_PATH and the fallback readFilepromptTemplate is just rawPromptTemplate.

Comment thread examples/TRIAGE.md
Comment thread scripts/triage/index.ts Outdated
Comment thread scripts/triage/index.ts Outdated
Comment thread TRIAGE.md Outdated
…ve fallback

- Create `examples/TRIAGE.md` combining the config skeleton from `TRIAGE.md`
  and the default prompt body from `scripts/triage/prompt.md`, mirroring
  `examples/WORKFLOW.md` structure
- Delete standalone `scripts/triage/prompt.md` — no longer needed as a
  separate file
- Remove `DEFAULT_PROMPT_PATH` constant and the silent fallback that read it;
  replace with direct use of `promptTemplate` from `loadTriageConfig`
- Remove unused `readFile` and `fileURLToPath` imports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@keinstn

keinstn commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

[Baton Reviewer] [must] This PR removes both required defaults from the issue contract: TRIAGE.md at the repo root and scripts/triage/prompt.md.

That breaks the documented default invocation path (node --experimental-strip-types scripts/triage/index.ts expects TRIAGE.md in cwd) and removes the shipped default evaluation prompt the script is supposed to provide. As a result, the delivered behavior no longer matches the issue scope/acceptance criteria and can fail or produce unreliable evaluator output when no inline prompt body is provided.

Please restore the contract from #131: keep a documented root TRIAGE.md example and restore scripts/triage/prompt.md as the default prompt template used by index.ts when config body is empty.

@keinstn
keinstn merged commit 113c498 into develop Jun 24, 2026
4 checks passed
@keinstn
keinstn deleted the agent/keinstn__baton-131 branch June 24, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

triage: Add entry point, prompt template, and TRIAGE.md example

1 participant