Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ concurrency:

jobs:
build-and-test:
name: Build and test
name: Check, test, and build
runs-on: ubuntu-latest

steps:
Expand All @@ -28,14 +28,17 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version: 22.18.0
cache: pnpm

- name: Install dependencies
run: pnpm install --frozen-lockfile

- name: Build
run: pnpm build
- name: Check
run: pnpm check:ci

- name: Test
run: pnpm test
run: pnpm test:ci

- name: Build and verify package
run: pnpm build:ci
23 changes: 16 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,27 @@ This repo uses a single-context domain docs layout. See `docs/agents/domain.md`.

- **Language**: TypeScript (ES2022, Node16 modules, strict mode)
- **Package manager**: pnpm with workspaces
- **Toolchain**: Vite+ (formatting, type-aware linting, tests, packaging, task orchestration)
- **TUI framework**: Ink 6 + React 19 (terminal UI rendered via Yoga layout)
- **Testing**: Vitest
- **Testing**: Vite+ Test (Vitest-compatible)
- **Parser**: YAML frontmatter in SKILL.md files

## Commands

```bash
pnpm install # install dependencies
pnpm build # build all packages (tsc)
pnpm test # run all tests
pnpm dev # watch mode for TUI
pnpm check # formatting, type-aware lint, and type checks
pnpm build # package Core and the TUI with Vite+ Pack
pnpm test # run all tests once
pnpm dev # build Core, then watch the TUI package

# CI-only entry points
pnpm check:ci
pnpm test:ci
pnpm build:ci # also verifies artifacts and npm package contents

# Run the app
node packages/tui/dist/bin/skillpack.js
node packages/tui/dist/skillpack.js
```

## Architecture
Expand Down Expand Up @@ -134,14 +141,16 @@ After any mutation (toggle, edit, delete, update), `refresh()` must be called. T
## Testing

- Tests live in `packages/core/tests/`
- Use `vitest` with `describe`/`it`/`expect`
- TUI tests live in `packages/tui/tests/`
- Import `describe`/`it`/`expect` and other test APIs from `vite-plus/test`
- Provider tests use temp directories (`mkdtemp`) cleaned up in `afterEach`
- TUI has no tests yet
- `pnpm test` and `pnpm test:ci` are one-shot commands; neither starts watch mode

## Common Pitfalls

- **Stale `selectedSkill`**: Always update `selectedSkill` after `refresh()` — it's a separate state from `skills[]`
- **Import extensions**: Must use `.js` in imports (`'./foo.js'`), not `.ts` — Node16 module resolution requires it
- **Async in `useInput`**: Fire-and-forget promises must have `.catch()` to avoid unhandled rejections crashing Ink
- **Ink JSX whitespace**: Encode intentional runs of spaces as expressions such as `{' '}` so formatting cannot change terminal alignment
- **`.pnpm-store/`**: Never commit — it's in `.gitignore`
- **ClaudeProvider custom scan**: `ClaudeProvider` overrides `scan()` with its own `scanFlat()` / `scanDeep()` — changes to `BaseProvider.scan()` don't apply to Claude skills. Any scan-level feature (symlink resolution, metadata enrichment, provider-native availability) must also be added to both Claude scan methods.
32 changes: 24 additions & 8 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ _Avoid_: Global Skill, managed skill
A skill distributed as part of a provider plugin. Its availability may depend on the owning plugin's access state as well as any provider-specific per-skill state.
_Avoid_: Provider-local skill, independent toggle target

**Built-In Skill**:
A Skill Provider bundled skill that has no user-level or plugin-provided source directory managed by the user.
_Avoid_: User Skill, Plugin-Owned Skill

**skills.sh**:
The external skill ecosystem and CLI used for installing, updating, and removing Global Skills.
_Avoid_: GitHub install source, package manager
Expand All @@ -61,11 +65,11 @@ A minimal normalized fact derived from provider session artifacts about one Skil
_Avoid_: Session transcript, prompt, response

**Invocation Record ID**:
A deterministic identifier for a Skill Invocation Record, derived from provider session evidence so repeated imports can be idempotent.
A deterministic identifier for a Skill Invocation, derived from provider session evidence so repeated imports are idempotent and later evidence can revise the same invocation.
_Avoid_: Random event ID, row number

**Skill Usage Log**:
The append-only JSONL storage for Skill Invocation Records, partitioned by Session-Producing Provider and month.
The append-only JSONL storage for Skill Invocation Records, partitioned by Session-Producing Provider and invocation month. Revisions of one Invocation Record ID remain in the same provider/month partition, and the latest revision is the current fact used by aggregates.
_Avoid_: Session transcript store, provider log, database

**Invocation Identity Confidence**:
Expand All @@ -77,9 +81,13 @@ The evidence-based runtime state for a Skill Invocation, indicating whether prov
_Avoid_: Skill Availability, Inventory Issue, inventory status

**Skill Source Path**:
The `SKILL.md` path recorded from provider session evidence for an invoked skill, falling back to a resolved path only when the evidence path is unavailable.
The optional `SKILL.md` path shown for an invocation or aggregate. An aggregate shows a common provider-facing path when one exists, otherwise a common resolved target, and otherwise leaves the path blank.
_Avoid_: Historical status, Scan Root, inventory path

**Skill Source Identity**:
The stable source-level identity used to group Skill Invocation Records within one Session-Producing Provider. A recognized Plugin-Owned Skill cache uses its normalized plugin identity first; other sources prefer the resolved target when known and fall back to the provider-facing Skill Source Path.
_Avoid_: Display path, skill name, Provider Instance

**Counted Invocation**:
A Skill Invocation Record included in default Skill Usage aggregates. Failed invocations are excluded from default intensity and ranking counts but can be shown as separate failure context.
_Avoid_: Successful invocation, completed task
Expand All @@ -89,11 +97,11 @@ User-facing aggregates derived from Skill Invocations, used to understand which
_Avoid_: Skill Inventory, Skill Availability, inventory status, session history, cross-provider skill rollup

**Provider Skill Ranking**:
A Skill Usage aggregate that ranks invoked skill source paths within a Skill Provider by exact Counted Invocation count, with failed invocation count and recency as supporting context.
A Skill Usage aggregate that ranks invoked skills within a Skill Provider by exact Counted Invocation count, separating rows by Skill Source Identity while treating versioned cache paths for one Plugin-Owned Skill as one source identity.
_Avoid_: Global leaderboard, inventory order

**Selected-Day Usage Detail**:
The Skill Usage detail for one selected heatmap day, scoped to one Session-Producing Provider and grouped by skill source path with exact Counted Invocation and failure context for that day.
The Skill Usage detail for one selected heatmap day, scoped to one Session-Producing Provider and grouped by skill name and stable source identity with exact Counted Invocation and failure context for that day.
_Avoid_: Session transcript, raw invocation history, provider-wide total

**Skill Usage Heatmap**:
Expand All @@ -109,7 +117,7 @@ The read-only process that derives Skill Invocation Records from provider-owned
_Avoid_: Runtime instrumentation, session sync

**Usage Import Diagnostic**:
A non-inventory finding from Skill Usage Import, such as skipped ambiguous provider evidence or unsupported session artifacts.
A non-inventory finding from Skill Usage Import, such as skipped provider evidence or unsupported session artifacts. Repeated findings may be summarized with a count.
_Avoid_: Inventory Issue, provider warning

**Usage Provider Adapter**:
Expand All @@ -120,6 +128,10 @@ _Avoid_: Heuristic log parser, runtime hook
Whether Skillpack can derive Skill Usage for a Skill Provider, distinguishing supported providers with zero Counted Invocations from providers that are unsupported or not configured for import.
_Avoid_: Skill Availability, zero usage

**Usage Coverage Reason**:
One specific missing prerequisite that explains a `not-configured` Usage Coverage State, such as missing Usage Artifact Roots or Skill Attribution Roots. Every Provider Skill Usage Overview carries a `coverageReasons` collection: it contains every applicable reason for `not-configured` coverage and is empty for other states. These are configuration context, not findings from an attempted import.
_Avoid_: Usage Import Diagnostic, import error

**Usage Import Consent**:
The user's persisted opt-in that allows Skillpack to read provider-owned session artifacts for Skill Usage Import.
_Avoid_: Config default, provider permission
Expand All @@ -138,7 +150,11 @@ _Avoid_: Session state, Skillpack-owned provider state

**Usage Artifact Root**:
A provider-specific directory or file root Skillpack reads during Skill Usage Import to find supported session artifacts.
_Avoid_: Scan Root, skill directory
_Avoid_: Scan Root, Skill Attribution Root, skill directory

**Skill Attribution Root**:
A configured skill-content root a Usage Provider Adapter uses to match invocation evidence to eligible user-level or plugin-provided Skill sources. The same directory may also serve as an Inventory Scan Root, but attribution and inventory discovery are separate responsibilities.
_Avoid_: Usage Artifact Root, session directory

**Provider Instance**:
A discovered skill as represented by one Skill Provider, including that provider's availability and provenance for the skill.
Expand All @@ -162,7 +178,7 @@ _Avoid_: Terminal UI Test, component test

**Scan Root**:
A directory Skillpack inspects to discover provider, shared global, or project skills.
_Avoid_: Skill, provider, install source
_Avoid_: Skill, provider, install source, Usage Artifact Root

**Inventory Notice**:
A non-problem inventory fact that helps the user understand provider coverage, provenance, or grouping confidence.
Expand Down
19 changes: 17 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Unified TUI manager for agent skills across Codex, Claude, and Global (`~/.agent
- **Project Skills** — scans read-only project skill directories (`.codex/skills`, `.claude/skills`, `.agents/skills`) in a separate view
- **Provider-native availability** — reads provider config when available and uses `.disabled-` renaming only as a fallback
- **Skill Inventory** — groups provider instances by Skill Identity and surfaces deterministic Health Signals
- **Skill Usage** — derives aggregate Codex skill usage from local session artifacts after explicit consent
- **Skill Usage** — derives aggregate Codex and Claude skill usage from local session artifacts after explicit consent
- **skills.sh installs** — installs, updates, and removes Global Skills through the skills.sh CLI
- **Manual updates** — checks updates only when requested
- **Fuzzy search** — filter skills by name or description
Expand Down Expand Up @@ -40,6 +40,21 @@ pnpm build
node packages/tui/dist/skillpack.js
```

## Development

Contributing requires a Vite+-supported Node.js release (20.19+, 22.18+, or 24.11+) and pnpm 11. The published CLI still supports Node.js 18 and newer.

```bash
pnpm check # format check, type-aware lint, and type check
pnpm format # format supported source and config files
pnpm lint # type-aware lint and type check
pnpm test # run all package tests once
pnpm build # package Core and the TUI
pnpm dev # build Core, then watch the TUI package
```

CI uses the explicit `pnpm check:ci`, `pnpm test:ci`, and `pnpm build:ci` entry points. The final command also verifies the Core exports, CLI shebang and executable bit, ESM syntax, source-map policy, and npm package contents.

## Quick Start

Launch `skillpack` to see discovered skills grouped into Skill Groups with provider status and Health Signals. Use `↑↓` arrow keys to navigate, `Tab` / `Shift+Tab` to filter by provider (All, Codex, Claude, Global), and `/` to search.
Expand Down Expand Up @@ -129,7 +144,7 @@ Skillpack stores its configuration at `~/.config/skillpack/config.json`. On firs
}
```

`usage.importConsent` defaults to `false`. When enabled from the TUI, Skillpack imports supported provider artifacts into derived JSONL records under `~/.local/share/skillpack/usage/`. The current importer supports Codex session artifacts. Claude appears in the Usage view for coverage, but Claude usage import is not supported yet.
`usage.importConsent` defaults to `false`. When enabled from the TUI, Skillpack imports supported Codex and Claude artifacts into derived JSONL records under `~/.local/share/skillpack/usage/`. Claude usage is derived from structured `Skill` tool calls that match user-level or plugin-provided skills in configured Claude Scan Roots. Built-In Skills and Claude Project Skills are not counted in the current Claude adapter.

The Settings view shows both Scan Roots and Usage Artifact Roots so you can verify which local directories Skillpack will inspect.

Expand Down
15 changes: 14 additions & 1 deletion docs/adr/0039-derive-skill-usage-from-provider-sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,17 @@

Skillpack will derive Skill Usage by reading provider-owned session artifacts read-only, not by adding runtime hooks or instrumentation to agent platforms. This keeps Skillpack aligned with provider-native state: providers remain the source of truth for Skill Invocations, while Skillpack normalizes their session evidence into usage history and aggregates.

The first supported adapter is Codex. It reads the real Codex session JSONL event streams under `~/.codex/sessions/**/*.jsonl` and `~/.codex/archived_sessions/*.jsonl`, using `session_meta` and `turn_context` for session/turn/cwd context and `response_item` `function_call` tool calls for evidence that the agent directly read a `.../skills/<skill>/SKILL.md` file. Codex usage is global across configured artifact roots and is not scoped to Skillpack's startup cwd. `~/.codex/logs_2.sqlite` is not a counting source because it includes rendered skill metadata and can confuse available skills with invoked skills. Claude remains unsupported for Skill Usage until its real session artifact format is implemented conservatively.
The first supported adapter is Codex. It reads the real Codex session JSONL event streams under `~/.codex/sessions/**/*.jsonl` and `~/.codex/archived_sessions/*.jsonl`, using `session_meta` and `turn_context` for session/turn/cwd context. Direct `.../skills/<skill>/SKILL.md` reads are invocation evidence in both historical `response_item` `function_call` / `exec_command` envelopes and current orchestrated `response_item` `custom_tool_call` / `exec` envelopes. Both envelopes remain supported because importer-version rebuilds reprocess historical sessions; the envelope is provider evidence, not part of Skill Invocation identity. Codex usage is global across configured artifact roots and is not scoped to Skillpack's startup cwd. `~/.codex/logs_2.sqlite` is not a counting source because it includes rendered skill metadata and can confuse available skills with invoked skills.

Representative provider shapes make the compatibility boundary explicit:

```json
{"type":"response_item","payload":{"type":"function_call","name":"exec_command","call_id":"legacy-call","arguments":"{\"cmd\":\"cat .agents/skills/tdd/SKILL.md\"}"}}
{"type":"response_item","payload":{"type":"custom_tool_call","name":"exec","call_id":"current-call","input":"await tools.exec_command({ cmd: \"cat .agents/skills/tdd/SKILL.md\" });"}}
```

Orchestrated `exec` input is executable-looking provider evidence, but Skillpack treats it as untrusted data. The core uses Acorn only to parse that input into an ESTree syntax tree, then a Skillpack-owned evaluator resolves the observed static subset: literals and static string composition, statically initialized `const` string/object/array bindings whose evaluated values are not observably mutated, property reads and shorthand properties, multiple direct calls, statically known array `.map()` and `for...of` iterations, and `Promise.all()` wrappers. Unshadowed Codex orchestration helpers (`text`, `image`, `generatedImage`, `store`, `load`, `notify`, and `yield_control`) are transparent only for ordered argument inspection; unknown calls, spread arguments, shadowed helpers or built-ins, implicit object coercion, thenable assimilation, observable mutation, unsupported control flow, and otherwise unprovable behavior stop the remaining path. Existing direct-read matching runs only after static evaluation has produced a concrete shell command, and ignores paths after an unquoted shell-comment marker. A raw-source `SKILL.md` check may inform diagnostics but must not gate parsing, because supported static composition can construct that token without containing it contiguously in source. Skillpack never evaluates or executes session input. Acorn does not identify skills, run commands, or provide general JavaScript interpretation; mutable bindings, runtime function results, dynamic properties, and otherwise unprovable commands are not Skill Invocation evidence.

Static interpretation is bounded per orchestrated envelope: at most 1,000,000 source code units, 10,000 evaluation steps, 10,000 static iterations, 100 nested evaluation frames, 256 resolved commands, 256 extracted Skill paths per command, 1,000,000 code units in one derived string, and 10,000,000 cumulatively retained derived-string code units. String limits are checked before concatenation so exponential static composition cannot allocate an oversized intermediate value. Exceeding any budget fails closed by discarding partial command evidence from that envelope. The adapter emits high-signal Usage Import Diagnostics only when parsing fails with potential Skill evidence, a real nested `tools.exec_command` with potential Skill evidence cannot resolve its command or working directory inside the supported static subset, or a budget is exceeded while fixed Skill evidence is present. Findings are aggregated by artifact and reason (`invalid-javascript`, `unresolved-command`, `unresolved-workdir`, or `analysis-budget-exceeded`) without retaining source JavaScript, commands, or output. Unrelated `exec` programs and dynamic calls with no Skill evidence are ignored so ordinary orchestration does not create diagnostic noise.

The Claude adapter reads known main-session and subagent JSONL transcript paths under configured Claude artifact roots and treats assistant `tool_use` blocks with `name === "Skill"` as invocation evidence. It follows JSONL file symlinks, deduplicates real artifacts, and does not parse unrelated JSONL path shapes or loader-log text as invocation evidence. Malformed lines are skipped with aggregated diagnostics, while valid auto-compact summaries are ignored because they cannot restore earlier structured calls.
Loading
Loading