Skip to content

feat: ops-traces seed skill + ns-agent-admin connector template - #21

Merged
Rippy1911 merged 1 commit into
mainfrom
feat/ops-traces-skill
Aug 3, 2026
Merged

feat: ops-traces seed skill + ns-agent-admin connector template#21
Rippy1911 merged 1 commit into
mainfrom
feat/ops-traces-skill

Conversation

@Rippy1911

Copy link
Copy Markdown
Owner

Summary

  • ops-traces seed skill (packages/core/src/skills/store.ts) — read-only ops playbook for the ns-agent service: setup of the admin connector, GET /v1/ops/summary?days=N as the primary telemetry call, /v1/runs + /v1/conversations for detail, read-only/no-secrets/summarize rules, and plainly-stated known gaps (ns-exec audit log has no read endpoint, Mac vault audit is CLI-only, no Discord REST tools — ask Combo ns_ops_report).
  • ns-agent-admin REST connector template (connectors/templates.ts, exported from index.ts) — https://agent.nextsolutions.studio, Bearer via vault label ns_agent_admin_key, modeled exactly on the ns-exec template.
  • WORK vault recipe (vault/recipes.ts) seeds the connector + adds ns_agent_admin_key to the secret-label checklist.
  • Pack wiring: SKILL_NAME_PACK_HINTS["ops-traces"] = TOOL_PACKS.rest (gating.ts) and PACK_SKILL_NAMES.rest = ["combo-rest", "ops-traces"] (promptCatalog.ts). PACK_SKILL_NAMES values now accept string | string[]; single-string packs render exactly as before (covered by the existing scrape → combo-scrape test).
  • SEED_REVISIONv1.9.2 so existing installs refresh seeds. No package.json version bumps.
  • No new agent tools — the playbook drives the existing rest_request against the connector.

Acceptance evidence (local gates, same as CI)

  • pnpm typecheck (tsc --build): green (exit 0, no errors)
  • pnpm test (vitest run): 104 test files passed, 649 tests passed / 6 skipped / 0 failed (skips are the env-gated llm-eval suite, pre-existing)
  • Tests updated: store.test.ts seed list 18 → 19 (ops-traces added, rest-pack toolHints asserted); docs/SKILLS.md table row added (OOTB seeds 17 → 18)
  • Branch created off origin/main; unrelated PR feat(browser): within row-scoping, excludeSelector, list_form_fields, stuck-loop guard (1.9.0) #20 branch untouched

Deployment note

Pairs with an ns-infra PR adding GET /v1/ops/summary to the ns-agent service — not yet deployed. Until that lands, rest_request on ns-agent-admin will 404; this is expected and documented in the playbook body. /v1/runs and /v1/conversations should work as soon as the ops key is in the vault label.

Test plan

  • CI green (typecheck, unit tests, Chrome/Firefox builds, web-ext lint, e2e)
  • After ns-infra deploy: reload extension, skill_read ops-traces, ask "conversations yesterday" → summary from /v1/ops/summary
  • 401 path: without the key, agent asks operator to paste the ops key into vault label ns_agent_admin_key (no value in chat)

Read-only ops telemetry path for the ns-agent service: a seed playbook
(ops-traces) that unlocks the existing rest pack, plus an ns-agent-admin
REST connector template (agent.nextsolutions.studio, vault label
ns_agent_admin_key) seeded by the work vault recipe.

- SKILL_NAME_PACK_HINTS + PACK_SKILL_NAMES map ops-traces to the rest
  pack (PACK_SKILL_NAMES values now allow a list; scrape/media/… output
  unchanged)
- SEED_REVISION bumped v1.7.9 -> v1.9.2 so existing installs refresh
- No new agent tools; playbook drives rest_request only
@nextsolutions-studio

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🏅 Score: 88
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Stale seed count

The OOTB seeds heading still says 18 after adding ops-traces, while store.test.ts expects nineteen seed names. Operators reading the skills doc will under-count installed playbooks versus what seedSkillDefinitions() actually ships.

## OOTB seeds (18)

| Skill | Unlocks | Notes |
|-------|---------|--------|
| `combo-scrape` | scrape pack | Catalog/PDP/login |
| `combo-rest` | rest/mcp pack | Saved connectors |
| `ops-traces` | rest/mcp pack | ns-agent ops telemetry (`/v1/ops/summary`); pairs with `ns-agent-admin` connector |
Query params undocumented

Template tools only declare bare paths (/v1/ops/summary, /v1/runs) while the skill playbook relies on query:{ days:"1" } and ?limit=50. If rest_request only merges query args when the tool schema advertises them, primary telemetry calls can omit the window/limit and return the wrong default slice—or the agent may invent path strings instead of using the named tools.

tools: [
  {
    name: "ops_summary",
    method: "GET",
    path: "/v1/ops/summary",
    description: "Ops summary ?days=N (conversations, runs, tool usage, gate events, tokens/cost)",
  },
  {
    name: "runs",
    method: "GET",
    path: "/v1/runs",
    description: "Recent runs ?limit= (model, latency_ms, tool_calls per run)",
  },
  {
    name: "conversations",
    method: "GET",
    path: "/v1/conversations",
    description: "Conversation list",
  },

@nextsolutions-studio

Copy link
Copy Markdown

PR Code Suggestions ✨

No code suggestions found for the PR.

@Rippy1911
Rippy1911 merged commit 6e2d68a into main Aug 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants