Feat/eng 2971#721
Conversation
…sk types Adds curate-tool-mode, query-tool-mode, dream-scan, dream-finalize to TaskTypes + TASK_TYPE_VALUES. After ENG-2925's rename the daemon dispatches these four types but the analytics enum still predates v4.0 — task_created / task_completed / task_failed silently rejected every tool-mode emit at the Zod boundary. - TaskTypes keeps the legacy 'curate' / 'query' / 'dream' values for back-compat with any constructor still building those payloads - per-event schemas (task_created / task_completed / task_failed) pick up the new types automatically via z.enum(TASK_TYPE_VALUES) - M12 per-flavor schemas (curate_run_completed / query_completed) still hardcode their own literals and continue to reject tool-mode types here — M14.2 migrates them to the canonical enum as a follow-up TDD: - new task-types tests assert TaskCreated/Completed/Failed accept all four new types - regression tests pin the M12 schemas' continued rejection so M14.2 has a clear flip point
…sk_type to TASK_TYPE_VALUES
Migrates the two M12 per-flavor schemas from hardcoded literal task_type
values to the canonical TASK_TYPE_VALUES enum so v4.0 tool-mode tasks
round-trip the wire boundary instead of being silently Zod-rejected
inside AnalyticsHook.
- curate_run_completed: z.enum(['curate', 'curate-folder']) →
z.enum(TASK_TYPE_VALUES). curate-tool-mode payloads now validate.
- query_completed: z.literal('query') → z.enum(TASK_TYPE_VALUES).
query-tool-mode payloads now validate.
The schemas no longer structurally constrain task_type to the curate
or query family; the hook is trusted to only emit each event for the
right flavor. Docblocks call out the widening for the next maintainer.
TDD:
- curate-run-completed.test asserts curate-tool-mode + legacy values
both succeed; an unknown task_type still rejects
- query-completed.test mirrors the same coverage for query-tool-mode
- task-types.test M14.1 regression assertions flipped from rejection
to acceptance for the M12 schemas; M14.1 docblock comments updated
Without this, M14.3's hook code would land but the M12 emits that fire
alongside the new generic task_* emits would silently disappear on
every tool-mode task — that's the bug operators noticed in Mixpanel.
…ed in AnalyticsHook Adds the three generic funnel-event emits described in M14's customer ask. Every daemon task (curate / curate-folder / curate-tool-mode (aliased from curate-html-direct) / query / query-tool-mode / dream-scan / dream-finalize / search) now produces, in order: task_created onTaskCreate (funnel entry — unconditional) ...optional per-op + M12 per-flavor terminal emit task_completed onTaskCompleted (terminal-event-last on success) task_failed onTaskError / onTaskCancelled (terminal-event-last) Three coupled changes shipped together so the wire stays consistent: 1. analytics-hook.ts grows three emits and a `toAnalyticsTaskType` alias-translator. The daemon still dispatches the pre-ENG-2925 name `curate-html-direct`; analytics canonicalises it to the post-rename `curate-tool-mode` so the wire enum matches TASK_TYPE_VALUES. Once ENG-2925 lands, the alias becomes a no-op identity. 2. CURATE_TASK_TYPES + QUERY_TASK_TYPES gain the tool-mode names so M12 per-flavor state init kicks in for tool-mode curates / queries. M12 counters stay all-zero on tool-mode today (no LLM tool calls fire) — that's a separate follow-up (FU-1 in plans/analytics-m14/follow-ups). 3. Both M12 payload builders route `task_type` through the same alias so curate-tool-mode tasks emit `task_type='curate-tool-mode'` on the curate_run_completed / query_completed events too. TDD: - new analytics-hook-m14.test.ts: 15 simulation tests covering every task type, the curate-html-direct → curate-tool-mode alias, has_files / has_folder semantics, both terminal paths, and the ordering invariant - existing analytics-hook.test.ts updated to filter out the new generic emits via `filterM12()` so M12-focused assertions keep their intent - integration stress tests updated for the new 13-event sequence (task_created → ops → curate_run_completed → task_completed) - repo full suite: 9121 passing, 0 failing
…ords/tags, structured related_paths
Tightens the curate / query M12 payloads per code review:
curate_operation_applied:
- rename absolute_path → relative_path (project-relative via path.relative
against the task's projectPath; falls back to identity when projectPath
is unset so search tasks at the daemon root still emit a usable string)
- keywords / tags promoted from optional to required arrays (default [])
so the wire shape stays uniform regardless of frontmatter read success
query_completed.read_paths_with_metadata:
- same absolute_path → relative_path rename
- same keywords / tags promotion (required arrays, default [])
- flat related: string[] → structured related_paths: [{relative_path,
keywords, tags}] so each linked topic carries its own metadata slot.
Keywords/tags default to [] until a future FU cascade-reads each
linked file's frontmatter — the wire shape doesn't need to change
when that lands.
Hook implementation:
- new `toRelativePath(filePath, projectPath)` helper using node:path.relative
- CurateTaskAnalyticsState stores projectPath captured at onTaskCreate so
per-op emits can relativize without threading task through processToolResult
- all four payload sites (curate-op, curate-run-completed, query-completed
read_paths_with_metadata) route through the helper
Inspection test added at test/unit/.../analytics-hook-toolmode-inspection
that pretty-prints every event + payload for curate-tool-mode / query
/ query-tool-mode flows — gives PMs a single place to verify the wire
shape end-to-end.
Privacy win: relative paths drop the /Users/{name} prefix from every
file-touched event, keeping host-identifiable PII off the analytics
wire while still letting PMs reason about which file inside a project
an operation touched.
Tests: full repo 9132 passing.
|
Claude finished @cuongdo-byterover's task in 6m 12s —— View job Review summaryOverall score: 3.5 / 5 — Solid M14 follow-up work with good test coverage for the new What's strong
Issues raised inline
Things I didn't flag inline but worth mentioning
Checklist
|
| function toAnalyticsTaskType(daemonType: string): TaskType { | ||
| if (daemonType === 'curate-html-direct') return TaskTypes.CURATE_TOOL_MODE | ||
| return daemonType as TaskType | ||
| } |
There was a problem hiding this comment.
suggestion (concern, non-blocking): the daemonType as TaskType cast bypasses validation. If a new daemon TaskTypeSchema value is added (e.g. a future vc-commit) but not mirrored in TASK_TYPE_VALUES, this silently emits a payload that will be rejected by the per-event schemas downstream (since they use z.enum(TASK_TYPE_VALUES)). Per CLAUDE.md: "Avoid as Type assertions — use type guards or proper typing instead."
A safer pattern:
const TASK_TYPE_SET: ReadonlySet<string> = new Set(TASK_TYPE_VALUES)
function toAnalyticsTaskType(daemonType: string): TaskType | undefined {
if (daemonType === 'curate-html-direct') return TaskTypes.CURATE_TOOL_MODE
return TASK_TYPE_SET.has(daemonType) ? (daemonType as TaskType) : undefined
}…and have emit skip the call when task_type is undefined (with a processLog for visibility). Today the daemon's TaskTypeSchema only diverges by curate-html-direct, so the practical fallout is bounded — but the comment promises this becomes a "no-op identity" once rename lands, while in practice it would still need the safety net.
| function toRelativePath(filePath: string, projectPath?: string): string { | ||
| if (!projectPath) return filePath | ||
| const rel = relativePath(projectPath, filePath) | ||
| // `path.relative` returns '' when paths are identical — defensively | ||
| // surface a leaf token rather than emit a zero-length wire string that | ||
| // would fail `z.string().min(1)`. | ||
| return rel === '' ? '.' : rel | ||
| } |
There was a problem hiding this comment.
issue (concern): The PII-removal claim is only partly true. path.relative('/project', '/Users/dev/foo.md') returns '../../Users/dev/foo.md' — the full host path is still on the wire, just rebased. The tests at analytics-hook.test.ts:149 (relative_path === '../a.md') and analytics-hook.test.ts:336 (['../a.md', '../b.md', '../c.md']) confirm this — these payloads still leak the parent dir info.
In real curate runs the affected files almost always live inside projectPath, so this only bites pathological inputs. But the comment header on the schema (curate-operation-applied.ts:13) sells this change as "privacy + portability across hosts" — the privacy claim doesn't hold when a path can escape the project root.
Two options worth considering:
- Clamp escapes: if
rel.startsWith('..'), emit a marker like<outside-project>(ornull) instead of the host-leaking string. - Document the residual leak explicitly in the schema header.
Also: the '.' fallback emits a directory marker for a file operation — surprising shape for downstream consumers, but currently unreachable (filePath always points to a file).
| export const CurateOperationAppliedSchema = z | ||
| .object({ | ||
| absolute_path: z.string().min(1), | ||
| confidence: z.enum(['high', 'low']).optional(), | ||
| impact: z.enum(['high', 'low']).optional(), | ||
| keywords: z.array(z.string().max(256)).max(50).optional(), | ||
| keywords: z.array(z.string().max(256)).max(50), | ||
| knowledge_path: z.string().min(1), | ||
| needs_review: z.boolean(), | ||
| operation_type: z.enum(['ADD', 'UPDATE', 'DELETE', 'MERGE', 'UPSERT']), | ||
| related: z.array(z.string().max(256)).max(50).optional(), | ||
| tags: z.array(z.string().max(256)).max(50).optional(), | ||
| relative_path: z.string().min(1), | ||
| tags: z.array(z.string().max(256)).max(50), | ||
| task_id: z.string().min(1), | ||
| }) | ||
| .strict() |
There was a problem hiding this comment.
issue (blocking) — wire-format breaking change:
This rev does three things on the same emit:
- renames
absolute_path→relative_path - promotes
keywords/tagsfrom optional → required - keeps
.strict()so unknown extras (incl. the oldabsolute_path) are rejected
If the analytics backend (telemetry-dev / prod) was previously consuming absolute_path or treating keywords / tags as nullable, this PR will start dropping rows post-deploy until the backend is updated in lockstep. The PR description doesn't mention coordination with the telemetry team.
Please confirm in the PR body:
- Backend has been (or will be) updated to read
relative_pathand requirekeywords/tags. - Whether any backfill is needed for existing dashboards.
- Roll-forward / roll-back plan if the deploy ordering slips.
The same review tightening applies to query-completed.ts (read_paths_with_metadata[*] entries).
| const dumpEvents = (label: string, trackStub: sinon.SinonStub): void => { | ||
| console.log(`\n┌─ ${label} ${'─'.repeat(Math.max(0, 70 - label.length))}`) | ||
| for (const [i, call] of trackStub.getCalls().entries()) { | ||
| const eventName = call.args[0] as string | ||
| const props = call.args[1] as Record<string, unknown> | ||
| console.log(`│ [${i}] ${eventName}`) | ||
| console.log(`│ ${JSON.stringify(props, null, 2).replaceAll('\n', '\n│ ')}`) | ||
| } | ||
|
|
||
| console.log(`└${'─'.repeat(72)}\n`) | ||
| } |
There was a problem hiding this comment.
suggestion (nitpick): dumpEvents writes 8+ console.log lines per test, and the file calls it from 6 of the 7 test cases. Per CLAUDE.md: "Suppress console logging in tests to keep output clean."
If the dumps are scaffolding for manual review against a fixed event shape that you'll keep eyeballing, that's fine — but please mark the suite as inspection-only (e.g. .skip by default + a script in package.json to run it), or guard dumpEvents on process.env.BRV_DUMP_ANALYTICS_EVENTS. As-is this adds ~200 lines of noise to every npm test run.
If the dumps were only to validate the M14 wire shape during PR review and aren't intended long-term, consider deleting dumpEvents and keeping only the expect(...) assertions that already cover the same ground in analytics-hook-m14.test.ts.
| keywords: frontmatter.keywords ?? [], | ||
| knowledge_path: op.path, | ||
| needs_review: op.needsReview ?? false, | ||
| operation_type: op.type, | ||
| ...(frontmatter.related ? {related: frontmatter.related} : {}), | ||
| ...(frontmatter.tags ? {tags: frontmatter.tags} : {}), | ||
| relative_path: toRelativePath(op.filePath, state.projectPath), | ||
| tags: frontmatter.tags ?? [], |
There was a problem hiding this comment.
issue (concern): When op.filePath is missing, processToolResult continues out at line 498 — but only AFTER the switch at lines 464–490 has already bumped the type counter. The aggregate curate_run_completed.operations_added (etc.) will count an op for which no curate_operation_applied event was ever emitted.
A downstream dashboard that reconstructs counts from curate_operation_applied (e.g. COUNT(*) GROUP BY operation_type per task_id) will diverge from the aggregate counter on curate_run_completed.
Two reasonable fixes:
- Skip the counter bump too: move the
if (!op.filePath) continuebefore theswitch(~line 463). - Emit with a sentinel like
relative_path: '<unknown>'so the per-op stream stays complete.
The comment at line 497 explicitly says these ops are rare, so (1) is the simplest reconciliation.
Summary
Type of change
Scope (select all touched areas)
Linked issues
Root cause (bug fixes only, otherwise write
N/A)Test plan
User-visible changes
List user-visible changes (including defaults, config, or CLI output).
If none, write
None.Evidence
Attach at least one:
Checklist
npm test)npm run lint)npm run typecheck)npm run build)mainRisks and mitigations
List real risks for this PR. If none, write
None.