Challenge submission: add trace metrics toolkit

README.md

@@ -14,7 +14,17 @@ To avoid the overhead of launching a separate tsserver, it currently queries the
 
 The `Tracer: tsc trace` command can be run to gather accurate timings.  As with real-time metrics, these create diagnostics in the editor open files. It also opens an interface to browse trace files. Editor and UI commands enable navigating between locations in the editor and the trace.
 
-If the trace is run with a version of `tsc` that includes timestamps in `types.json`, type count metrics are also displayed. A PR to include these timestamps or tooling to automatically patch them in are works in progress.  See https://github.com/typeholes/TypeScript/tree/trace-data-5-4 for a version you can build yourself.
+Each trace run writes a `metrics.json` file next to the generated trace files. The artifact records the command, working directory, trace directory, start/end time, wall time, exit code, output summaries, discovered trace JSON files, parse status, and any parsed `--extendedDiagnostics` counters. This makes traces easier to compare across branches and machines without reopening the full trace viewer.
+
+Type count metrics are displayed when a timestamped `types.json` file is available. Stock TypeScript `types.json` files are also parsed safely, so a normal `npx tsc --generateTrace` run still reports the total type count even when per-span type attribution is unavailable.
+
+### Focused traces and comparisons
+
+- `Tracer: Trace current file` runs the configured trace command from the active TypeScript file context and saves the run under a workspace-relative trace name.
+- `Tracer: Compare trace metrics` opens two `metrics.json` files and renders a Markdown before/after report for wall time, exit code, trace file coverage, parse status, compiler diagnostics, and output deltas.
+- `Tracer: Open trace viewer` remains the main path for inspecting trace trees and jumping from expensive spans back to source.
+
+For a short reproducible demo path, see [docs/challenge-demo.md](./docs/challenge-demo.md). For artifact examples, see [docs/challenge-evidence.md](./docs/challenge-evidence.md). For challenge review and submission packaging, see [docs/tsperf-submission.md](./docs/tsperf-submission.md).
 
 ### Use in mono repos
 

docs/challenge-demo.md

@@ -0,0 +1,43 @@
+# TSPerf Challenge Demo
+
+This is a short demo path for showing that Type Complexity Tracer now reports TypeScript type complexity and time-to-load evidence from repeatable trace runs.
+
+## Setup
+
+1. Open a TypeScript workspace in VS Code.
+2. Install dependencies in this extension repo with `pnpm install`.
+3. Run `pnpm build`.
+4. Start the extension in an Extension Development Host.
+
+## Demo Flow
+
+1. Run `Tracer: tsc trace` against the workspace.
+2. Open the generated trace directory and show `metrics.json`.
+3. Highlight these fields:
+   - `wallTimeMs`
+   - `exitCode`
+   - `traceJsonFiles`
+   - `parse.status`
+   - `extendedDiagnostics.types`
+   - `extendedDiagnostics.instantiations`
+   - `extendedDiagnostics.checkTimeMs`
+   - `extendedDiagnostics.totalTimeMs`
+4. Open the trace viewer and inspect the trace tree.
+5. Use `Tracer: Trace current file` from a TypeScript source file to create a focused run with a predictable workspace-relative save name.
+6. Run a second trace after changing a type-heavy code path.
+7. Run `Tracer: Compare trace metrics`, select the baseline and comparison `metrics.json` files, and show the generated Markdown report.
+
+For example artifact shapes, see [challenge-evidence.md](./challenge-evidence.md).
+
+## Submission Angle
+
+The branch demonstrates a complete measurement loop:
+
+- run a TypeScript compiler trace from VS Code
+- record a portable `metrics.json` artifact
+- tolerate stock TypeScript trace output
+- parse `--extendedDiagnostics` counters
+- inspect trace tree details in the UI
+- compare before/after runs in a readable report
+
+That maps to the challenge requirement for a VS Code plugin that shows TypeScript type complexity and load-time cost.

docs/challenge-evidence.md

@@ -0,0 +1,95 @@
+# TSPerf Challenge Evidence
+
+This page gives reviewers a quick look at the artifacts produced by the challenge branch before they run the extension locally.
+
+## Metrics Artifact
+
+Each `Tracer: tsc trace` run writes a `metrics.json` file next to the TypeScript trace output. A representative artifact has this shape:
+
+```json
+{
+  "version": 1,
+  "command": "npx tsc --noEmit --generateTrace .trace/tsperf-main",
+  "cwd": "/workspace/example-project",
+  "traceDir": "/workspace/example-project/.trace/tsperf-main",
+  "startedAt": "2026-05-20T09:42:11.000Z",
+  "endedAt": "2026-05-20T09:42:14.250Z",
+  "wallTimeMs": 3250,
+  "exitCode": 0,
+  "stdout": {
+    "bytes": 0,
+    "truncated": false,
+    "text": ""
+  },
+  "stderr": {
+    "bytes": 382,
+    "truncated": false,
+    "text": "Files: 42\nTypes: 1400\nInstantiations: 700\nMemory used: 78000K\nCheck time: 0.32s\nTotal time: 1.10s\n"
+  },
+  "extendedDiagnostics": {
+    "types": 1400,
+    "instantiations": 700,
+    "memoryUsedKb": 78000,
+    "checkTimeMs": 320,
+    "totalTimeMs": 1100
+  },
+  "traceJsonFiles": [
+    {
+      "fileName": "trace.json",
+      "bytes": 183224
+    },
+    {
+      "fileName": "types.json",
+      "bytes": 92130
+    }
+  ],
+  "parse": {
+    "status": "ok"
+  }
+}
+```
+
+The high-signal TSPerf fields are:
+
+- `wallTimeMs`: end-to-end trace command duration.
+- `traceJsonFiles`: whether trace output was actually produced.
+- `parse.status`: whether generated JSON can be parsed.
+- `extendedDiagnostics.types`: total TypeScript type count.
+- `extendedDiagnostics.instantiations`: generic/type instantiation pressure.
+- `extendedDiagnostics.checkTimeMs`: checker cost.
+- `extendedDiagnostics.totalTimeMs`: total compiler time.
+
+## Comparison Report
+
+`Tracer: Compare trace metrics` turns two `metrics.json` files into a Markdown delta report:
+
+```markdown
+# Trace Metrics Comparison
+
+Exit code: 0 -> 0
+Parse status: ok -> ok
+
+| Metric | Before | After | Delta |
+| --- | ---: | ---: | ---: |
+| Wall time ms | 3250 | 2785 | -465 |
+| Trace JSON files | 2 | 2 | 0 |
+| Types | 1400 | 1100 | -300 |
+| Instantiations | 700 | 450 | -250 |
+| Memory used KB | 78000 | 69000 | -9000 |
+| Parse time ms | 120 | 110 | -10 |
+| Bind time ms | 80 | 78 | -2 |
+| Check time ms | 320 | 210 | -110 |
+| Emit time ms | 0 | 0 | 0 |
+| Total time ms | 1100 | 870 | -230 |
+```
+
+This is the reviewable loop for the challenge:
+
+1. Run a baseline trace.
+2. Change a type-heavy branch, file, or dependency version.
+3. Run a comparison trace.
+4. Use the generated report to see whether type count, instantiations, memory, check time, and total time improved or regressed.
+
+## Focused Current-File Trace
+
+`Tracer: Trace current file` gives reviewers a quicker path for a smaller trace. From a TypeScript editor, the command creates a workspace-relative trace name for the active file and runs the configured trace command in that context. That makes it easier to test a single type-heavy area without manually naming trace directories.

docs/tsperf-submission.md

@@ -0,0 +1,66 @@
+# TSPerf Submission Package
+
+This package summarizes the challenge-ready branch for Algora's TSPerf Type Challenge.
+
+## Candidate
+
+- Fork: https://github.com/jianmosier/tracer
+- Branch: `codex/tsperf-team-stack`
+- Challenge fit: VS Code extension support for measuring TypeScript type complexity and load-time cost from repeatable compiler traces.
+
+## Reviewer Path
+
+1. Install dependencies with `pnpm install`.
+2. Run validation:
+   - `pnpm lint`
+   - `pnpm typecheck`
+   - `pnpm build`
+   - `pnpm exec vitest run`
+3. Start the extension in an Extension Development Host.
+4. Open a TypeScript workspace.
+5. Run `Tracer: tsc trace`.
+6. Inspect the generated `metrics.json` next to the trace output.
+7. Run `Tracer: Trace current file` from a TypeScript editor to collect a focused run.
+8. Run `Tracer: Compare trace metrics` on two `metrics.json` files and inspect the Markdown delta report.
+
+## What To Evaluate
+
+- `metrics.json` records trace command, cwd, trace directory, timestamps, wall time, exit code, output summaries, trace JSON discovery, parse status, and parsed `--extendedDiagnostics` counters.
+- `docs/challenge-evidence.md` shows representative `metrics.json` and comparison-report output for quick review.
+- Stock TypeScript `types.json` files are handled without requiring tracer-specific timestamp fields.
+- Extended diagnostics expose high-signal type-system counters including `types`, `instantiations`, memory, parse time, bind time, check time, emit time, and total time.
+- The comparison command turns two trace runs into a compact before/after report suitable for branch, commit, or code-path comparisons.
+- The focused current-file trace command gives reviewers a fast way to collect a smaller trace from the active TypeScript source file.
+
+## Suggested Submission Text
+
+This branch turns Type Complexity Tracer into a repeatable TSPerf measurement loop inside VS Code. It adds a portable `metrics.json` artifact for each compiler trace, parses stock TypeScript trace/type output, captures `--extendedDiagnostics` counters, and provides a `Tracer: Compare trace metrics` command that renders before/after deltas for wall time, type counts, instantiations, memory, compiler timing, parse status, and output changes.
+
+The key workflow is:
+
+1. Run `Tracer: tsc trace`.
+2. Review the generated `metrics.json` artifact.
+3. Make a type-heavy code change or switch branches.
+4. Run another trace.
+5. Use `Tracer: Compare trace metrics` to produce a Markdown comparison report.
+
+This maps directly to the challenge requirement: an open-source MIT VS Code plugin that shows TypeScript type complexity and time-to-load evidence.
+
+## Validation Evidence
+
+Record the latest command output in the submission after running validation locally:
+
+```text
+pnpm lint
+pnpm typecheck
+pnpm build
+pnpm exec vitest run
+```
+
+Optional package smoke test on Node 22:
+
+```text
+PATH="/opt/homebrew/opt/node@22/bin:$PATH" pnpm package
+```
+
+Node note: packaging can fail on Node 25 because a transitive package reads `SlowBuffer.prototype`; Node 22 packages successfully.

package.json

@@ -297,6 +297,16 @@
         "command": "tsperf.tracer.runTrace",
         "category": "Tracer"
       },
+      {
+        "title": "Trace current file",
+        "command": "tsperf.tracer.runTraceActiveFile",
+        "category": "Tracer"
+      },
+      {
+        "title": "Compare trace metrics",
+        "command": "tsperf.tracer.compareTraceMetrics",
+        "category": "Tracer"
+      },
       {
         "title": "Send Trace to Trace Viewer",
         "command": "tsperf.tracer.sendTrace",

scripts/generate-contributes.ts

@@ -49,6 +49,12 @@ const commandRecord: Record<CommandId, Command> = {
       explorerContext: 'resourceFilename =~ /./',
     },
   },
+  'tsperf.tracer.runTraceActiveFile': {
+    title: 'Trace current file',
+  },
+  'tsperf.tracer.compareTraceMetrics': {
+    title: 'Compare trace metrics',
+  },
   'tsperf.tracer.sendTrace': {
     title: 'Send Trace to Trace Viewer',
     when: {

shared/src/traceData.ts

@@ -6,7 +6,7 @@ export const typeLine = z.object({
   intrinsicName: z.string().optional(),
   recursionId: z.number().optional(),
   flags: z.array(z.string()).optional(),
-  ts: z.number(),
+  ts: z.number().optional(),
   dur: z.number().optional(),
   display: z.string().optional(),
 })
@@ -39,3 +39,29 @@ export type DataLine = TraceLine | TypeLine
 
 export type TraceData = z.infer<typeof traceData>
 export const traceData = z.array(typeLine.or(traceLine))
+
+export interface TraceDataSummary {
+  totalTypes: number
+  timestampedTypes: number
+  untimestampedTypes: number
+  parseWarning?: string
+}
+
+export function hasTypeTimestamp(line: TypeLine): line is TypeLine & { ts: number } {
+  return typeof line.ts === 'number'
+}
+
+export function getTraceDataSummary(data: TraceData): TraceDataSummary {
+  const typeLines = data.filter((line): line is TypeLine => 'id' in line)
+  const timestampedTypes = typeLines.filter(hasTypeTimestamp).length
+  const untimestampedTypes = typeLines.length - timestampedTypes
+
+  return {
+    totalTypes: typeLines.length,
+    timestampedTypes,
+    untimestampedTypes,
+    parseWarning: untimestampedTypes > 0
+      ? `${untimestampedTypes} type entr${untimestampedTypes === 1 ? 'y is' : 'ies are'} missing timestamps and cannot be attributed to trace spans.`
+      : undefined,
+  }
+}