Skip to content

Add ecosystem fingerprinting to enrich session analysis #461

Description

@willwashburn

Problem

burn knows which tools were called in a session (via the classifier + TOOL_ALIASES) but has no reified model of the user's stack: language ecosystem, test runner, formatter/linter, package manager, VCS, shell, MCP servers in use, frameworks. analyze/quality.rs does light harness detection, and patterns/hotspots see tool calls, but nothing rolls up to "this session was a Python project using pytest + ruff under uv, on git/zsh, with the github MCP server installed."

This matters for two reasons:

  1. Rollups by stack become possible. "Median cost per session for Rust projects" or "retry-loop rate when using pytest vs. cargo test" can't be answered today.
  2. Future remediation surfaces need it. Pattern detection often has stack-specific fixes (a ruff bloat fix differs from a clippy bloat fix). Knowing the stack lets findings carry stack-appropriate guidance.

Relevant files

  • crates/relayburn-sdk/src/reader/classifier.rs — existing tool/category classifier; ecosystem detection should consume the same event stream
  • crates/relayburn-sdk/src/reader/types.rs — where new fingerprint types likely live
  • crates/relayburn-sdk/src/analyze/quality.rs — current basic harness/outcome detection
  • crates/relayburn-sdk/src/analyze/mod.rs — new ecosystem module slot
  • crates/relayburn-sdk/src/query_verbs.rs — public query surface
  • crates/relayburn-cli/src/commands/summary.rs — natural place to surface a one-line stack summary

Suggested scope

Keep this regex/signature-based, no runtime probing of the user's machine. The goal is "what does the transcript imply about the stack?" not "what's installed?"

  • New module crates/relayburn-sdk/src/analyze/ecosystem.rs with a Fingerprint struct: language: Vec<LangSignal>, package_manager: Option<…>, test_runner: Option<…>, formatter, linter, vcs, shell, mcp_servers: Vec<String>, frameworks: Vec<String>.
  • Each detected signal carries a source list (e.g. bash:pytest, file_path:Cargo.toml, mcp_config:github) and a confidence tier so consumers can filter weak signals.
  • A curated, closed registry of detectors lives in the module — patterns for ~30–50 common tools (cargo, npm/pnpm/yarn, uv/pip/poetry, pytest, jest, vitest, ruff, mypy, eslint, prettier, clippy, rustfmt, git, mercurial, bash/zsh/fish, common MCP server names). Treat the registry as a versioned data table; document that PRs adding detectors must include a test.
  • Expose fingerprint_session(session_id) and fingerprint_sessions(filter) as SDK verbs. Persistence is optional for v1 — recomputing on demand is acceptable since classification is cheap.
  • CLI: add a --stack flag (or similar) to burn summary that prints a one-liner per session, and document that hotspots/patterns may filter by stack in a follow-up.

Definition of done

  • analyze/ecosystem.rs exists with a documented public type and at least 30 detector entries.
  • A new SDK verb returns a Fingerprint for a session; covered by unit tests against fixtures in tests/.
  • burn summary --stack (or chosen flag) prints stack info for recent sessions.
  • Detector registry has a // add detector here extension point and a test that asserts every registry entry has a non-empty source pattern.
  • No runtime filesystem probing — input is strictly the existing ingested event stream.

Out of scope

  • Runtime probing of which <tool> or version detection by shelling out. Transcript-only signals.
  • Per-stack remediation routing. That's a follow-up once the fingerprint exists.
  • Persisting fingerprints to the ledger. v1 recomputes from events.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions