Skip to content

[Feature]: Expose runtime coverage and static test support through get_test_health #145

Description

@Luqueee

Problem

An agent needs to know whether production code is protected by tests, but users may forget to generate or provide a runtime coverage report. Treating a missing report as 0% would be misleading, while requiring a manually uploaded report makes the signal disappear in normal local workflows.

Kivgraph already contains symbol locations, typed relationships, call edges, repository boundaries and completeness metadata, but it does not expose test support as a queryable result.

Proposed solution

Add a read-only get_test_health MCP query with an auto mode:

  • Detect standard local coverage artifacts when present, such as Go cover profiles, LCOV, Cobertura and JaCoCo reports.
  • Return runtime line, function or branch coverage only when backed by an observed report.
  • When no runtime report exists, derive static test support from indexed test-to-production relationships.
  • Distinguish direct test calls, transitive potential reachability and convention-based candidates.
  • Keep runtime coverage and static test support in separate fields; never combine them into one percentage.
  • Report explicit states such as MEASURED, ESTIMATED, UNAVAILABLE, STALE and TESTS_NOT_INDEXED.
  • Bind runtime observations to the repository commit and/or file content digest where possible.
  • Keep the initial static index derived from the published graph rather than adding a new canonical edge or mutating the repository.

The static result should be described as test support or potential reachability, not as runtime coverage: a call path proves that code may be reached from a test, not that every branch executed.

Alternatives considered

  • Require users to upload a coverage report manually.
  • Treat imports, file names or test naming conventions as runtime coverage.
  • Run arbitrary test commands from the MCP server.
  • Add a TESTED_BY relation to the canonical graph before the evidence and freshness contract is defined.

Acceptance criteria

  • Standard coverage files are auto-discovered within registered repositories without requiring an upload step.
  • A missing report is reported as unavailable and is never rendered as 0% runtime coverage.
  • Runtime coverage includes its source report, commit/digest or freshness status, and supported metric basis.
  • Static results identify production symbols and test symbols with repository, path and line ranges.
  • Direct, transitive and convention-based evidence are separated and carry distinct confidence.
  • Missing or disabled test indexing is distinguishable from a genuine absence of test support.
  • Stale, ambiguous or path-mismatched reports fail closed or are clearly marked as unverified.
  • The response includes snapshot and completeness metadata, bounded/paged results, and no filesystem or repository mutation.
  • Negative tests cover missing reports, stale reports, ambiguous paths, unresolved calls, imports mistaken for coverage, and unindexed tests.
  • The tool is documented in the MCP protocol, skill and user-facing tool reference, and its response cost is measured against the MCP surface budget.

Additional context

Relevant prior art:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions