Problem
The MCP tool get_run_stats(run_id) returns the full nested result tree. This is useful but can be very large. In MCP clients/agent workflows, the response may be truncated in the client view, so callers cannot reliably inspect all descendant leaves or compute exact grouping counts from the visible response.
And get_run_stats() is the only MCP tool that takes run_id and allows to explore test results.
This makes workflows such as "summarize failing leaves by package/path" unreliable unless the caller fetches the raw API separately and processes the full JSON outside MCP.
Expected improvement
Add a compact MCP/API methods for recursive result summaries, for example:
summarize_unexpected_leaves(run_id, requirements=None, group_by=["top_package", "package_path"])
or:
list_unexpected_leaves(run_id, leaf_only=true, page_size=..., cursor=...)
Useful output:
- run_id
- root_result_id
- total unexpected leaf count
- counts by top-level package
- grouped counts by package path
- optional example result_id/path per group
- optional verdict/artifact/error signature per group
- optional pagination/cursor for leaf-level output
The tool should avoid returning the full result tree when the caller only needs summary counts or failing leaf records.
Minimum documentation improvement
Document that get_run_stats can return very large result trees and may be truncated in MCP clients. Recommend using a compact summary/listing endpoint for exact recursive leaf counts once available.
Problem
The MCP tool get_run_stats(run_id) returns the full nested result tree. This is useful but can be very large. In MCP clients/agent workflows, the response may be truncated in the client view, so callers cannot reliably inspect all descendant leaves or compute exact grouping counts from the visible response.
And get_run_stats() is the only MCP tool that takes run_id and allows to explore test results.
This makes workflows such as "summarize failing leaves by package/path" unreliable unless the caller fetches the raw API separately and processes the full JSON outside MCP.
Expected improvement
Add a compact MCP/API methods for recursive result summaries, for example:
or:
Useful output:
The tool should avoid returning the full result tree when the caller only needs summary counts or failing leaf records.
Minimum documentation improvement
Document that get_run_stats can return very large result trees and may be truncated in MCP clients. Recommend using a compact summary/listing endpoint for exact recursive leaf counts once available.