mcp: add compact run and result navigation by okt-limonikas · Pull Request #330 · ts-factory/bublik

okt-limonikas · 2026-06-15T21:41:00Z

Stacked on top of #324

Replace the fragmented run metadata tools and the ambiguous direct-child
list_results interface with two validated Markdown workflows for MCP clients.

What changed

This PR introduces two higher-level run navigation tools:

get_run_overview
get_run_leaf_results

Together they replace the previous flow where clients had to call separate run
metadata/stat/source tools and manually reconstruct how to navigate from a run
tree node to concrete test executions.

New tools

`get_run_overview`

Returns a complete Markdown overview for a run, including:

run metadata
status and conclusion
source URL
compromised details
recursive result statistics tree
aggregate Result IDs for test leaves

The tool also supports:

requirements: semicolon-separated requirement filter
unexpected_only: return only leaf tests containing unexpected or abnormal results

Each test row includes a Result ID that can be passed directly to
get_run_leaf_results.

`get_run_leaf_results`

Returns paginated concrete executions represented by one test leaf from
get_run_overview.

The input is intentionally an aggregate test Result ID from the overview, not an
individual execution ID. The tool validates that the ID points to a real test
leaf before resolving executions.

It supports two usage modes:

unexpected_only=true: common failure-investigation mode, returning only
unexpected or abnormal executions
advanced filters:
- requirements
- results, for example PASSED;FAILED;SKIPPED;KILLED;CORED;FAKED;INCOMPLETE
- result_properties, for example expected;unexpected;not_run

unexpected_only is mutually exclusive with the advanced filters.

Why this is better

The old MCP interface exposed lower-level pieces of the UI model. Clients had to
combine run details, source, stats, and direct-child result listing themselves,
while also knowing how Bublik lazy-loads the run tree.

The new flow is explicit:

Call get_run_overview(run_id) to inspect the run and find interesting test leaves.
Pick a test row Result ID.
Call get_run_leaf_results(leaf_result_id) to inspect concrete executions.

This makes MCP usage more reliable and cheaper for large runs because clients no
longer need to fetch or reason over unrelated execution details.

Token impact

Estimated token count changes:

Full run stats: ~26,000 tokens -> ~10,000 tokens
Unexpected-only run stats leaves: ~26,000 tokens -> ~2,000 tokens

Validation and rendering

The new run payloads are validated with strict Pydantic models before Markdown
rendering. This keeps the MCP output compact and human-readable while still
catching unexpected service payload changes early.

Issue: #326
Issue: #327

The API module intended to declare public exports, but used a plain all variable instead of Python's __all__ convention. Fix the export declaration separately so later run and result API refactors do not carry an unrelated module-level cleanup. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

ol-nata · 2026-06-16T11:08:30Z

+    from datetime import datetime, timedelta
+
+
+class RunCompromisedDetails(BaseModel):


PR #324 introduces a typed DTO layer in bublik/core/run/dto.py that covers the same domain structures — RunDetailsResult, RunStatsResult, RunCompromisedDetails, RunRevision, and others — using dataclasses. This PR defines equivalent models in bublik/mcp/run/models.py using Pydantic. If both are merged, the codebase will have two parallel definitions of the same entities with no single source of truth. It may be worth coordinating with #324 to avoid the duplication — the MCP Pydantic models could potentially be built on top of the DTO layer rather than alongside it.

I've made changes and removed redundant pydantic models and run formatting via ruff like in this PR #331

ol-nata

Please use a more goal-oriented commit summary: replace mcp: add compact run and result navigation with something like mcp: enforce structured run/result navigation workflow to prevent incorrect tool usage

ol-nata · 2026-06-17T14:14:09Z


 from pydantic import AnyUrl, BaseModel, ConfigDict, Field, RootModel

-


This change is unrelated to the MCP navigation workflow and should not be included in this commit. Please remove it to keep the architectural change focused, consistent, and reviewable.

Split run and result API views into dedicated modules to keep entity-specific code isolated and easier to evolve independently. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

Replace dict-based service responses with typed DTOs to introduce explicit and structured data contracts across service boundaries and move serialization to API/MCP boundary to improve separation of concerns between service layer and API/MCP layers. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

Ensure consistency between OpenAPI schemas and API responses by introducing explicit request/response serializers and binding them via drf-spectacular. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

Replace the fragmented run metadata tools and the ambiguous direct-child `list_results` interface with two validated Markdown workflows. `get_run_overview` combines run metadata with the recursive statistics tree and can reduce it to unexpected leaves. `get_run_leaf_results` accepts an aggregate test result ID from that overview and delegates concrete execution filtering and pagination to the existing result service. This gives MCP clients a compact way to inspect large run trees without reconstructing UI lazy-loading semantics, while preserving strict payload validation before rendering. Issue: ts-factory#326 Issue: ts-factory#327 Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>

Link: https://peps.python.org/pep-0257/ Link: ts-factory#331 Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>

okt-limonikas · 2026-06-18T01:10:38Z

Please use a more goal-oriented commit summary: replace mcp: add compact run and result navigation with something like mcp: enforce structured run/result navigation workflow to prevent incorrect tool usage

Changed to mcp: enforce run/result navigation to reduce context usage
Also rebased

okt-limonikas force-pushed the mcp-improve-run branch 6 times, most recently from 158c578 to 9693777 Compare June 16, 2026 01:02

okt-limonikas marked this pull request as ready for review June 16, 2026 01:02

ol-nata suggested changes Jun 16, 2026

View reviewed changes

okt-limonikas force-pushed the mcp-improve-run branch 3 times, most recently from 34daf38 to 5eb5350 Compare June 16, 2026 17:59

ol-nata suggested changes Jun 17, 2026

View reviewed changes

ol-zayatsm added 4 commits June 17, 2026 19:49

run: prepare API modules for independent evolution

360f5bd

Split run and result API views into dedicated modules to keep entity-specific code isolated and easier to evolve independently. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

run: align OpenAPI schemas with actual API responses

7bcbb28

Ensure consistency between OpenAPI schemas and API responses by introducing explicit request/response serializers and binding them via drf-spectacular. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

result: align OpenAPI schemas with actual API responses

dd7e981

Ensure consistency between OpenAPI schemas and API responses by introducing explicit request/response serializers and binding them via drf-spectacular. Signed-off-by: Mikhail Zayats <mikhail.zayats@oktetlabs.ru>

okt-limonikas force-pushed the mcp-improve-run branch 3 times, most recently from e1073d7 to 1bf28e6 Compare June 18, 2026 01:08

okt-limonikas added 2 commits June 18, 2026 04:10

mcp: fix docstring quotes to use " as adviced in PEP 257

4551d96

Link: https://peps.python.org/pep-0257/ Link: ts-factory#331 Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>

okt-limonikas force-pushed the mcp-improve-run branch from 1bf28e6 to 4551d96 Compare June 18, 2026 01:10

okt-limonikas requested a review from ol-nata June 18, 2026 01:10

This was referenced Jun 18, 2026

Improve Bublik MCP list_results() behavior and documentation #326

Open

Add compact recursive run failure summary MCP/API for large get_run_stats trees #327

Open

ol-nata approved these changes Jun 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mcp: add compact run and result navigation#330

mcp: add compact run and result navigation#330
okt-limonikas wants to merge 7 commits into
ts-factory:mainfrom
okt-limonikas:mcp-improve-run

okt-limonikas commented Jun 15, 2026 •

edited

Loading

Uh oh!

ol-nata Jun 16, 2026

Uh oh!

okt-limonikas Jun 16, 2026 •

edited

Loading

Uh oh!

ol-nata left a comment

Uh oh!

ol-nata Jun 17, 2026

Uh oh!

okt-limonikas Jun 18, 2026

Uh oh!

okt-limonikas commented Jun 18, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		from datetime import datetime, timedelta


		class RunCompromisedDetails(BaseModel):


		from pydantic import AnyUrl, BaseModel, ConfigDict, Field, RootModel

Conversation

okt-limonikas commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

New tools

get_run_overview

get_run_leaf_results

Why this is better

Token impact

Validation and rendering

Uh oh!

ol-nata Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

okt-limonikas Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ol-nata left a comment

Choose a reason for hiding this comment

Uh oh!

ol-nata Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

okt-limonikas Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

okt-limonikas commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

okt-limonikas commented Jun 15, 2026 •

edited

Loading

`get_run_overview`

`get_run_leaf_results`

okt-limonikas Jun 16, 2026 •

edited

Loading

okt-limonikas commented Jun 18, 2026 •

edited

Loading