Skip to content

TUI: Gemini response mode breakdown and drift detection #28

Description

@coreyt

Problem

The state store tracks gemini_outcomes (text, thought_only, tool) per client and per provider. The Overview screen shows raw counters in detail panes, but there's no way to:

  • See the distribution of response modes over time
  • Detect drift in thinking-mode behavior (e.g. a model suddenly returning thought_only for prompts that used to produce text)
  • Correlate Gemini response mode with error rates

Scope

  • Mode distribution in provider and client detail panes — show as percentages rather than raw counts (e.g. "text: 72%, thought_only: 23%, tool: 5%")
  • Drift indicator — if the ratio shifts significantly from the rolling average, surface a warning (could be an alert engine rule)
  • Correlation view — in the Guards screen detail, show Gemini response mode alongside composite score to help operators spot patterns

Backend references

  • fast/state.pygemini_outcomes deque on ClientState and ProviderState, recent_gemini_outcome_count() method
  • Guards screen already parses airlock_gemini and airlock_gemini_response from log records

Approach

Start with percentage display (simple math on existing counters). Drift detection requires a baseline — either a configurable expected ratio or a rolling window comparison. Keep it simple initially: flag when any single mode exceeds 80% of recent responses.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions