Skip to content

fix(0.6.1): dashboard p95 fallback hint -> compact tooltip #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nicolotognoni wants to merge 1 commit into
base: fix/0.6.1-dashboard-p95-sample-threshold
Choose a base branch
from
Open

fix(0.6.1): dashboard p95 fallback hint -> compact tooltip #98

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## 0.6.1 (2026-05-12)

### Changed — Dashboard p95 fallback hint moved to a compact "?" tooltip

The p95→p50 fallback (shipped earlier in this release) rendered a verbose explanatory row under each latency card — `"6 turns — p95 hidden until ≥10, showing p50"` — and parenthetical column labels like `"END-TO-END P50 (N<10)"`. On short calls this added two lines of repeated reasoning to every panel and the call-table row, drowning out the actual numbers. Replaced both with a single inline `?` badge next to the label that shows the same explanation on hover: `"p95 hidden — needs ≥10 turns (n=X). Currently showing p50."`. The label itself is now honest about what is being shown (`"p50 round-trip"`, `"end-to-end p50"`, `"p50"`, `"p50 wait"`) rather than masquerading as p95 with a parenthetical disclaimer. CallTable column tooltip phrasing updated to match. Files: `dashboard-app/src/components/LatencyPanel.tsx`, `dashboard-app/src/components/MetricsPanel.tsx`, `dashboard-app/src/components/CallTable.tsx`, `dashboard-app/src/styles/dashboard.css` (new `.info-q` badge style). Bundle re-synced to `libraries/{typescript,python}/.../dashboard/ui.html`.

### Changed — Dashboard percentile threshold raised back to 10 turns (with p50 fallback)

The PR-#82 follow-up lowered the percentile sample threshold from 5 → 2 turns to keep the per-call detail pane in sync with the call-list column. In practice that produced misleading headline numbers on short calls: a live test with n=5 turns surfaced `p95=1977 ms` while `p50=309 ms` — the dashboard showed the 1977 ms outlier as "latency" because at n=5 the 95th-percentile collapses to "the single slowest turn" rather than a true tail estimate. Raised the threshold back to 10 (where p95 interpolates between samples 9 and 10 and starts being statistically meaningful), but instead of returning to a blank `—`, every surface now falls back to **p50** below the threshold and labels itself accordingly:
Expand Down
2 changes: 1 addition & 1 deletion dashboard-app/src/components/CallTable.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ function CallRow({ call, isSelected, onSelect, isNew }: CallRowProps) {
const warn = (latencyValue ?? 0) > 600;
const latencyTooltip = usePct95
? undefined
: `p95 hidden until ≥${MIN_TURNS_FOR_P95_COLUMN} turns — showing p50 instead (n=${turns})`;
: `p95 hidden — needs ≥${MIN_TURNS_FOR_P95_COLUMN} turns (n=${turns}). Currently showing p50.`;

const totalCost =
call.cost.total ??
Expand Down
29 changes: 16 additions & 13 deletions dashboard-app/src/components/LatencyPanel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ export function LatencyPanel({ call }: LatencyPanelProps) {
const showPercentiles = turns >= MIN_TURNS_FOR_PERCENTILES;
const dash = '—';

const lowSampleHint = `p95 hidden until ≥${MIN_TURNS_FOR_PERCENTILES} turns — showing p50 instead (n=${turns})`;
// Compact tooltip explaining why the p95 box is rendering the p50 value.
// The "?" badge next to the label is the only visual cue — the verbose
// inline hint row was removed to keep the card uncluttered.
const lowSampleHint = `p95 hidden — needs ≥${MIN_TURNS_FOR_PERCENTILES} turns (n=${turns}). Currently showing p50.`;

return (
<div className="rr-card">
Expand All @@ -53,12 +56,14 @@ export function LatencyPanel({ call }: LatencyPanelProps) {
className={
'latbox' + (showPercentiles && (call.latencyP95 ?? 0) > 600 ? ' warn' : '')
}
title={showPercentiles ? undefined : lowSampleHint}
>
<div className="l">
{showPercentiles
? 'p95 round-trip'
: `p50 round-trip (n<${MIN_TURNS_FOR_PERCENTILES})`}
{showPercentiles ? 'p95 round-trip' : 'p50 round-trip'}
{!showPercentiles && (
<span className="info-q" title={lowSampleHint} aria-label={lowSampleHint}>
?
</span>
)}
</div>
<div className="v">
{showPercentiles ? call.latencyP95 ?? dash : call.latencyP50 ?? dash}
Expand All @@ -79,10 +84,14 @@ export function LatencyPanel({ call }: LatencyPanelProps) {
'latbox' +
(showPercentiles && (call.agentResponseP95 ?? 0) > 600 ? ' warn' : '')
}
title={showPercentiles ? undefined : lowSampleHint}
>
<div className="l">
{showPercentiles ? 'p95 wait' : `p50 wait (n<${MIN_TURNS_FOR_PERCENTILES})`}
{showPercentiles ? 'p95 wait' : 'p50 wait'}
{!showPercentiles && (
<span className="info-q" title={lowSampleHint} aria-label={lowSampleHint}>
?
</span>
)}
</div>
<div className="v">
{showPercentiles
Expand All @@ -94,12 +103,6 @@ export function LatencyPanel({ call }: LatencyPanelProps) {
</div>
</div>
</div>
{!showPercentiles && (
<div style={{ marginTop: -6, marginBottom: 10, fontSize: 11, opacity: 0.6 }}>
{turns} {turns === 1 ? 'turn' : 'turns'} — p95 hidden until ≥
{MIN_TURNS_FOR_PERCENTILES}, showing p50
</div>
)}

<div className="waterfall">
<div className="wf-row">
Expand Down
42 changes: 18 additions & 24 deletions dashboard-app/src/components/MetricsPanel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ function LatencyView({ call }: { call: Call }) {
if (isRealtime) {
const turnsRt = call.turnCount ?? 0;
const showPctRt = turnsRt >= MIN_TURNS_FOR_PERCENTILES;
const lowSampleHint = `p95 hidden until ≥${MIN_TURNS_FOR_PERCENTILES} turns — showing p50 instead (n=${turnsRt})`;
const lowSampleHint = `p95 hidden — needs ≥${MIN_TURNS_FOR_PERCENTILES} turns (n=${turnsRt}). Currently showing p50.`;
return (
<>
<div className="lat-grid">
Expand All @@ -91,25 +91,21 @@ function LatencyView({ call }: { call: Call }) {
{p50 > 0 && <span className="u">ms</span>}
</div>
</div>
<div
className={'latbox' + (showPctRt && p95 > 600 ? ' warn' : '')}
title={showPctRt ? undefined : lowSampleHint}
>
<div className={'latbox' + (showPctRt && p95 > 600 ? ' warn' : '')}>
<div className="l">
{showPctRt ? 'end-to-end p95' : `end-to-end p50 (n<${MIN_TURNS_FOR_PERCENTILES})`}
{showPctRt ? 'end-to-end p95' : 'end-to-end p50'}
{!showPctRt && (
<span className="info-q" title={lowSampleHint} aria-label={lowSampleHint}>
?
</span>
)}
</div>
<div className="v">
{showPctRt ? p95 || '—' : p50 || '—'}
{(showPctRt ? p95 : p50) > 0 && <span className="u">ms</span>}
</div>
</div>
</div>
{!showPctRt && (
<div style={{ marginTop: -6, marginBottom: 8, fontSize: 11, opacity: 0.6 }}>
{turnsRt} {turnsRt === 1 ? 'turn' : 'turns'} — p95 hidden until ≥
{MIN_TURNS_FOR_PERCENTILES}, showing p50
</div>
)}
<div className="waterfall">
<div className="wf-row">
<span className="lbl">e2e</span>
Expand Down Expand Up @@ -145,7 +141,7 @@ function LatencyView({ call }: { call: Call }) {
// p50 — robust to outliers — and label the box accordingly.
const turns = call.turnCount ?? 0;
const showPct = turns >= MIN_TURNS_FOR_PERCENTILES;
const lowSampleHint = `p95 hidden until ≥${MIN_TURNS_FOR_PERCENTILES} turns — showing p50 instead (n=${turns})`;
const lowSampleHint = `p95 hidden — needs ≥${MIN_TURNS_FOR_PERCENTILES} turns (n=${turns}). Currently showing p50.`;

return (
<>
Expand All @@ -157,11 +153,15 @@ function LatencyView({ call }: { call: Call }) {
{call.latencyP50 != null && <span className="u">ms</span>}
</div>
</div>
<div
className={'latbox' + (showPct && p95 > 600 ? ' warn' : '')}
title={showPct ? undefined : lowSampleHint}
>
<div className="l">{showPct ? 'p95' : `p50 (n<${MIN_TURNS_FOR_PERCENTILES})`}</div>
<div className={'latbox' + (showPct && p95 > 600 ? ' warn' : '')}>
<div className="l">
{showPct ? 'p95' : 'p50'}
{!showPct && (
<span className="info-q" title={lowSampleHint} aria-label={lowSampleHint}>
?
</span>
)}
</div>
<div className="v">
{showPct ? p95 || '—' : call.latencyP50 ?? '—'}
{(showPct ? p95 : call.latencyP50) != null &&
Expand All @@ -183,12 +183,6 @@ function LatencyView({ call }: { call: Call }) {
</div>
</div>
</div>
{!showPct && (
<div style={{ marginTop: -6, marginBottom: 8, fontSize: 11, opacity: 0.6 }}>
{turns} {turns === 1 ? 'turn' : 'turns'} — p95 hidden until ≥
{MIN_TURNS_FOR_PERCENTILES}, showing p50
</div>
)}

<div className="waterfall">
<div className="wf-row">
Expand Down
5 changes: 5 additions & 0 deletions dashboard-app/src/styles/dashboard.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,11 @@ tbody tr.new-row{animation:slideIn 400ms var(--easing)}
.latbox .v .u{font-size:11px;color:#aaa;font-weight:500;margin-left:2px}
.latbox.warn{background:#fff8ef;border-color:#EFC5AC}
.latbox.warn .v{color:#c97a4c}
/* Small "?" affordance shown next to a label when the metric is in a degraded
* mode (e.g. p95 falling back to p50 below the sample threshold). Hovering
* reveals the explanation via the native title tooltip. */
.info-q{display:inline-flex;align-items:center;justify-content:center;width:12px;height:12px;margin-left:4px;border-radius:50%;border:1px solid currentColor;font-family:var(--font-mono);font-size:9px;line-height:1;font-weight:600;color:var(--fg-tertiary);cursor:help;vertical-align:1px;opacity:.7}
.info-q:hover{opacity:1}

/* latency waterfall */
.waterfall{margin-top:12px;display:flex;flex-direction:column;gap:6px}
Expand Down
Loading
Loading