feat(ui): cross-cluster comparison + cross-cluster chat reasoning#3
Conversation
The per-cluster chat only loaded the clicked cluster's stats, so the LLM
could not answer cross-cluster questions ('why are clusters 0 and 3 both
high-X but 0 leans Y while 3 leans Z?'). Two changes fix this:
- /api/cluster-chat now appends a compact roster of EVERY cluster (name,
size, stronger/weaker features + ratios) to the system prompt, with the
clicked cluster kept as the focus. The model can now compare and contrast
any clusters the user names in the Named Clusters tab.
- New /api/cluster-comparison endpoint runs an LLM contrasting analysis
across all clusters (separating axes, look-alike-but-diverging pairs,
merge candidates) on the evidence ledger, surfaced as an on-demand
'Cross-cluster comparison' card in the Data & Evidence tab. Cached by
cluster signature so re-opens don't re-bill.
Shared _clusters_overview_lines / _sorted_cluster_ids helpers back both.
There was a problem hiding this comment.
Pull request overview
Adds cross-cluster reasoning to the existing per-cluster chat and introduces a new on-demand cross-cluster comparison card in the Data & Evidence tab. The chat system prompt now embeds a roster of every cluster (so the LLM can contrast clusters), and a new /api/cluster-comparison endpoint produces a contrasting analysis billed to the evidence ledger with a signature-based cache.
Changes:
- Augment
cluster_chatsystem prompt with an "ALL CLUSTERS IN THIS RUN" roster via two new helpers (_sorted_cluster_ids,_clusters_overview_lines); update chat copy/placeholder. - Add
POST /api/cluster-comparisonwith a TTL+lock-guarded cache, calling the persona agent bus undercategory='evidence'. - Add
buildClusterComparisonCard/wireComparisonButtonin the Evidence tab UI, shown when ≥2 personas exist.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| ui/app.py | New helpers for cluster roster; expanded chat system prompt with cross-cluster roster; new /api/cluster-comparison endpoint with TTL cache. |
| ui/static/app.js | New cross-cluster comparison card + wiring in Evidence tab; updated chat help text and placeholder. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 46a3a6f6d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| sig = "|".join( | ||
| f"{cid}:{personas[cid].get('persona', {}).get('name', '')}" | ||
| for cid in _sorted_cluster_ids(personas) | ||
| ) + f"||focus={focus}" |
There was a problem hiding this comment.
When the same Flask process is used for another run that produces the same cluster IDs and persona names within the 10-minute TTL, this signature still hits even though the sizes/features passed to _clusters_overview_lines() may have changed. In that case the Data & Evidence tab returns an old cross-cluster analysis marked as cached, which can mislead experiment interpretation; include the roster/stat payload, output mtime, or run id in the key rather than only id/name/focus.
Useful? React with 👍 / 👎.
| const focus = (document.getElementById('ev-compare-focus')?.value || '').trim(); | ||
| const out = document.getElementById('ev-compare-out'); | ||
| btn.disabled = true; | ||
| btn.innerHTML = `<span class="spinner"></span>Comparing (evidence ledger)…`; | ||
| try { | ||
| const r = await api('POST', '/api/cluster-comparison', {focus}); |
There was a problem hiding this comment.
The evidence tab is re-rendered by startGlobalRefreshWatchdog() every 20 seconds while visible, so if this LLM call takes longer than that the DOM nodes captured here are detached and the completed analysis is written into an invisible old out element; the replacement card also shows a fresh enabled button, allowing duplicate requests. Store the in-flight/result state outside the card or re-resolve the current elements after the await so long comparisons remain visible and cannot be rebilled accidentally.
Useful? React with 👍 / 👎.
Problem
When clustering finishes, users click a named-cluster card to chat with the LLM about it. But the chat (
POST /api/cluster-chat) only loaded the single clicked cluster's stats into the system prompt, so the LLM had no data about any other cluster. Cross-cluster questions like "why are clusters 0 and 3 both high-spend, but 0 leans X while 3 leans Y?" couldn't be answered — the model literally couldn't see cluster 3's numbers.Changes
1. Cross-cluster reasoning in the Named Clusters chat (
ui/app.py)2. Cross-cluster comparison in the Data & Evidence tab (
ui/app.py+ui/static/app.js)POST /api/cluster-comparisonendpoint runs an LLM contrasting analysis across all clusters: separating axes, look-alike-but-diverging pairs, overlap/merge candidates, and a takeaway.Shared
_clusters_overview_lines/_sorted_cluster_idshelpers back both paths.Verification
python -m py_compile ui/app.pyandnode --check ui/static/app.jsboth pass.flaskinstalled in the session, andpersonas.jsonis cleared on this branch), but the new code reads the identical persona/stats fields the existing card and chat code already rely on.https://claude.ai/code/session_01QL4MpBtJ46QLcwBezp6Qk7
Generated by Claude Code