Skip to content

Charts only render when the user explicitly asks for one #29

@sagar-develop

Description

@sagar-develop

Summary

Charts in chat bubbles only appear when the prompt explicitly requests a chart (e.g. "show a monthly budget as a donut chart"). The model almost never decides on its own to visualize numeric/tabular answers, even when a chart would clearly help.

Steps to reproduce

  1. Use a chart-capable model (e.g. Gemma 4 E2B).
  2. Ask a question whose answer is naturally chartable without naming a chart, e.g. "Break down a typical monthly budget" or "Compare Q1–Q4 sales: 10, 14, 9, 18".
  3. The answer comes back as prose/table — no chart.
  4. Re-ask adding "as a donut/bar chart" → chart renders correctly.

Expected

When the answer is genuinely easier to grasp visually, the assistant should proactively render an appropriate chart without the user having to name the chart type.

Root cause (current design)

Chart emission is entirely model-initiated and permission-based:

  • lib/src/commonMain/kotlin/com/sagar/aicore/chart/ChartInstruction.kt — the SYSTEM string frames charts as optional ("you MAY include ONE chart… Use it only when it genuinely helps"). It's permission + examples, not a directive, so conservative models stay text-only.
  • NativeLmViewModel.systemInstruction() (~line 801) only appends ChartInstruction.SYSTEM when model.supportsCharts (enabled for ≥1.5B models; tiny models are excluded because they over-emit).
  • MarkdownText.kt (~line 286) only renders a chart from a fenced block tagged ```chart/```json/untagged that ChartParser can parse. There is no app-side heuristic that auto-visualizes tabular/numeric answers.

So "charts only on explicit ask" is the emergent result of a deliberately cautious prompt.

Possible directions (not committing to one)

  • Strengthen the prompt to encourage (vs merely permit) a chart for clearly numeric answers, with concrete trigger criteria (proportions, comparisons of 2–6 items, trends).
  • Add a few-shot example where the user does NOT name a chart but the assistant chooses one.
  • Optional app-side fallback: detect a clean numeric table in the answer and offer/auto-render a chart.

Filed from observed behavior; no fix applied yet.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions