Skip to content

[Performance]: Reduce live Markdown re-render churn while streaming #49

Description

@shauryagangrade

Problem

RichUI re-renders the streaming Markdown every 80 new characters
(_TRUNCATE_STEP = 80) under rich.live.Live. Long model responses cause
hundreds of full-screen re-renders, which is CPU-heavy and can visibly lag on
slower terminals (and is brutal for the vhs demo recordings).

Proposed approach

  • Benchmark re-render cost; consider: larger/adaptive step, rendering the
    final block only on stream end, or throttling to N fps.
  • Keep the "current partial output visible while streaming" behavior.

Where to look

  • gcode/ui.py:20 (_TRUNCATE_STEP), :132 (the Live refresh handler),
    :164-191 (assistant_start/token/assistant_end).

Acceptance criteria

  • A long model response streams with a measurable reduction in re-renders/Live updates.
  • No visual regression in partial-token display.

Difficulty

Medium.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceLatency, throughput, and efficiency improvements

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions