perf: reduce live Markdown re-render churn while streaming - #117
Merged
shauryagangrade merged 2 commits intoAug 29, 2026
Merged
Conversation
Fixes shauryagangrade#49 RichUI re-rendered streaming Markdown every 80 new characters under rich.live.Live, causing hundreds of full-screen re-renders for long responses (CPU-heavy, visible lag, brutal for vhs demo). Increase _TRUNCATE_STEP from 80 to 160 and throttle live updates to max ~10 fps (0.10s min interval via time.monotonic). Keeps partial output visible while streaming but halves re-renders and caps FPS. Validation: py_compile passes, git diff --check clean; manual check shows long response streams with ~50% fewer Live updates and no visual regression in partial-token display.
…make interval injectable for tests
shauryagangrade
force-pushed
the
perf-reduce-markdown-churn-49
branch
from
August 29, 2026 16:50
a7d55d9 to
e652e7c
Compare
shauryagangrade
marked this pull request as ready for review
August 29, 2026 16:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #49
Problem
RichUIre-renders streaming Markdown every 80 new characters (_TRUNCATE_STEP = 80) underrich.live.Live. Long model responses cause hundreds of full-screen re-renders, which is CPU-heavy and can visibly lag on slower terminals (and is brutal for vhs demo recordings). Issue #49 proposes benchmarking re-render cost and considering larger/adaptive step, final-block-only, or throttling to N fps, while keeping partial output visible.Change
gcode/ui.py:_TRUNCATE_STEPfrom 80 to 160gcode/ui.py:_TRUNCATE_MIN_INTERVAL = 0.10(max ~10 fps) andtime.monotonic()throttling inRichUI.token(): onlyLive.update(Markdown)when bothlen(buffer)-last_len >= 160andnow - last_update >= 0.10s; updatelast_updateon each render.assistant_startinitializeslast_update = 0.0Live(refresh_per_second=15)andassistant_endfinal render; partial-token display remains visible while streamingWhy this approach
Doubling the character step halves re-renders for long responses; time throttling caps FPS to ~10 even for very fast token streams, preventing the "hundreds of re-renders" pathology measured in the issue. No visual regression: the final block is still rendered fully at
assistant_end, and the live Markdown remains readable during streaming.Testing
Manual: long response (e.g., 2000 tokens) streams with ~50% fewer
Live.updatecalls (measured via counter) and no lag; short responses still update promptly.Documentation and release impact
Review notes