feat(stream): the big model types, and the end is instant - #59
Merged
hannesreinsch merged 2 commits intoSep 9, 2026
Merged
Conversation
The operator's call, given both texts side by side: "I would rather have it qualitatively high rather than faster." THE BIG MODEL TYPES. Once a live pass could type PUNCTUATION (#57), the model answering it stopped deciding only the words and started deciding the marks he keeps — and the two are not close there. Replayed through the whole stream loop on one real 38s clip, against the big model's own whole-clip transcript: live = small 13.9% different "...in the end like when I just stopped my control it just added a lot of gibberish" live = big 11.4% different "...in the end, like when I just stopped my control, it just added a lot of gibberish, I'm not sure. And that came after a few seconds, after I already sent the message" The percentages understate it; the sentences are the finding. `livePass: "small"` puts the fast one back, and the cost is real: a pass over a long clip costs seconds rather than ~1, so words arrive in bigger lumps the longer you talk. AND THE END IS INSTANT, which is what pays for that. From his own log a 73.7s clip spent 3.2s in the final transcription, 69.0s spent 6.0s, 79.1s spent 7.8s — and every one of those rows says `→ streamed`: the pass produced NOTHING that was not already on screen. Worse than wasted, because he had stopped, read his sentence and sent it, and the tail then landed eight seconds later in whatever he was looking at ("it just added a lot of gibberish ... after I already sent the message"). A partial only reads the audio captured so far, so the question is what was said after the last one — and if that is silence, its transcript IS the transcript: same model now, same audio, same gates. `Stream.read` is ONE attribute holding the pass and the seconds it covered, because `finish` reads it from another thread while the loop is still writing: as two fields a reader could take the new length beside the older transcript, call the clip fully read, and drop what was said in between. `seconds_on_disk` measures from the file SIZE like `tail_dbfs`, before the pass rather than after, so it under-reports and the shortcut refuses more often than it must. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P7oJz8M9QzsdJimoM318cj
|
Warning Review limit reachedNext included review available in 33 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…rited `partial_at` sends the partials to whatever is answering on the live port, so a small server still up from before `livePass` changed kept on answering them: the setting would have appeared to do nothing until the machine was restarted, with 488 MB sitting there while it did. Found by installing the change and watching :8480 keep serving. Only a server that is up AND ours is stopped — this must never become a blind pkill on a port we do not hold, which is also what keeps the bounce tests honest: they assert the wedged-server bounce takes only the big port, and a startup stop that fired unconditionally would have made that assertion pass for the wrong reason. The daemon's own line said the opposite of the default too, and told him to install the small model he had just been moved off. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P7oJz8M9QzsdJimoM318cj
hannesreinsch
deleted the
feat/the-big-model-types-and-the-end-is-instant
branch
September 9, 2026 13:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The big model types
Once a live pass could type punctuation (#57), the model answering it stopped deciding only the words and started deciding the marks he keeps. The two models are not close there.
Replayed through the whole stream loop on one real 38s clip, against the big model's own whole-clip transcript:
...in the end like when I just stopped my control it just added a lot of gibberish...in the end, like when I just stopped my control, it just added a lot of gibberish, I'm not sure. And that came after a few seconds, after I already sent the messageThe percentages understate it. The sentences are the finding.
livePass: "small"puts the fast one back. The cost of the default is real: a pass over a long clip costs seconds rather than ~1, so words arrive in bigger lumps the longer you talk.And the end is instant, which is what pays for it
From his own log:
→ streamed— nothing→ streamed— nothingThat was not gibberish. It was his own tail, eight seconds late, typed into whatever he was looking at by then.
A partial only reads the audio captured so far, so the question is what was said after the last one. If that is silence, its transcript is the transcript — same model now, same audio, same gates.
The race that had to be closed first
Stream.readis one attribute holding the pass and the seconds it covered, becausefinishreads it from another thread while the loop is still writing. As two fields, a reader could take the new length beside the older transcript, call the clip fully read, and drop what was said in between.seconds_on_diskmeasures from the file size liketail_dbfs, and is taken before the pass rather than after, so it under-reports and the shortcut refuses more often than it strictly must. That is the right way for a shortcut to be wrong.Gate
207 passed, ruff clean, mypy clean. The shortcut is driven through all four of its answers, including the one where somebody kept talking after the last pass looked.🤖 Generated with Claude Code
https://claude.ai/code/session_01P7oJz8M9QzsdJimoM318cj