Skip to content

WebSockets: do not yield chunks unless request ID matches#94

Merged
bryananderson merged 2 commits intomasterfrom
discard-irrelevant-chunks
Jan 29, 2025
Merged

WebSockets: do not yield chunks unless request ID matches#94
bryananderson merged 2 commits intomasterfrom
discard-irrelevant-chunks

Conversation

@bryananderson
Copy link
Contributor

Right now, if a new request is made over a connected websocket before a previous request completes (for example, a user interrupts a voice agent and the agent needs to abandon an ongoing TTS request and start a different one), the client does not associate chunks with a specific request, so it may get chunks from the previous request and think they are from this one.

Here we pay attention to the "start" message instead of just mindlessly yielding audio chunks. Then we make sure any "end" message matches the same request ID.

We also make sure the number of sent requests and received responses stays in sync, to avoid an edge case where we sent a bunch of requests in quick succession and get back a bunch of "start" messages. We want to make sure that if we just made the nth request, we only pay attention to the nth response.

@bryananderson bryananderson force-pushed the discard-irrelevant-chunks branch from fff1b25 to 6a4ee6d Compare January 29, 2025 01:46
Copy link
Contributor

@TobiahRex TobiahRex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition!
Seems everything is relatively GTG, but in the scenario where "end" is happening without a matching request_id and we continue feels like an Exception or a break? But curious if i'm missing something obvious.

@bryananderson
Copy link
Contributor Author

Nice addition! Seems everything is relatively GTG, but in the scenario where "end" is happening without a matching request_id and we continue feels like an Exception or a break? But curious if i'm missing something obvious.

It should be a continue because if an old request was abandoned, we may get its remaining frames followed by its "end" message. And the right thing to do is to wait for the current request.

@bryananderson bryananderson merged commit 3eff5c5 into master Jan 29, 2025
9 checks passed
@bryananderson bryananderson deleted the discard-irrelevant-chunks branch January 29, 2025 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants