The widget currently waits for the full answer before showing anything (server/src/llm.js calls the LLM non-streaming, widget/widget.js awaits the full JSON response). Wire up streaming (SSE from the backend, or fetch streaming) so answers appear incrementally, which will feel much more responsive.
Suggested scope:
Add a streaming code path in llm.js for both OpenAI and Anthropic
Switch POST /api/chat to stream chunks (SSE) when the client requests it
Update widget.js to append tokens as they arrive instead of waiting for the full response
The widget currently waits for the full answer before showing anything (server/src/llm.js calls the LLM non-streaming, widget/widget.js awaits the full JSON response). Wire up streaming (SSE from the backend, or fetch streaming) so answers appear incrementally, which will feel much more responsive.
Suggested scope:
Add a streaming code path in llm.js for both OpenAI and Anthropic
Switch POST /api/chat to stream chunks (SSE) when the client requests it
Update widget.js to append tokens as they arrive instead of waiting for the full response