Skip to content

Implement token-by-token streaming for topic and class AI descriptions #109

Description

@nilskntl

Summary

Topic description streaming never sends TOKEN_DELTA events because the backend uses a blocking completions client and only calls publishDone(). Class description has no streaming at all. Both need proper token streaming, correct stream lifecycle handling after page refresh, and topic preview streaming on ClassDetailPage.

Current Behaviour

  • TopicDescriptionGenerationService uses OpenAIChatCompletionsClient (blocking, non-streaming). It never calls topicDescriptionStreamService.publishToken() — only publishDone(). The SSE stream therefore sends a single STREAM_DONE event, causing the description to pop in complete rather than streaming token-by-token.
  • ClassDescriptionGenerationService has no streaming infrastructure at all; no ClassDescriptionStreamService or SSE endpoint exists.
  • The stream trigger on TopicDetailPage only starts the stream when hasProcessingDocuments is true. On a cold page load / refresh this may not fire in time and the stream is never opened.
  • Topic preview cards in TopicList on ClassDetailPage show no streaming state for in-progress topics.

Desired Behaviour

  • Topic and class descriptions stream token-by-token while being generated.
  • Streaming reconnects correctly after a page refresh if documents are still processing.
  • No errors thrown when the user navigates away mid-stream.
  • Topic preview cards on ClassDetailPage also show the streaming description for any topic currently being generated.

Affected Module(s)

  • minerva-api
  • minerva-hub
  • minerva-converter
  • minerva-infra

Proposed Implementation

Backend (minerva-api):

  • Switch TopicDescriptionGenerationService to a streaming LLM client; call topicDescriptionStreamService.publishToken(topicId, token) for each token delta, then publishDone() at completion.
  • Create ClassDescriptionStreamService and ClassDescriptionStreamEmitter mirroring the existing topic stream infrastructure.
  • Switch ClassDescriptionGenerationService to use streaming and integrate with the new stream service.
  • Add SSE endpoint GET /api/classes/description/stream/{classId} in ClassesController.

Frontend (minerva-hub):

  • Fix the topic description stream trigger: subscribe immediately on page load regardless of hasProcessingDocuments; the SSE backend already returns STREAM_DONE immediately if the description is already generated, so this is safe and handles the post-refresh case.
  • Create useClassDescriptionStream hook mirroring useTopicDescriptionStream.
  • Integrate class description streaming in ClassDetailPage.
  • Pass the live streaming description into topic preview cards via TopicList on ClassDetailPage.
  • Ensure all SSE subscriptions are torn down on page unmount (not on re-render) to avoid errors when navigating away.

Note

In the Frontend Token Streaming for the aiDescription should only be shown when a user has not set a manual description. Otherwise there is no need for Token Streaming (both frontend and backend) and the visual description should stay the same

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions