Study guide will take some time to generate, would be nice if it didn't have to wait and would just stream.
for message_chunk, metadata in graph.stream(
{"topic": "ice cream"},
stream_mode="messages",
):
if message_chunk.content:
print(message_chunk.content, end="", flush=True)
Here it streams content of message chunk. The metadata contains which node is producing stream from LLM.
However, ToolNode for some reason streams additional metadata in a strange format. So for ToolNode streaming, need to tell LLM to wrap content in some markers like "::output::Here is your response from LLM::output"
Study guide will take some time to generate, would be nice if it didn't have to wait and would just stream.
Here it streams content of message chunk. The metadata contains which node is producing stream from LLM.
However, ToolNode for some reason streams additional metadata in a strange format. So for ToolNode streaming, need to tell LLM to wrap content in some markers like "::output::Here is your response from LLM::output"