Thanks for creating this library. It greatly simplifies calling the Gradient API. I am wondering how we can add something to the streaming chunk. Our current code, just plain javascript, is below and we are trying to simplify it using this SDK, but we are not sure how to add the "type": "sources" to the stream, and this is essential for our front end RAG application. Can you please let me know how we can use the Stream part of the SDK to do something like below? Specifically, it's this line we are not sure how to implement with the SDK: controller.enqueue(textEncoder.encode(JSON.stringify({ type: "sources", data: usedSources })));
const aiRequest = await fetch(
`${AGENT_ENDPOINT}/api/v1/chat/completions`,
{
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${AGENT_KEY}`
},
method: 'POST',
body: JSON.stringify({
messages: [
{
role: "user",
content: aiContent //the prompt
}
],
stream: true,
include_functions_info: false,
include_retrieval_info: false,
include_guardrails_info: false
}),
}
)
const stream = new ReadableStream({
async start(controller) {
const reader = aiRequest?.body?.getReader();
if (!reader) {
controller.error("ReadableStream reader not available");
return;
}
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) { break; }
const chunk = decoder.decode(value, { stream: true });
// Split SSE lines
for (const line of chunk.split("\n")) {
if (!line.startsWith("data: ")) { continue; }
const data = line.replace("data: ", "").trim();
if (data === "[DONE]") { continue; }
try {
const json = JSON.parse(data);
const token = json.choices?.[0]?.delta?.content;
if (token) {
controller.enqueue(new TextEncoder().encode(token));
}
} catch {
continue;
}
}
}
const usedSources = items.slice(0, 5); // example: top 5 sources used
controller.enqueue(textEncoder.encode(JSON.stringify({ type: "sources", data: usedSources })));
controller.close();
},
});
//console.log ("stream", stream)
return new Response(stream, {
headers: { "Content-Type": "text/plain; charset=utf-8" },
});
Thanks for creating this library. It greatly simplifies calling the Gradient API. I am wondering how we can add something to the streaming chunk. Our current code, just plain javascript, is below and we are trying to simplify it using this SDK, but we are not sure how to add the "type": "sources" to the stream, and this is essential for our front end RAG application. Can you please let me know how we can use the Stream part of the SDK to do something like below? Specifically, it's this line we are not sure how to implement with the SDK: controller.enqueue(textEncoder.encode(JSON.stringify({ type: "sources", data: usedSources })));
Current code