User Story
As a user, I want to upload images and PDFs to the knowledge base, so that the AI can automatically caption images and parse PDFs, and summarize their contents.
Context & Motivation
The single-file API (/api/v1/ingest) supports vision processing, but the knowledge base upload process uses the batch ingestion endpoint (POST /api/v1/ingest/sync), which is currently missing this logic.
As a result:
When the batch endpoint encounters a Base64-encoded image or PDF, it skips decoding.
It embeds the raw Base64 string instead of parsing the file.
Summarization fails because the AI cannot interpret raw Base64.
This issue adds the missing Base64 decoding and vision captioning logic to the batch endpoint.
Acceptance Criteria
Given a batch ingestion request containing an image or PDF artifact, when _assemble_content is called, then it should not prepend # {title} to the base64 string, allowing it to be decoded cleanly.
Given a parsed chunk of kind image, when _ingest_one processes it, then it should decode the Base64 payload back to bytes.
Given decoded image bytes, when the AI processes the chunk, then it should call llm.caption_image to generate a rich text description of the image.
Given a generated image caption, when the chunk is saved, then it should replace the original Base64 payload with the generated text caption.
Given a PDF artifact, when _ingest_one is preparing the content_bytes to pass to parse(), it should decode the Base64 payload into raw binary bytes so pdf_parser can read it.
Edge case: Given a vision model outage (e.g., LLMUnavailableError), when the captioning fails, then the chunk should gracefully log a warning and mark the artifact as failed so it can be retried later.
Technical Notes
Affected Component: ingest_run.py in sprintstart-ai.
Implementation hint:
In _assemble_content, check if artifact.mime.startswith("image/") or equals application/pdf. If so, return body_text directly to avoid prepending the title.
In _ingest_one, if artifact.mime == "application/pdf", Base64 decode content_bytes back into raw binary before passing it to parse().
In _ingest_one, after parsing chunks, iterate over them and check if chunk.kind == "image". Apply the base64.b64decode and llm.caption_image(image_bytes) logic exactly as it is already implemented in ingest.py.
User Story
As a user, I want to upload images and PDFs to the knowledge base, so that the AI can automatically caption images and parse PDFs, and summarize their contents.
Context & Motivation
The single-file API (/api/v1/ingest) supports vision processing, but the knowledge base upload process uses the batch ingestion endpoint (POST /api/v1/ingest/sync), which is currently missing this logic.
As a result:
When the batch endpoint encounters a Base64-encoded image or PDF, it skips decoding.
It embeds the raw Base64 string instead of parsing the file.
Summarization fails because the AI cannot interpret raw Base64.
This issue adds the missing Base64 decoding and vision captioning logic to the batch endpoint.
Acceptance Criteria
Given a batch ingestion request containing an image or PDF artifact, when _assemble_content is called, then it should not prepend # {title} to the base64 string, allowing it to be decoded cleanly.
Given a parsed chunk of kind image, when _ingest_one processes it, then it should decode the Base64 payload back to bytes.
Given decoded image bytes, when the AI processes the chunk, then it should call llm.caption_image to generate a rich text description of the image.
Given a generated image caption, when the chunk is saved, then it should replace the original Base64 payload with the generated text caption.
Given a PDF artifact, when _ingest_one is preparing the content_bytes to pass to parse(), it should decode the Base64 payload into raw binary bytes so pdf_parser can read it.
Edge case: Given a vision model outage (e.g., LLMUnavailableError), when the captioning fails, then the chunk should gracefully log a warning and mark the artifact as failed so it can be retried later.
Technical Notes
Affected Component: ingest_run.py in sprintstart-ai.
Implementation hint:
In _assemble_content, check if artifact.mime.startswith("image/") or equals application/pdf. If so, return body_text directly to avoid prepending the title.
In _ingest_one, if artifact.mime == "application/pdf", Base64 decode content_bytes back into raw binary before passing it to parse().
In _ingest_one, after parsing chunks, iterate over them and check if chunk.kind == "image". Apply the base64.b64decode and llm.caption_image(image_bytes) logic exactly as it is already implemented in ingest.py.