diff --git a/tools/voyageai_batch_api/voyageai_batch_api_openai_compatibility.ipynb b/tools/voyageai_batch_api/voyageai_batch_api_openai_compatibility.ipynb new file mode 100644 index 00000000..cc6b8e37 --- /dev/null +++ b/tools/voyageai_batch_api/voyageai_batch_api_openai_compatibility.ipynb @@ -0,0 +1,533 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "SpjKsFOxhp2c" + }, + "source": [ + "# Voyage AI Batch API - OpenAI SDK compatibility and migration guide" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mongodb-developer/GenAI-Showcase/blob/main/notebooks/tooling/voyageai_batch_api_openai_compatibility.ipynb)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4QALqYT0hypz" + }, + "source": [ + "You can fully manage batch job lifecycles for Voyage AI using the OpenAI SDK (Python or TypeScript/JavaScript). Both the Files API and the Batch API follow OpenAI’s specification, so migrating is straightforward. Switching from an OpenAI model to a Voyage AI model requires only minor code updates.\n", + "\n", + "This guide explains how to:\n", + "\n", + "* Convert an existing OpenAI batch input file to a Voyage AI-compatible format\n", + "* Use the OpenAI Python SDK to interact with Voyage AI Files and Batch API" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "T12efBVzkyvc" + }, + "outputs": [], + "source": [ + "%pip install -q openai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "nBnr4Ttgjpx9" + }, + "source": [ + "## Batch input files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "EAjmzhd2juUQ" + }, + "source": [ + "Below is an example of requests following OpenAI's batch input format:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "aCyuhT4Ehyca" + }, + "outputs": [], + "source": [ + "# {\"custom_id\": \"request-1\", \"method\": \"POST\", \"url\": \"/v1/embeddings\", \"body\": {\"model\": \"text-embedding-3-small\", \"input\": \"This is a banana.\"}}\n", + "# {\"custom_id\": \"request-2\", \"method\": \"POST\", \"url\": \"/v1/embeddings\", \"body\": {\"model\": \"text-embedding-3-small\", \"input\": \"Why is the sky blue?\"}}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SbYoP6AekC44" + }, + "source": [ + "OpenAI treats each line as an independent request and requires specifying the endpoint and model for every line. In contrast, Voyage AI defines the endpoint and model at the batch level. You can use the helper function below to convert your existing OpenAI batch input files into Voyage AI–compatible ones." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "EGXegWQphUo4" + }, + "outputs": [], + "source": [ + "openai_requests = \"\"\"{\"custom_id\": \"request-1\", \"method\": \"POST\", \"url\": \"/v1/embeddings\", \"body\": {\"model\": \"text-embedding-3-small\", \"input\": \"This is a banana.\"}}\n", + "{\"custom_id\": \"request-2\", \"method\": \"POST\", \"url\": \"/v1/embeddings\", \"body\": {\"model\": \"text-embedding-3-small\", \"input\": \"Why is the sky blue?\"}}\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "TzoYc7cuCgER" + }, + "outputs": [], + "source": [ + "import json\n", + "from pathlib import Path\n", + "\n", + "\n", + "def convert_requests(source, output_file_name):\n", + " \"\"\"Convert OpenAI-style embedding JSONL to VoyageAI JSONL.\n", + " `source` can be a file path or a JSONL string.\n", + " `output_file_name` is the file to write the converted JSONL to.\n", + " \"\"\"\n", + " if Path(source).is_file():\n", + " with open(source, encoding=\"utf-8\") as f:\n", + " lines = f.readlines()\n", + " else:\n", + " lines = source.strip().splitlines()\n", + "\n", + " output = []\n", + " for idx, line in enumerate(lines, 1):\n", + " if not line.strip():\n", + " continue\n", + " data = json.loads(line)\n", + "\n", + " custom_id = data.get(\"custom_id\", f\"request-{idx}\")\n", + " inp = data[\"body\"][\"input\"]\n", + "\n", + " if isinstance(inp, str):\n", + " input_list = [inp]\n", + " elif isinstance(inp, list):\n", + " input_list = [str(x) for x in inp]\n", + " else:\n", + " raise ValueError(f\"Unsupported input type for {custom_id}: {type(inp)}\")\n", + "\n", + " new_data = {\"custom_id\": custom_id, \"body\": {\"input\": input_list}}\n", + " output.append(json.dumps(new_data, ensure_ascii=False))\n", + "\n", + " with open(output_file_name, \"w\", encoding=\"utf-8\") as f:\n", + " f.write(\"\\n\".join(output))\n", + "\n", + " return output_file_name" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1t6_1y4P0OiJ" + }, + "source": [ + "Let's use the function above to convert the OpenAI-formatted batch input file to a Voyage AI-formatted input file." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "PmrhmP5GJKAe" + }, + "outputs": [], + "source": [ + "# Convert OpenAI-formatted requests into Voyage AI-compatible ones\n", + "input_file_name = \"batch_input_file.jsonl\"\n", + "voyageai_requests_file = convert_requests(openai_requests, input_file_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "W0uiF0UglIgY" + }, + "source": [ + "Instantiate an OpenAI client with your Voyage API key and by providing the base URL:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "id": "lvitRxBMnrvS" + }, + "outputs": [], + "source": [ + "from openai import OpenAI\n", + "\n", + "client = OpenAI(api_key=\"your_voyage_api_key\", base_url=\"https://api.voyageai.com/v1\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "S57KcHH-3QmD" + }, + "source": [ + "# Upload file\n", + "\n", + "Let's upload the batch input file to Voyage AI's Files API." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Xoll_V8jhn-9", + "outputId": "cf6fed0a-530e-4843-bb53-a2faa973b705" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "FileObject(id='file-7vT5tYiajeqrFpsqCT8fbxCp46vwf8R4Qwdh8', bytes=140, created_at='2025-12-01T19:18:23.455349+00:00', filename='batch_input_file.jsonl', object='file', purpose='batch', status=None, expires_at='2025-12-31T19:18:23.455349+00:00', status_details=None)\n" + ] + } + ], + "source": [ + "# Upload file\n", + "batch_input_file = client.files.create(\n", + " file=open(input_file_name, \"rb\"), purpose=\"batch\"\n", + ")\n", + "print(batch_input_file)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BZt48Y4Gnwoy" + }, + "source": [ + "The file has been uploaded using the Files API. You can list all the files available with:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OAUe9bSTn2fp", + "outputId": "e3f376a2-f848-48c9-bb02-d9e65e804219" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[FileObject(id='file-7vT5tYiajeqrFpsqCT8fbxCp46vwf8R4Qwdh8', bytes=140, created_at='2025-12-01T19:18:23.455349+00:00', filename='batch_input_file.jsonl', object='file', purpose='batch', status=None, expires_at='2025-12-31T19:18:23.455349+00:00', status_details=None)]\n" + ] + } + ], + "source": [ + "# Listing files\n", + "files = client.files.list().data\n", + "print(files)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dWOAOpohnZZZ" + }, + "source": [ + "# Create batch job" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9Ihiy3WKnfrE" + }, + "source": [ + "The Voyage AI Batch API only requires specifying the model name through the extra_body attribute of the OpenAI client. Additional parameters, such as dimensionality or output data type, can also be provided.\n", + "\n", + "Let's create the batch by specifying the previously uploaded file ID, the endpoint (/v1/embeddings or /v1/rerank), the completion window, and any additional parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "C2Yd7YEdmCw_", + "outputId": "adb00b38-c6f1-44e8-d89a-7e56828dbe40" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Batch(id='batch-7vT5tYo5G6MNA3N1PMKrgMqNMErVVkmBNJq9U', completion_window='12h', created_at='2025-12-01T19:18:49.309117+00:00', endpoint='/v1/embeddings', input_file_id='file-7vT5tYiajeqrFpsqCT8fbxCp46vwf8R4Qwdh8', object='batch', status='validating', cancelled_at=None, cancelling_at=None, completed_at=None, error_file_id=None, errors=None, expired_at=None, expires_at=None, failed_at=None, finalizing_at=None, in_progress_at=None, metadata=None, model='voyage-3.5', output_file_id=None, request_counts=BatchRequestCounts(completed=0, failed=0, total=0), usage=None, expected_completion_at='2025-12-02T07:18:49.309117+00:00')\n" + ] + } + ], + "source": [ + "# Create batch\n", + "batch_input_file_id = batch_input_file.id\n", + "batch = client.batches.create(\n", + " input_file_id=batch_input_file_id,\n", + " endpoint=\"/v1/embeddings\",\n", + " completion_window=\"12h\",\n", + " extra_body={\n", + " \"request_params\": {\n", + " \"model\": \"voyage-3.5\",\n", + " \"input_type\": \"document\",\n", + " \"output_dimension\": 512,\n", + " \"output_dtype\": \"uint8\",\n", + " }\n", + " },\n", + ")\n", + "print(batch)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "20J3pHDSnTIy" + }, + "source": [ + "# Checking batch status\n", + "\n", + "Once the batch job has been submitted, you can check its status as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rhkpqp4rp1Sa", + "outputId": "f8842513-b9d2-4825-f523-09f15a188fb7" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "completed\n" + ] + } + ], + "source": [ + "batch_job = client.batches.retrieve(batch.id)\n", + "print(batch_job.status)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7IZKluX1UV0h" + }, + "source": [ + "# Retrieve results\n", + "\n", + "A batch job moves through several phases such as validation and in_progress, then eventually ends in a final state: completed, failed, canceled, or expired. Before you can retrieve the results, you must wait for the job to reach a terminal state so the output file and any optional error file are available. In the example below, we wait for the job to finish and, if it completes successfully, download the output files with the results." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4W-Lw0S_UYLs", + "outputId": "1b255a08-1be7-4306-f8c1-f30067a16a1a" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Output file saved: output_file.jsonl\n" + ] + } + ], + "source": [ + "TERMINAL_STATUSES = {\"completed\", \"failed\", \"cancelled\", \"expired\"}\n", + "\n", + "batch_id = batch.id # or set manually\n", + "\n", + "batch_job = client.batches.retrieve(batch_id)\n", + "\n", + "# Check batch job status\n", + "if batch_job.status in TERMINAL_STATUSES:\n", + " if batch_job.status == \"completed\":\n", + " output_file_id = batch_job.output_file_id\n", + " result = client.files.content(output_file_id).content\n", + "\n", + " output_file_name = \"output_file.jsonl\"\n", + " with open(output_file_name, \"wb\") as f:\n", + " f.write(result)\n", + "\n", + " print(f\"Output file saved: {output_file_name}\")\n", + " else:\n", + " print(f\"Batch ended with status: {batch_job.status}\")\n", + "else:\n", + " print(f\"Batch job is still in progress. Current status: {batch_job.status}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4_HyVnCNXPUg" + }, + "source": [ + "# Check results\n", + "\n", + "Let's take a look at the output file and check the results:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Cq7R-4-gXmVA", + "outputId": "4dd7f4e2-775c-4254-f6ce-8c982b91886b" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"custom_id\": \"request-1\", \"response\": {\"status_code\": 200, \"body\": {\"object\": \"list\", \"data\": [{\"object\": \"embedding\", \"embedding\": [114, 147, 126, 152, 131, 108, 88, 112, 119, 133, 177, 148, 116, 137, 104, 143, 132, 152, 132, 127, 119, 114, 81, 138, 153, 123, 135, 101, 104, 113, 123, 97, 150, 96, 111, 146, 120, 115, 147, 131, 142, 87, 113, 137, 118, 114, 105, 158, 99, 143, 113, 177, 131, 90, 148, 117, 116, 109, 133, 139, 115, 104, 127, 104, 160, 127, 82, 125, 106, 134, 146, 144, 112, 99, 119, 135, 125, 115, 121, 100, 138, 121, 151, 120, 161, 124, 78, 141, 130, 123, 92, 140, 126, 127, 134, 74, 147, 84, 114, 131, 133, 99, 112, 110, 84, 137, 128, 109, 101, 138, 108, 127, 123, 103, 146, 134, 108, 127, 148, 103, 130, 111, 119, 171, 102, 144, 131, 87, 146, 125, 139, 140, 150, 128, 139, 104, 121, 104, 77, 141, 119, 135, 135, 112, 127, 129, 102, 121, 120, 127, 177, 155, 133, 111, 94, 112, 106, 137, 127, 108, 96, 163, 147, 104, 111, 109, 142, 127, 100, 162, 148, 116, 148, 139, 135, 159, 118, 155, 118, 163, 98, 82, 106, 144, 129, 151, 130, 109, 122, 153, 136, 134, 130, 93, 129, 179, 117, 114, 136, 98, 101, 117, 137, 160, 103, 138, 104, 98, 109, 156, 93, 156, 120, 122, 115, 149, 153, 118, 133, 81, 108, 138, 179, 118, 156, 115, 131, 133, 102, 124, 124, 142, 112, 140, 111, 126, 114, 94, 127, 132, 125, 116, 120, 152, 115, 149, 133, 134, 127, 103, 107, 155, 100, 136, 113, 132, 118, 148, 136, 129, 138, 163, 133, 129, 104, 117, 147, 130, 123, 106, 120, 137, 134, 122, 138, 143, 145, 152, 156, 111, 146, 146, 126, 118, 124, 158, 155, 126, 131, 124, 133, 112, 114, 147, 130, 107, 137, 172, 105, 146, 141, 147, 144, 117, 144, 133, 170, 142, 142, 127, 123, 110, 119, 116, 124, 174, 153, 95, 138, 107, 131, 130, 134, 142, 143, 137, 160, 141, 111, 141, 99, 136, 138, 140, 151, 111, 111, 139, 123, 118, 176, 116, 138, 111, 99, 178, 133, 126, 126, 128, 120, 125, 119, 144, 79, 123, 148, 125, 121, 117, 92, 127, 160, 101, 113, 113, 126, 140, 138, 148, 127, 121, 120, 137, 125, 117, 132, 138, 96, 125, 123, 133, 129, 136, 113, 122, 111, 119, 153, 128, 104, 94, 112, 145, 124, 153, 115, 163, 157, 120, 126, 108, 124, 98, 89, 104, 156, 103, 84, 113, 131, 83, 126, 146, 148, 127, 118, 109, 132, 138, 118, 121, 110, 156, 118, 139, 151, 126, 125, 116, 119, 125, 107, 132, 118, 101, 131, 127, 158, 105, 118, 127, 144, 122, 116, 109, 133, 142, 95, 106, 83, 133, 117, 127, 130, 109, 109, 129, 88, 121, 133, 140, 122, 121, 127, 119, 128, 124, 170, 114, 138, 147, 142, 139, 134, 134, 148, 131, 153, 128, 144, 145, 130, 134, 123, 141, 121, 102, 117, 134, 119, 120, 102, 111, 136, 139, 120, 106, 111, 132, 126, 144, 94, 126, 133, 91, 130, 140, 154, 128, 125, 122], \"index\": 0}], \"model\": \"voyage-3.5\", \"usage\": {\"total_tokens\": 4}}}, \"error\": null}\n", + "{\"custom_id\": \"request-2\", \"response\": {\"status_code\": 200, \"body\": {\"object\": \"list\", \"data\": [{\"object\": \"embedding\", \"embedding\": [114, 152, 127, 136, 161, 122, 117, 104, 90, 126, 158, 154, 99, 130, 108, 118, 126, 133, 117, 169, 123, 105, 85, 148, 154, 112, 148, 87, 101, 140, 119, 106, 141, 113, 112, 129, 137, 107, 145, 142, 150, 111, 139, 125, 122, 111, 113, 154, 101, 144, 112, 152, 118, 112, 104, 128, 130, 104, 136, 152, 107, 119, 125, 110, 178, 140, 86, 114, 109, 144, 150, 129, 98, 112, 106, 109, 113, 110, 136, 113, 123, 106, 160, 126, 137, 138, 97, 140, 128, 106, 99, 132, 142, 130, 129, 95, 134, 88, 125, 119, 131, 94, 130, 115, 77, 131, 139, 118, 104, 134, 118, 100, 108, 138, 119, 117, 127, 148, 167, 95, 142, 103, 136, 162, 99, 137, 109, 72, 126, 140, 115, 144, 144, 136, 124, 121, 113, 110, 90, 134, 106, 149, 136, 118, 112, 130, 118, 121, 133, 124, 152, 146, 130, 119, 109, 132, 129, 132, 138, 120, 94, 147, 127, 109, 108, 122, 121, 126, 114, 165, 150, 118, 144, 164, 127, 140, 119, 133, 132, 163, 118, 95, 98, 131, 111, 132, 143, 138, 120, 154, 140, 103, 149, 103, 131, 171, 125, 112, 140, 123, 98, 131, 146, 167, 108, 104, 107, 101, 149, 150, 105, 162, 162, 108, 90, 165, 165, 91, 117, 121, 105, 124, 179, 109, 137, 101, 161, 116, 109, 98, 114, 135, 138, 155, 103, 133, 101, 109, 121, 116, 123, 113, 125, 144, 95, 120, 151, 133, 125, 128, 150, 177, 120, 127, 102, 133, 112, 132, 142, 135, 137, 147, 129, 117, 106, 113, 135, 129, 153, 133, 113, 116, 145, 137, 137, 117, 119, 147, 126, 140, 140, 170, 128, 112, 95, 147, 171, 125, 108, 138, 131, 130, 114, 159, 130, 130, 146, 160, 104, 157, 139, 153, 151, 130, 160, 145, 161, 145, 125, 128, 101, 119, 117, 116, 125, 142, 174, 110, 140, 106, 126, 127, 96, 141, 137, 131, 151, 140, 111, 159, 115, 151, 122, 128, 131, 126, 128, 138, 128, 121, 160, 113, 131, 134, 120, 180, 112, 140, 124, 148, 127, 131, 89, 144, 92, 109, 146, 138, 131, 112, 88, 111, 154, 92, 114, 130, 119, 143, 134, 164, 126, 112, 122, 138, 131, 119, 143, 137, 111, 130, 127, 145, 130, 143, 100, 134, 130, 110, 145, 143, 112, 113, 122, 130, 127, 137, 105, 140, 155, 106, 137, 104, 132, 133, 102, 120, 146, 116, 107, 105, 148, 102, 122, 149, 136, 130, 98, 141, 145, 101, 112, 122, 128, 155, 134, 145, 166, 136, 123, 95, 110, 123, 116, 156, 110, 138, 119, 135, 155, 130, 127, 155, 118, 129, 105, 109, 128, 147, 97, 98, 85, 148, 112, 119, 152, 107, 125, 144, 86, 142, 122, 126, 119, 116, 103, 104, 107, 118, 142, 129, 140, 159, 135, 141, 124, 139, 151, 143, 156, 147, 134, 151, 136, 132, 119, 129, 123, 102, 131, 128, 123, 137, 119, 114, 147, 157, 129, 87, 107, 121, 139, 142, 90, 101, 142, 113, 111, 136, 147, 126, 93, 96], \"index\": 0}], \"model\": \"voyage-3.5\", \"usage\": {\"total_tokens\": 5}}}, \"error\": null}\n" + ] + } + ], + "source": [ + "# Display the first 2 rows of the saved output file\n", + "with open(\"output_file.jsonl\", encoding=\"utf-8\") as f:\n", + " for i, line in enumerate(f):\n", + " if i >= 2:\n", + " break\n", + " print(line.rstrip())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "r79v8QRKVGyB" + }, + "source": [ + "# Listing all batches\n", + "\n", + "You can list all the batches as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "i1hCh4I5oxN3", + "outputId": "2bd33ccc-0d8b-47ae-b7a4-e6d368c29ffe" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "batch-7vT5tYo5G6MNA3N1PMKrgMqNMErVVkmBNJq9U\n" + ] + } + ], + "source": [ + "# Listing all batches\n", + "batches = client.batches.list().data\n", + "for batch in batches:\n", + " print(batch.id)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "huU7y3O-ozNN" + }, + "source": [ + "# Not supported" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "up1GpX_lo7BI" + }, + "source": [ + "Some methods from the OpenAI SDK are not available with Voyage AI, such as:\n", + "- Listing models - client.models.list()\n", + "- Retrieving a model - client.models.retrieve(\"xxx\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "w2YNCy12pClx" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": {} + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}