fix(models): handle binary file downloads in tool execution (#190)

jerann · claude · web-flow · commit 8983c68dd2fd · 2026-06-03T15:44:47.000+02:00
* fix(models): handle binary file downloads in tool execution

Download actions (e.g. googledrive_unified_download_file) serve raw binary
with the file's own MIME type and a Content-Disposition header, so
StackOneTool.execute() calling response.json() unconditionally raised
UnicodeDecodeError on the first non-UTF-8 byte.

Branch on Content-Type: parse JSON only for JSON media types (application/json
and +json suffixes); otherwise return the file as
{content: bytes, content_type, status_code, headers, file_name}, matching the
StackOne generated SDKs' download response shape. file_name is parsed from
Content-Disposition (incl. RFC 5987 filename*). Existing JSON behavior is
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) &lt;noreply@anthropic.com&gt;

* docs(readme): document file-download return shape from tool execution

Add a "File Downloads" section covering the bytes-plus-metadata dict that
execute()/call() return for non-JSON responses, the Content-Type detection
rule, and the not-JSON-serializable caveat for LLM-facing re-serialization.

Co-Authored-By: Claude Opus 4.8 (1M context) &lt;noreply@anthropic.com&gt;

* fix(models): honour declared charset in RFC 5987 filename* parsing

The Content-Disposition filename* branch captured the charset in its
regex but discarded it, so unquote() always decoded with UTF-8. RFC 5987
also permits ISO-8859-1, so an ISO-8859-1 percent-encoded filename
decoded to mojibake despite the helper claiming RFC 5987 support.

- Decode the extended value with its declared charset; fall back to
  UTF-8 for an unknown or empty charset label (LookupError) instead of
  raising.
- Strip surrounding quotes off non-conformant quoted filename* values.
- Add tests for ISO-8859-1, an unknown charset, and a quoted filename*.

Co-Authored-By: Claude Opus 4.8 (1M context) &lt;noreply@anthropic.com&gt;

---------

Co-authored-by: Claude Opus 4.8 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/README.md b/README.md
@@ -114,6 +114,35 @@ tools = toolset.fetch_tools(providers=["hibob"])
   - Glob pattern: `["*_list_employees"]` matches all tools ending with `_list_employees`
   - Provider prefix: `["workday_*"]` matches all Workday tools
 
+## File Downloads
+
+Actions that return a file — e.g. `googledrive_unified_download_file`, `documents_download_file`, any `*_unified_download_file` — resolve to **raw bytes plus metadata**, not parsed JSON. The SDK decides this from the response `Content-Type`: a JSON content type is parsed as before; anything else is treated as a file download. This applies to both `tool.execute()` and `tool.call()`.
+
+```python
+tools = toolset.fetch_tools(actions=["googledrive_*"], account_ids=[account_id])
+download = tools.get_tool("googledrive_unified_download_file")
+
+result = download.execute({"id": "file-id"})
+
+# `result` is a dict describing the file — write the bytes straight to disk:
+with open(result["file_name"] or "download.bin", "wb") as f:
+    f.write(result["content"])
+```
+
+The returned dict:
+
+| Key            | Type          | Description                                                                                  |
+| -------------- | ------------- | -------------------------------------------------------------------------------------------- |
+| `content`      | `bytes`       | Raw file bytes. **Not JSON-serializable** — see the caveat below.                            |
+| `content_type` | `str`         | The file's MIME type (e.g. `application/pdf`), or `application/octet-stream` if unspecified. |
+| `status_code`  | `int`         | HTTP status of the download response.                                                        |
+| `headers`      | `dict`        | Response headers.                                                                            |
+| `file_name`    | `str \| None` | Filename from the `Content-Disposition` header (handles RFC 5987 `filename*`), else `None`.  |
+
+> **Caveat:** `content` holds raw bytes, which are not JSON-serializable. If you forward tool results to an LLM — or anywhere that re-serializes them to JSON — handle or strip the `content` key (for example, base64-encode it on the LLM-facing path).
+
+JSON responses are unchanged: any action returning `application/json` (or a `…+json` type) is parsed and returned as a dict exactly as before.
+
 ## Implicit Feedback (Beta)
 
 The Python SDK can emit implicit behavioral feedback to LangSmith so you can triage low-quality tool results without manually tagging runs.
diff --git a/stackone_ai/models.py b/stackone_ai/models.py
@@ -3,11 +3,12 @@
 import base64
 import json
 import logging
+import re
 from collections.abc import Sequence
 from datetime import datetime, timezone
 from enum import Enum
 from typing import TYPE_CHECKING, Annotated, Any, ClassVar, TypeAlias, cast
-from urllib.parse import quote
+from urllib.parse import quote, unquote
 
 import httpx
 from langchain_core.tools import BaseTool
@@ -57,6 +58,47 @@ def validate_method(v: str) -> str:
     return method
 
 
+def _is_json_content_type(content_type: str) -> bool:
+    """Whether a response body should be parsed as JSON based on its Content-Type.
+
+    Only genuine JSON media types are parsed (``application/json`` and structured
+    suffixes such as ``application/problem+json``). Anything else - including a
+    missing Content-Type - is treated as opaque content (a file download), so the
+    raw bytes are returned instead of being force-decoded as UTF-8/JSON. This mirrors
+    how the StackOne generated SDKs default unknown bodies to ``application/octet-stream``.
+    """
+    media_type = content_type.split(";", 1)[0].strip().lower()
+    return media_type == "application/json" or media_type.endswith("+json")
+
+
+def _filename_from_content_disposition(value: str | None) -> str | None:
+    """Extract the filename from a Content-Disposition header value, if present.
+
+    Handles both the plain ``filename="example.pdf"`` form and the RFC 5987 extended
+    ``filename*=UTF-8''example%20file.pdf`` form (which takes precedence when present).
+    The extended form is percent-decoded using its declared charset (RFC 5987 permits
+    both ``UTF-8`` and ``ISO-8859-1``); an unknown or empty charset falls back to UTF-8.
+    """
+    if not value:
+        return None
+    extended = re.search(r"filename\*\s*=\s*([^']*)'[^']*'([^;]+)", value, re.IGNORECASE)
+    if extended:
+        charset = extended.group(1).strip() or "utf-8"
+        encoded = extended.group(2).strip().strip('"')
+        try:
+            return unquote(encoded, encoding=charset, errors="replace") or None
+        except LookupError:
+            # Unrecognised charset label - decode as UTF-8 rather than failing.
+            return unquote(encoded, encoding="utf-8", errors="replace") or None
+    quoted = re.search(r'filename\s*=\s*"([^"]*)"', value, re.IGNORECASE)
+    if quoted:
+        return quoted.group(1).strip() or None
+    bare = re.search(r"filename\s*=\s*([^;]+)", value, re.IGNORECASE)
+    if bare:
+        return bare.group(1).strip().strip('"') or None
+    return None
+
+
 class ExecuteConfig(BaseModel):
     """Configuration for executing a tool against an API endpoint"""
 
@@ -206,7 +248,14 @@ def execute(
             options: Execution options (e.g. feedback metadata)
 
         Returns:
-            API response as dict
+            For JSON responses, the parsed API response as a dict.
+
+            For file downloads (any non-JSON Content-Type, e.g. a
+            ``documents_download_file`` action), a dict describing the file:
+            ``{"content": <bytes>, "content_type": str, "status_code": int,
+            "headers": dict, "file_name": str | None}``. Note ``content`` holds
+            the raw bytes and is therefore not JSON-serializable - callers that
+            re-serialize tool results (e.g. for an LLM) should handle this key.
 
         Raises:
             StackOneAPIError: If the API request fails
@@ -257,9 +306,23 @@ def execute(
             response_status = response.status_code
             response.raise_for_status()
 
-            result = response.json()
-            result_payload = cast(JsonDict, result) if isinstance(result, dict) else {"result": result}
-            return result_payload
+            content_type = response.headers.get("content-type", "")
+            if _is_json_content_type(content_type):
+                result = response.json()
+                result_payload = cast(JsonDict, result) if isinstance(result, dict) else {"result": result}
+                return result_payload
+
+            # Non-JSON bodies are file downloads (e.g. documents_download_file), which the
+            # API serves as raw binary with the file's own MIME type and a Content-Disposition
+            # header. Return the bytes plus metadata rather than forcing a JSON/UTF-8 decode.
+            # The shape mirrors the StackOne generated SDKs' download response.
+            return {
+                "content": response.content,
+                "content_type": content_type or "application/octet-stream",
+                "status_code": response.status_code,
+                "headers": dict(response.headers),
+                "file_name": _filename_from_content_disposition(response.headers.get("content-disposition")),
+            }
 
         except json.JSONDecodeError as exc:
             status = "error"
diff --git a/tests/test_tool_calling.py b/tests/test_tool_calling.py
@@ -7,7 +7,12 @@
 import respx
 
 from stackone_ai import StackOneTool
-from stackone_ai.models import ExecuteConfig, ToolParameters
+from stackone_ai.models import (
+    ExecuteConfig,
+    ToolParameters,
+    _filename_from_content_disposition,
+    _is_json_content_type,
+)
 from stackone_ai.toolset import _StackOneRpcTool
 from tests.conftest import TEST_BASE_URL
 
@@ -332,3 +337,183 @@ def test_extract_record_with_non_dict(self, rpc_tool):
         assert rpc_tool._extract_record("string") is None
         assert rpc_tool._extract_record(123) is None
         assert rpc_tool._extract_record(None) is None
+
+
+class TestBinaryDownloadResponse:
+    """File-download actions return raw bytes + metadata instead of failing on JSON parsing.
+
+    The StackOne API serves file downloads as raw binary with the file's own MIME type
+    (e.g. application/pdf) and a Content-Disposition header - never JSON. The returned
+    shape mirrors the StackOne generated SDKs' download response (content + content_type +
+    status_code + headers), with content as raw bytes (the Python analog of the Java
+    client's byte[] body / the TypeScript client's response stream).
+    """
+
+    @respx.mock
+    def test_binary_response_returns_content_dict(self, mock_tool):
+        """A non-JSON (binary) body is returned as bytes + metadata, not JSON-parsed."""
+        # Leading bytes of a real PDF; the 0xc4 byte is invalid UTF-8 and is exactly
+        # what makes the unconditional response.json() raise UnicodeDecodeError.
+        pdf_bytes = b"%PDF-1.4\n%\xc4\xe5\xf2\xe5\xeb\xa7\xf3\xa0\xd0\xc4\xc6\n1 0 obj\n"
+        respx.post("https://api.example.com/test").mock(
+            return_value=httpx.Response(
+                200,
+                headers={
+                    "content-type": "application/pdf",
+                    "content-disposition": 'attachment; filename="download.pdf"',
+                },
+                content=pdf_bytes,
+            )
+        )
+
+        result = mock_tool.execute({"name": "report", "value": 1})
+
+        assert result["content"] == pdf_bytes
+        assert result["content_type"] == "application/pdf"
+        assert result["status_code"] == 200
+        assert result["file_name"] == "download.pdf"
+        assert result["headers"]["content-type"] == "application/pdf"
+
+    @respx.mock
+    def test_rpc_download_action_returns_content_dict(self):
+        """The RPC download path (e.g. googledrive_unified_download_file) returns bytes.
+
+        Reproduces the reported failure: a download action invoked through /actions/rpc
+        previously raised UnicodeDecodeError because the binary body was JSON-parsed.
+        """
+        parameters = ToolParameters(
+            type="object",
+            properties={"id": {"type": "string", "description": "File ID"}},
+        )
+        tool = _StackOneRpcTool(
+            name="googledrive_unified_download_file",
+            description="Download a file",
+            parameters=parameters,
+            api_key="test_api_key",
+            base_url=TEST_BASE_URL,
+            account_id="test_account",
+        )
+
+        rtf_bytes = b"{\\rtf1\\ansi\\ansicpg1252\\\xc4\xe5 hello}"
+        respx.post(f"{TEST_BASE_URL}/actions/rpc").mock(
+            return_value=httpx.Response(
+                200,
+                headers={
+                    "content-type": "application/rtf",
+                    "content-disposition": 'attachment; filename="download.rtf"',
+                },
+                content=rtf_bytes,
+            )
+        )
+
+        result = tool.execute({"path": {"id": "file-123"}})
+
+        assert result["content"] == rtf_bytes
+        assert result["content_type"] == "application/rtf"
+        assert result["file_name"] == "download.rtf"
+
+    @respx.mock
+    def test_octet_stream_without_filename(self, mock_tool):
+        """A binary body with no Content-Disposition still returns content with file_name=None."""
+        blob = b"\x00\x01\x02\xc4\xff\xfe"
+        respx.post("https://api.example.com/test").mock(
+            return_value=httpx.Response(
+                200,
+                headers={"content-type": "application/octet-stream"},
+                content=blob,
+            )
+        )
+
+        result = mock_tool.execute({})
+
+        assert result["content"] == blob
+        assert result["content_type"] == "application/octet-stream"
+        assert result["file_name"] is None
+
+    @respx.mock
+    def test_json_response_still_parsed(self, mock_tool):
+        """Regression guard: JSON responses are unchanged - parsed to a dict, not wrapped."""
+        respx.post("https://api.example.com/test").mock(
+            return_value=httpx.Response(200, json={"id": "123", "ok": True})
+        )
+
+        result = mock_tool.execute({"name": "x", "value": 1})
+
+        assert result == {"id": "123", "ok": True}
+        assert "content" not in result
+
+    @respx.mock
+    def test_json_with_charset_param_still_parsed(self, mock_tool):
+        """A JSON Content-Type with parameters (charset) is still parsed as JSON."""
+        respx.post("https://api.example.com/test").mock(
+            return_value=httpx.Response(
+                200,
+                headers={"content-type": "application/json; charset=utf-8"},
+                content=b'{"ok": true}',
+            )
+        )
+
+        result = mock_tool.execute({})
+
+        assert result == {"ok": True}
+
+    @respx.mock
+    def test_missing_content_type_returns_bytes(self, mock_tool):
+        """A body with no Content-Type is treated as opaque content (bytes), not JSON.
+
+        Pins the deliberate contract: the SDK trusts Content-Type to decide JSON vs
+        file, so an absent Content-Type is returned as raw bytes rather than risking
+        a UTF-8/JSON decode of binary. (StackOne always labels JSON as application/json.)
+        """
+        blob = b"\xff\xd8\xff\xe0\x00\x10JFIF"  # JPEG magic bytes, no content-type
+        respx.post("https://api.example.com/test").mock(return_value=httpx.Response(200, content=blob))
+
+        result = mock_tool.execute({})
+
+        assert result["content"] == blob
+        assert result["content_type"] == "application/octet-stream"
+        assert result["file_name"] is None
+
+
+class TestResponseHelpers:
+    """Unit tests for the Content-Type and Content-Disposition helpers."""
+
+    @pytest.mark.parametrize(
+        ("content_type", "expected"),
+        [
+            ("application/json", True),
+            ("application/json; charset=utf-8", True),
+            ("APPLICATION/JSON", True),
+            ("application/problem+json", True),
+            ("application/vnd.api+json", True),
+            ("", False),
+            ("application/pdf", False),
+            ("application/octet-stream", False),
+            ("text/plain", False),
+            ("text/json-but-not-really", False),
+        ],
+    )
+    def test_is_json_content_type(self, content_type, expected):
+        assert _is_json_content_type(content_type) is expected
+
+    @pytest.mark.parametrize(
+        ("header", "expected"),
+        [
+            ('attachment; filename="download.pdf"', "download.pdf"),
+            ("attachment; filename=download.pdf", "download.pdf"),
+            ('inline; filename="my report.docx"', "my report.docx"),
+            # RFC 5987 extended form is percent-decoded and takes precedence.
+            ("attachment; filename=\"fallback.txt\"; filename*=UTF-8''na%C3%AFve.txt", "naïve.txt"),
+            # Non-UTF-8 charset is honoured: 0xA3 is "£" in ISO-8859-1, not UTF-8.
+            ("attachment; filename*=ISO-8859-1'en'%A3%20rates.txt", "£ rates.txt"),
+            # Unknown charset label falls back to UTF-8 instead of raising.
+            ("attachment; filename*=bogus-charset''%C2%A3.txt", "£.txt"),
+            # Non-conformant quoted extended value: surrounding quotes are stripped.
+            ("attachment; filename*=\"UTF-8''na%C3%AFve.txt\"", "naïve.txt"),
+            ("attachment", None),
+            (None, None),
+            ("", None),
+        ],
+    )
+    def test_filename_from_content_disposition(self, header, expected):
+        assert _filename_from_content_disposition(header) == expected
