Skip to content

Fix inconsistent handling of Builder gateway 500 envelope #3030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
builder-io-integration wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Fix inconsistent handling of Builder gateway 500 envelope #3030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d1b6866
fix: report the gateway 500 under one code and stop re-dispatching it
builderio-bot Aug 18, 2026
54c8ac0
Merge remote-tracking branch 'refs/remotes/origin/main' into ai_main_…
builderio-bot Aug 18, 2026
440df2d
fix: reject action schemas JSON cannot represent
builderio-bot Aug 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .changeset/gateway-internal-error-one-code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
"@agent-native/core": patch
---

Stop the Builder gateway's 500 envelope from ending chats three different ways, and stop emitting the malformed tool schema that provokes it.

The gateway answers 200 and then delivers its own internal-error envelope ("Sorry,
we ran into an issue processing your request. ERROR ID: …") as an in-stream frame,
so nothing structured reaches the layer that catches it. `runAgentLoopWithResume`
dropped straight to `internal_error` without asking the shared classifier, so one
upstream failure was persisted under three codes depending on which layer caught it
— `builder_gateway_internal_error`, `internal_error`, `unknown` — and only one of
those is on the client's recoverable list. Identical gateway failures therefore
ended some chats on the first attempt and sent others into a re-dispatch chain.
Both catch sites now classify the message first; `internal_error` stays the last
resort for a message nothing recognises.

That envelope is also no longer auto-recoverable at the run level. Measured across
three production databases, 7 turns reached it and 0 of 7 ever finished, across 97
runs — one turn burned 28 runs over 16 minutes on a single "Hey", with overlapping
concurrent runs on the same turn. The engine's in-request retry still covers the
genuinely transient case, and the recovery card keeps a deliberate Retry, because
its own copy ends with "Retry in a moment".

The fallback Zod-to-JSON-Schema converter read a literal's value from `def.value`,
which Zod v4 does not define — it stores `values`. Every literal therefore emitted
`{"type":"undefined"}`, a type keyword no JSON Schema dialect defines, and an
`enum` of `[null]` instead of the real value. Literals are most often a
discriminated union's discriminator, so ordinary action schemas were affected. The
Builder gateway validates only a tool's top-level `input_schema.type`, so an
invented type on a nested field passes validation, reaches the provider, and comes
back as the opaque ERROR ID envelope — on every retry, because the same malformed
schema is resent verbatim. Literal values are now read from `values`, and no
literal can emit a type outside the seven JSON Schema names.

An action schema carrying a literal that JSON cannot represent is now rejected when
the action is defined, rather than producing a broken request later. A `bigint`
literal made `JSON.stringify` throw while the request was being built, and
`undefined`, `NaN`, and `±Infinity` serialized to `null` — advertising a value the
Zod validator still rejects, so the schema and the validator disagreed with nothing
in the output left to detect it from. The check runs against the Zod def as well as
the emitted schema, because Zod's own converter turns `z.literal(NaN)` into
`const: null` before the emitted schema can be inspected.
255 changes: 168 additions & 87 deletions packages/core/src/action.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,174 @@ describe("defineAction schema mode — tool parameter JSON Schema", () => {
expect(params.properties.title.description).toBe("Form title");
});

// JSON Schema has exactly seven type names and JavaScript's `typeof` is not a
// way to spell them. The Builder gateway validates only a tool's TOP-LEVEL
// `input_schema.type`, so an invented type on a NESTED field passes validation,
// reaches the provider, and comes back as the gateway's opaque "ERROR ID" 500
// envelope — on every retry, because the same malformed schema is resent.
it("never emits a non-JSON-Schema type for a literal", () => {
const legalTypes = new Set([
"string",
"number",
"integer",
"boolean",
"object",
"array",
"null",
]);
const action = defineAction({
description: "literal types",
schema: z.object({
nothing: z.literal(null),
name: z.literal("fixed"),
count: z.literal(7),
ratio: z.literal(1.5),
flag: z.literal(true),
nested: z.object({ inner: z.literal(null) }),
items: z.array(z.literal(null)),
}),
run: async () => "ok",
});

// The fallback converter is what runs for a schema that exposes no
// standard-schema `jsonSchema` hook, so exercise that path explicitly rather
// than only Zod's own converter — the bug lived exclusively in the fallback.
const withoutJsonSchemaHook = <T>(schema: T): T => {
const standard = (schema as any)["~standard"];
(schema as any)["~standard"] = { ...standard, jsonSchema: undefined };
return schema;
};
const fallbackAction = defineAction({
description: "literal types via fallback converter",
schema: withoutJsonSchemaHook(
z.object({
kind: z.literal("weekly"),
nested: z.object({ inner: z.literal(3) }),
mode: z.union([z.literal("a"), z.literal("b")]),
}),
),
run: async () => "ok",
});

const illegal: string[] = [];
const walk = (node: unknown) => {
if (!node || typeof node !== "object") return;
if (Array.isArray(node)) {
node.forEach(walk);
return;
}
const record = node as Record<string, unknown>;
const type = record.type;
if (typeof type === "string" && !legalTypes.has(type)) illegal.push(type);
Object.values(record).forEach(walk);
};
walk(action.tool.parameters);
walk(fallbackAction.tool.parameters);

expect(illegal).toEqual([]);
const props = (action.tool.parameters as any).properties;
expect(props.nothing).toMatchObject({ type: "null" });
expect(props.name).toMatchObject({ type: "string" });
expect(props.flag).toMatchObject({ type: "boolean" });

// The fallback path must also carry the literal's VALUE. Reading the
// singular `def.value` returned undefined, which serialized to `[null]` and
// told the model the discriminator accepts null.
const fallbackProps = (fallbackAction.tool.parameters as any).properties;
expect(fallbackProps.kind).toMatchObject({
type: "string",
enum: ["weekly"],
});
expect(fallbackProps.nested.properties.inner).toMatchObject({
type: "integer",
enum: [3],
});
expect(fallbackProps.mode).toMatchObject({
type: "string",
enum: ["a", "b"],
});
});

// A tool schema only ever travels as JSON, so a value JSON cannot carry
// describes a contract no caller can satisfy. `bigint` makes `JSON.stringify`
// THROW while the request is being built; `undefined`/`NaN`/`±Infinity`
// serialize to `null`, advertising a value the Zod validator still rejects.
// Both are worse than failing at definition time.
it.each([
["a BigInt", z.literal(1n), "a BigInt"],
["undefined", z.literal(undefined as any), "undefined"],
["NaN", z.literal(NaN), "NaN"],
["negative Infinity", z.literal(-Infinity), "Infinity"],
])("rejects a literal of %s", (_label, literal, expected) => {
expect(() =>
defineAction({
description: "unrepresentable literal",
schema: z.object({ field: literal as any }),
run: async () => "ok",
}),
).toThrow(new RegExp(`field is a literal of ${expected}`));
});

// The value must be found wherever it hides, including the places a converter
// would otherwise launder it before anything could inspect the output.
it.each([
[
"nested in an object",
z.object({ outer: z.object({ field: z.literal(1n) }) }),
"outer.field",
],
["inside an array", z.object({ field: z.array(z.literal(1n)) }), "field"],
[
"behind optional",
z.object({ field: z.literal(NaN).optional() }),
"field",
],
[
"in a union branch",
z.object({ field: z.union([z.literal("s"), z.literal(NaN)]) }),
"field|1",
],
])("rejects an unrepresentable literal %s", (_label, schema, path) => {
let caught: any;
try {
defineAction({
description: "d",
schema: schema as any,
run: async () => "ok",
});
} catch (err) {
caught = err;
}
expect(caught?.errorCode).toBe("SCHEMA_NOT_JSON_REPRESENTABLE");
expect(caught.details.problems.join(";")).toContain(path);
});

// The failure this guards against was a throw during serialization, so assert
// on serialization itself rather than only on the object graph.
it("emits tool parameters that survive JSON.stringify", () => {
const action = defineAction({
description: "serializable",
schema: z.object({
title: z.string(),
count: z.number().int().optional(),
kind: z.literal("weekly"),
nothing: z.literal(null),
ratio: z.literal(1.5),
mode: z.union([z.literal("a"), z.literal("b")]),
choice: z.enum(["x", "y"]),
}),
run: async () => "ok",
});

expect(() => JSON.stringify(action.tool.parameters)).not.toThrow();
const roundTripped = JSON.parse(JSON.stringify(action.tool.parameters));
// A silent `null` here is the divergence case: it would mean a value was
// dropped or coerced on the way to JSON.
expect(JSON.stringify(roundTripped)).not.toContain('"const":null,"type"');
expect(roundTripped.properties.kind).toMatchObject({ const: "weekly" });
expect(roundTripped.properties.nothing).toMatchObject({ type: "null" });
});

it("strips the $schema key so the Claude API (draft 2020-12) does not reject it", () => {
const action = defineAction({
description: "with schema key",
Expand Down Expand Up @@ -389,93 +557,6 @@ describe("defineAction schema mode — tool parameter JSON Schema", () => {
expect(params.properties.cfg.default).toEqual({ propertyNames: "x" });
});

// OpenAI answers a `oneOf` anywhere in a function schema with
// "Invalid schema for function 'x': ... 'oneOf' is not permitted" and 400s
// the whole request before a token streams. Zod emits `oneOf` for every
// discriminated union, so this was 178k errors across 786 users over seven
// weeks from one action.
it("rewrites oneOf to anyOf so OpenAI does not reject the function schema", () => {
const action = defineAction({
description: "with a discriminated union",
schema: z.object({
operations: z.array(
z.discriminatedUnion("op", [
z.object({ op: z.literal("add"), panelId: z.string() }),
z.object({
op: z.literal("remove"),
panelIds: z.array(z.string()),
}),
]),
),
}),
run: async () => "ok",
});
const json = JSON.stringify(action.tool.parameters);
expect(json).not.toContain('"oneOf"');
expect(json).toContain('"anyOf"');
});

it("keeps every branch when rewriting a nested union", () => {
const action = defineAction({
description: "nested union",
schema: z.object({
outer: z.object({
inner: z.discriminatedUnion("kind", [
z.object({ kind: z.literal("a"), a: z.string() }),
z.object({ kind: z.literal("b"), b: z.string() }),
z.object({ kind: z.literal("c"), c: z.string() }),
]),
}),
}),
run: async () => "ok",
});
const params = action.tool.parameters as any;
const inner = params.properties.outer.properties.inner;
expect(inner.oneOf).toBeUndefined();
expect(inner.anyOf).toHaveLength(3);
});

// OpenAI rejects a schema position with no `type` — "schema must have a
// 'type' key" — and 400s the whole request, exactly like `oneOf` did. This
// surfaced only after the oneOf fix let the validator reach the next layer.
it("gives z.unknown() a typed value union so OpenAI accepts it", () => {
const action = defineAction({
description: "typeless field",
schema: z.object({ value: z.unknown() }),
run: async () => "ok",
});
const value = (action.tool.parameters as any).properties.value;
expect(Array.isArray(value.anyOf)).toBe(true);
expect(value.anyOf.map((b: any) => b.type)).toContain("string");
expect(value.anyOf.map((b: any) => b.type)).toContain("object");
});

it("types the value schema inside a record so nothing is left bare", () => {
const action = defineAction({
description: "record of unknown",
schema: z.object({ patch: z.record(z.string(), z.unknown()) }),
run: async () => "ok",
});
const patch = (action.tool.parameters as any).properties.patch;
expect(patch.type).toBe("object");
const extra = patch.additionalProperties;
if (extra && typeof extra === "object") {
expect(Array.isArray(extra.anyOf)).toBe(true);
}
});

// An enum carries its own shape; adding a value union would widen it.
it("leaves an enum-only schema alone", () => {
const action = defineAction({
description: "enum field",
schema: z.object({ mode: z.enum(["a", "b"]) }),
run: async () => "ok",
});
const mode = (action.tool.parameters as any).properties.mode;
expect(mode.enum).toEqual(["a", "b"]);
expect(mode.anyOf).toBeUndefined();
});

it("stores the original schema on the entry for downstream re-validation", () => {
const schema = z.object({ x: z.string() });
const action = defineAction({
Expand Down
Loading
Loading