Skip to content

Merge main into spiceai - #41

Merged
krinart merged 75 commits into
spiceaifrom
merge-main-into-spiceai
Oct 9, 2026
Merged

krinart merged 75 commits into
spiceaifrom
merge-main-into-spiceai

Conversation

@Jeadie

@Jeadie Jeadie commented Aug 12, 2026 •

Copy link
Copy Markdown

Summary

  • merges the updated upstream-synced main branch into the spiceai branch
  • retains Spice-specific Azure authentication, response OpenAPI schemas, and rate-limit aggregation while adopting the current Responses API and upstream middleware/client changes
  • adds the upgrade-async-openai agent skill documenting the required two-stage update process and focused validation

Validation

  • cargo check -p async-openai --features responses
  • cargo test -p async-openai --features response-types --test responses_input_item_serde --test responses_output_to_input
  • cargo test -p async-openai --features responses --lib## Codex gateway fixes

These commits let Spice act as a gateway in front of the Codex CLI. A gateway deserializes each Responses request into CreateResponse and serializes it again before it forwards the request. Real Codex traffic showed two failure types: a hard deserialize failure (HTTP 422) and a silent loss of fields. This change corrects both.

Request transport:

  • Send one Content-Type header on each JSON request. The client applied the config headers and then appended a second Content-Type. A forwarding config already supplies Content-Type, so the request carried two. The Codex backend rejects two Content-Type headers with Unsupported content type.

Responses input items (found with a sweep of captured Codex bodies):

  • OutputMessage.status is now optional. A replayed assistant message omits it, so a required field failed the whole input array.
  • OutputTextContent.annotations now defaults to empty and is skipped when empty. A replayed assistant message omits it.
  • InputMessage keeps a new optional id. The gateway forwards it instead of dropping it.
  • Reasoning keeps a new optional context (for example all_turns).
  • CustomToolCall keeps a new optional status.
  • OutputTextContent.logprobs is skipped when it is None, so the body does not gain a null field.

Validation

  • cargo test -p async-openai --no-default-features --features "rustls full" --test codex_gateway_sweep
  • The sweep deserializes captured Codex bodies into CreateResponse and compares a re-serialize against the original to find hard failures and lost fields. All fixtures pass.

64bit and others added 30 commits December 20, 2025 19:35
* utils.rs for wasm

* image api group for wasm

* download.rs not availbale in wasm

* config.rs for wasm

* completions.rs for wasm

* client.rs for wasm

* chat.rs for wasm

* types/input_source.rs for wasm

* async-openai/src/types/impls.rs for wasm

* async-openai/src/assistants/runs.rs for wasm

* async-openai/src/assistants/threads.rs for wasm

* async-openai/src/audio/speech.rs for wasm

* async-openai/src/audio/transcriptions.rs for wasm

* async-openai/src/responses/responses_.rs for wasm

* async-openai/src/types/assistants/stream.rs for wasm

* async-openai/src/types/audio/sdk.rs for wasm

* async-openai/src/types/audio/stream.rs for wasm

* async-openai/src/types/chat/chat_.rs for wasm

* async-openai/src/types/completions/completion.rs for wasm

* async-openai/src/types/images/stream.rs for wasm

* async-openai/src/types/images/sdk.rs for wasm

* async-openai/src/types/images/impls.rs for wasm

* async-openai/src/types/realtime/client_event.rs for wasm

* async-openai/src/types/responses/stream.rs for wasm

* async-openai/src/error.rs for wasm

* README.md for wasm

* add examples/wasm-responses

* async-openai/Cargo.toml for wasm

* add getrandom in example cargo toml

* async-openai/src/types/images/impls.rs warning fix for wasm

* wasm-pr-checks.yml

* cargo fmt

* fix: targets -> target

* fix clippy

* update docs

* remove clippy from wasm pr checks

* fix wasm-pr-checks.yml

* add .cargo/config.toml https://docs.rs/getrandom/latest/getrandom/\#opt-in-backends

* updated Cargo.toml

* fix: wasm pr check

* fix example
…it#512)

Add serde attribute to omit None values for include_usage and
include_obfuscation. Add tests to verify correct serialization and
deserialization behavior.
* updated spec - and corresponding code

* cargo clippy fixes

* cargo clippy fixes for assistants feature
* fix: make open_page url optional

* fix: replace `find` with `find_in_page`

* Update async-openai/src/types/responses/response.rs

* Update async-openai/src/types/responses/response.rs

* Update async-openai/src/types/responses/response.rs

---------

Co-authored-by: Himanshu Neema <himanshun.iitkgp@gmail.com>
* feat: add Skills API support

Add types and API wrappers for the Skills API endpoints including
skill CRUD, skill versions, and content downloads.

* add name in RealtimeServerEventResponseFunctionCallArgumentsDone

* skill types

* updates to response type

* image generation action in ImageGenTool

* new types in image.rs

* updated spec

* enviroment

* handle keepalive in stream_mapped_raw_events

* cargo fmt

* add new feature flags

* add Skills in Readme
…ion (64bit#527)

default EasyInputMessage.type to message when missing in JSON input

default WebSearchApproximateLocation.type to approximate when missing in JSON input

add focused response-types integration tests for both deserialize-default + canonical serialization

register responses_input_item_serde integration test in Cargo.toml with required-features

Refs:

- openapi.documented.yml: EasyInputMessage (required role+content; type const not required)

- openapi.documented.yml: WebSearchApproximateLocation (type has default approximate)

- https://github.com/openai/openai-node/blob/03ee9c5f42c6ce9636fbbb365a97746feb17e8cd/src/resources/responses/responses.ts#L411

- https://github.com/openai/openai-python/blob/656e3cab4a18262a49b961d41293367e45ee71b9/src/openai/types/responses/easy_input_message_param.py#L13

- https://github.com/openai/openai-go/blob/aeccc5f54d88662b744cee4eb3d256142d6ec76b/responses/response.go#L10755
* updates to response types

* update MCPTool type

* updated ImageDetail type

* updated Voice type

* video character and edit apis + types

* fix examples compilation from updated types

* updated spec

* cargo clippy fix

* cargo fmt
While this is a required field in the OpenAPI spec, the spec is wrong; OpenAI itself accepts a missing `id` and popular tools like OpenCode send requests without an `id`. This aligns the behavior with the real world usage to allow deserializing OpenCode (and other) requests
* add agents.md

* agents.md

* update adgents

* updated to async-openai/src/types/vectorstores/vector_store.rs:

* updates to async-openai/src/types/responses/conversation.rs

* updated types in async-openai/src/types/responses/response.rs

* update example for updated types

* updated spec
)

- Bump reqwest from 0.12 to 0.13
- Add "query" feature (now opt-in in reqwest 0.13)
- Replace abandoned reqwest-eventsource 0.6 with aha-reqwest-eventsource
  0.1 (fork that supports reqwest 0.13), aliased via package rename so no
  source-level changes are needed
- Update rustls/rustls-webpki-roots feature mappings: reqwest 0.13 unified
  TLS roots behind the 'rustls' feature (uses rustls-platform-verifier);
  separate native-roots/webpki-roots features were removed upstream
64bit#538)

* Add From<OutputItem> for Item / InputItem (multi-turn responses)

The Responses API requires Reasoning items to be echoed back into the
next request's input alongside the FunctionCall items they preceded.
Without that pairing the server rejects the request with:

    Item 'fc_...' of type 'function_call' was provided without its
    required 'reasoning' item: 'rs_...'

In the Python and Node SDKs the canonical pattern is `input_list +=
response.output` because their input/output item types are the same.
In Rust the schemas split into Item vs OutputItem, with several
variants that have *Resource (output, required id/status) and
*ItemParam (input, optional id/status) flavors, so callers had to
hand-write a per-variant conversion to do what's a one-liner in the
official SDKs.

Add From<OutputItem> for Item and From<OutputItem> for InputItem,
plus the supporting per-variant From impls (status enum mappings,
shell environment/outcome mappings, apply_patch operation mappings,
resource-to-param mappings). Required id/status fields fold into
Some(...) on the way in; the output-only created_by annotation is
dropped because the input schema doesn't accept it. The canonical
SDK pattern now works:

    input_items.extend(response.output.into_iter().map(InputItem::from));

Includes round-trip tests covering the headline reasoning +
function_call pairing case plus the resource-to-param folding for
ApplyPatch, Compaction, and FunctionCallOutput. New
responses-multi-turn-reasoning example demonstrates the pattern with
a reasoning model.

Closes 64bit#492.

* chore: fmt
* initial implementation of middleware

* allow error to be Into<OpenAIError>

* documentation and default reqwest service

* namspaced in middleware

* park tower error handling for now TODO

* working compilation of all crates

* support for middleware in wasm

* impl Default for ReqwestService

* add tower example

* add tower-wasm example

* introduce Boxed variant

* add both display and debug

* update comment

* updated doc

* updated doc

* refactor: rename http_executor to executor; create a middleware dir for doc.rs

* move retry stuff in its own module

* update examples

* document and log retry headers

* add openai retry layer

* fix logical error in openai layer

* optimize openai retry layer

* return early on success

* update doc

* updated doc

* add middleware to full

* add middleware section

* nest retry stuff inside ::middleware::retry

* change of module path

* updated doc

* updated doc

* update doc

* updated doc

* fix the warning

* update tower example

* cleanup

* middleware.md

* only compile it for wasm

* cleanup client

* cleanup client

* cleanup

* renmae midddlware

* update readme

* cleanup streaming and form creation

* fix test

* add larger delay for retrieval to work

* make image-edit example work

* allow tower user input to be random

* update readme

* update workflows to build middleware feature

* fix warnings

* fix warnings

* only keep wasm32-unknown-unknown for workflows checks

* bump async-openai-macros version

* update doc comment

* remove duplicate From tower BoxError for wasm/non-wasm

* remove redundant doc comment

* cleanup comment

* in readme use link to github instead of relative path - so that it works on crates.io

* remove comment

* update lib.rs WASM section

* add docs on middleware error handling
Comment thread .github/workflows/pr-checks.yml Fixed
Comment thread .github/workflows/pr-checks.yml Fixed
Comment thread .github/workflows/wasm-pr-checks.yml Fixed
Comment thread .github/workflows/wasm-pr-checks.yml Fixed
@Jeadie Jeadie self-assigned this Aug 12, 2026
Jeadie added 7 commits August 13, 2026 11:50
build_request_builder applies the Config's default headers, then the JSON
factory appended Content-Type: application/json. A Config that forwards a
caller's headers (the Codex gateway) already supplies Content-Type, so the
outgoing request carried two Content-Type headers. The ChatGPT Codex backend
rejects that with {"detail":"Unsupported content type"}.

Set Content-Type by inserting (replacing) on the built request instead of
appending on the builder, so there is always exactly one.
When a Responses request is deserialized and reserialized by a gateway (Spice
in front of Codex), replayed input items broke two ways: hard deserialize
failures and silent field drops. Real Codex request bodies exposed:

- OutputMessage.status was required, but a replayed assistant message omits it
  -> the whole input array failed to deserialize (HTTP 422). Now optional.
- OutputTextContent.annotations was required, but a replayed assistant message
  omits it -> same 422. Now defaults to empty and is skipped when empty.
- InputMessage had no id field, so message ids were dropped on reserialize.
- Reasoning dropped its context field (Codex sends context: all_turns).
- CustomToolCall dropped its status field on reserialize.
- OutputTextContent.logprobs reserialized as null; now skipped when None.

Adds a sweep test that deserializes real captured Codex bodies into
CreateResponse and diffs a reserialize against the original to catch both
hard failures and silent field loss.
Replace local paths, installation/session ids, and other machine-specific
values in the captured fixtures with neutral placeholders. The sweep only
checks item structure and self-consistent roundtrip, so placeholder values
keep it passing while keeping the corpus free of local details.
Codex tool-call turns rely on x-codex-turn-state: the backend returns it
as a response header and replays it on the next request in the same
turn. Client::post/post_stream discarded response headers entirely, so
a caller had no way to read it back. Add create_with_headers and
create_stream_with_headers alongside the existing create/create_stream.
The merge from main kept the spiceai [features] block, which dropped the
rustls-no-provider feature. The TLS feature CI matrix still builds it.
@krinart krinart self-assigned this Oct 9, 2026
…iceai

# Conflicts:
#	async-openai/tests/ser_de.rs
…iceai

# Conflicts:
#	async-openai/tests/ser_de.rs
@krinart
krinart merged commit b7f69df into spiceai Oct 9, 2026
104 checks passed
@krinart
krinart deleted the merge-main-into-spiceai branch October 9, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.