OpenAI Responses API: compatible with reasoning summary and encryption

[Sha1rholder]

Please see the issue for details. Fixes #145

The code changes are actually minimal. Mostly are actually focused on adapting the test content.

before:

After:

The logs show that the compatibility improvements do not affect cache hits.

~/Desktop/ninfer> nu ~/Desktop/ninfer/.local/serve.nu -b
Starting profile=128K, context=131072, KV=131072, concurrency=2, vision=off
[2026-09-01 08:31:34.850] [info] ninfer-serve: loading model...
[2026-09-01 08:31:35.147] [info] ninfer-serve: load        weights                      0.00%           0 B /      1[2026-09-01 08:31:35.349] [info] ninfer-serve: load        weights                      5.39%      1.06 GiB /      1[2026-09-01 08:31:35.556] [info] ninfer-serve: load        weights                     11.09%      2.19 GiB /      1[2026-09-01 08:31:35.762] [info] ninfer-serve: load        weights                     17.11%      3.37 GiB /      1[2026-09-01 08:31:35.968] [info] ninfer-serve: load        weights                     22.81%      4.50 GiB /      1[2026-09-01 08:31:36.172] [info] ninfer-serve: load        weights                     28.83%      5.69 GiB /      1[2026-09-01 08:31:36.373] [info] ninfer-serve: load        weights                     34.53%      6.81 GiB /      1[2026-09-01 08:31:36.575] [info] ninfer-serve: load        weights                     40.23%      7.94 GiB /      1[2026-09-01 08:31:36.777] [info] ninfer-serve: load        weights                     45.93%      9.06 GiB /      1[2026-09-01 08:31:36.978] [info] ninfer-serve: load        weights                     51.64%     10.19 GiB /      1[2026-09-01 08:31:37.184] [info] ninfer-serve: load        weights                     57.66%     11.37 GiB /      1[2026-09-01 08:31:37.386] [info] ninfer-serve: load        weights                     63.36%     12.50 GiB /      1[2026-09-01 08:31:37.589] [info] ninfer-serve: load        weights                     69.38%     13.69 GiB /      1[2026-09-01 08:31:37.799] [info] ninfer-serve: load        weights                     75.08%     14.81 GiB /      1[2026-09-01 08:31:38.007] [info] ninfer-serve: load        weights                     81.10%     16.00 GiB /      1[2026-09-01 08:31:38.214] [info] ninfer-serve: load        weights                     87.12%     17.19 GiB /      1[2026-09-01 08:31:38.421] [info] ninfer-serve: load        weights                     92.82%     18.31 GiB /      1[2026-09-01 08:31:38.631] [info] ninfer-serve: load        weights                     98.84%     19.50 GiB /      1[2026-09-01 08:31:38.673] [info] ninfer-serve: load        weights                    100.00%     19.73 GiB /      19.73 GiB     3.525 s
[2026-09-01 08:31:40.894] [info] ninfer-serve: model loaded in 6.04412 s
[2026-09-01 08:31:40.894] [info] ninfer-serve: KV capacity explicit resolved=131072 tokens pages=2048/4096 runtime=5.20 GiB free-after-weights=9.42 GiB free-after-startup=4.36 GiB headroom=0.00 MiB slack=4.22 GiB graph-allowance=164.00 MiB context-cache=on device-state=2-cache+2-active host-state=8 host-kv=4.50 GiB private=4 shared=2 anchors=2 context-cost-transfer=compiled-default context-cost-prefill=compiled-default cost-profile=nvidia-geforce-rtx-5090-sm120/qwen3.8-27b/nvfp4
[2026-09-01 08:31:40.894] [info] ninfer-serve: warming up...
[2026-09-01 08:31:40.953] [info] ninfer-serve: listening on http://127.0.0.1:8080 (model id: qwen3.8-27b, auth: disabled)
[2026-09-01 08:31:45.888] [warning] ninfer-serve: request 1 OpenAI Responses compatibility downgrade: reasoning.summary returns placeholder; reasoning.encrypted_content returns raw reasoning content
[2026-09-01 08:31:45.888] [info] ninfer-serve: [req 1] openai_responses stream msgs=4 max_tokens=8192 (server default) tools=17 tool_choice=auto tool_history=no thinking=on reasoning_effort=xhigh->xhigh preserve_thinking=on preserve_change=no sampler=[temp=1.00 top_p=0.95 top_k=20 seed=4450369916847309437] → submitted
[2026-09-01 08:31:45.953] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=0.0tok/s running=1 prefilling=1 decode_ready=0 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=n/a host=0.00ms decode-host=n/a wait=n/a boundary=0.00ms maintenance=0.00ms
[2026-09-01 08:31:49.307] [info] ninfer-serve: [req 1] done finish=tool_calls tool_calls=1 prompt=15939 gen=163 cache=0 reuse=root ttft=2318ms prefill=6923.4tok/s decode=145.5tok/s wall=3.43s host=295.98ms decode-host=41.4us/round wait=19855.1us/round speculative=mtp 2.95tok/round (64.9%)
[2026-09-01 08:31:50.953] [info] ninfer-serve: throughput interval=5.000s prefill=3187.7tok/s decode=32.4tok/s running=0 prefilling=0 decode_ready=0 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=297.66ms decode-host=41.4us/round wait=19855.1us/round boundary=1.81ms maintenance=0.04ms
[2026-09-01 08:31:51.348] [warning] ninfer-serve: request 2 OpenAI Responses compatibility downgrade: reasoning.summary returns placeholder; reasoning.encrypted_content returns raw reasoning content
[2026-09-01 08:31:51.348] [info] ninfer-serve: [req 2] openai_responses stream msgs=6 max_tokens=8192 (server default) tools=17 tool_choice=auto tool_history=yes thinking=on reasoning_effort=xhigh->xhigh preserve_thinking=on preserve_change=no sampler=[temp=1.00 top_p=0.95 top_k=20 seed=10991629769501871895] → submitted
[2026-09-01 08:31:55.953] [info] ninfer-serve: throughput interval=5.000s prefill=50.4tok/s decode=114.6tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=34.05ms decode-host=55.2us/round wait=19463.9us/round boundary=1.94ms maintenance=0.13ms
[2026-09-01 08:32:00.953] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=128.0tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=9.55ms decode-host=36.9us/round wait=19234.8us/round boundary=0.17ms maintenance=0.11ms
[2026-09-01 08:32:02.585] [info] ninfer-serve: [req 2] done finish=tool_calls tool_calls=2 prompt=16186 gen=1464 cache=15934 reuse=private_response_replay ttft=168ms prefill=1647.2tok/s decode=132.1tok/s wall=11.25s host=46.67ms decode-host=47.0us/round wait=19293.8us/round speculative=mtp 2.56tok/round (51.9%)
[2026-09-01 08:32:05.953] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=50.0tok/s running=0 prefilling=0 decode_ready=0 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=4.80ms decode-host=55.7us/round wait=19020.1us/round boundary=0.10ms maintenance=0.05ms
[2026-09-01 08:32:28.168] [warning] ninfer-serve: request 3 OpenAI Responses compatibility downgrade: reasoning.summary returns placeholder; reasoning.encrypted_content returns raw reasoning content
[2026-09-01 08:32:28.168] [info] ninfer-serve: [req 3] openai_responses stream msgs=9 max_tokens=8192 (server default) tools=0 tool_choice=auto tool_history=yes thinking=on reasoning_effort=xhigh->xhigh preserve_thinking=on preserve_change=no sampler=[temp=1.00 top_p=0.95 top_k=20 seed=14014704682098349753] → submitted
[2026-09-01 08:32:28.195] [warning] ninfer-serve: request 4 OpenAI Responses compatibility downgrade: reasoning.summary returns placeholder; reasoning.encrypted_content returns raw reasoning content
[2026-09-01 08:32:28.195] [info] ninfer-serve: [req 4] openai_responses stream msgs=10 max_tokens=8192 (server default) tools=17 tool_choice=auto tool_history=yes thinking=on reasoning_effort=xhigh->xhigh preserve_thinking=on preserve_change=no sampler=[temp=1.00 top_p=0.95 top_k=20 seed=5069263073640451643] → submitted
[2026-09-01 08:32:28.288] [info] ninfer-serve: [req 3] done finish=cancelled prompt=7596 gen=0 cache=0 reuse=root ttft=7ms prefill=n/a decode=n/a wall=0.13s host=13.53ms decode-host=n/a wait=n/a speculative=mtp n/a
[2026-09-01 08:32:28.288] [info] ninfer-serve: [req 3] error client disconnected
[2026-09-01 08:32:30.954] [info] ninfer-serve: throughput interval=5.000s prefill=507.2tok/s decode=62.2tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=79.90ms decode-host=11.4us/round wait=19551.9us/round boundary=6.69ms maintenance=0.02ms
[2026-09-01 08:32:35.954] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=129.2tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=2.80ms decode-host=11.0us/round wait=19593.9us/round boundary=0.03ms maintenance=0.03ms
[2026-09-01 08:32:40.954] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=127.6tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=2.87ms decode-host=11.1us/round wait=19375.0us/round boundary=0.03ms maintenance=0.03ms
[2026-09-01 08:32:45.954] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=120.0tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=2.83ms decode-host=11.1us/round wait=19472.0us/round boundary=0.03ms maintenance=0.03ms
[2026-09-01 08:32:50.954] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=116.8tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=2.89ms decode-host=11.2us/round wait=19403.5us/round boundary=0.03ms maintenance=0.03ms
[2026-09-01 08:32:55.954] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=130.6tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=2.94ms decode-host=11.5us/round wait=19466.9us/round boundary=0.03ms maintenance=0.03ms
[2026-09-01 08:33:00.955] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=126.0tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=3.67ms decode-host=14.5us/round wait=19688.5us/round boundary=0.04ms maintenance=0.05ms
[2026-09-01 08:33:05.955] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=149.6tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=5.33ms decode-host=20.4us/round wait=19111.5us/round boundary=0.07ms maintenance=0.07ms
[2026-09-01 08:33:06.932] [info] ninfer-serve: [req 4] done finish=stop_token prompt=17693 gen=4961 cache=16181 reuse=private_response_replay ttft=460ms prefill=4249.5tok/s decode=129.6tok/s wall=38.75s host=84.41ms decode-host=13.1us/round wait=19441.7us/round speculative=mtp 2.52tok/round (50.7%)
[2026-09-01 08:33:10.955] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=30.0tok/s running=0 prefilling=0 decode_ready=0 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=1.23ms decode-host=23.2us/round wait=19145.3us/round boundary=0.04ms maintenance=0.02ms
[2026-09-01 08:34:11.665] [warning] ninfer-serve: request 5 OpenAI Responses compatibility downgrade: reasoning.summary returns placeholder; reasoning.encrypted_content returns raw reasoning content
[2026-09-01 08:34:11.665] [info] ninfer-serve: [req 5] openai_responses stream msgs=12 max_tokens=8192 (server default) tools=17 tool_choice=auto tool_history=yes thinking=on reasoning_effort=xhigh->xhigh preserve_thinking=on preserve_change=no sampler=[temp=1.00 top_p=0.95 top_k=20 seed=15739670518802352294] → submitted
[2026-09-01 08:34:15.956] [info] ninfer-serve: throughput interval=5.000s prefill=1014.0tok/s decode=90.8tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=153.73ms decode-host=41.4us/round wait=19632.4us/round boundary=9.82ms maintenance=0.09ms
[2026-09-01 08:34:20.956] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=127.2tok/s running=1 prefilling=0 decode_ready=1 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=10.93ms decode-host=43.2us/round wait=19700.8us/round boundary=0.18ms maintenance=0.12ms
[2026-09-01 08:34:23.032] [info] ninfer-serve: [req 5] done finish=stop_token prompt=22758 gen=1388 cache=17688 reuse=private_response_replay ttft=971ms prefill=5328.7tok/s decode=133.2tok/s wall=11.39s host=158.02ms decode-host=39.7us/round wait=19684.4us/round speculative=mtp 2.63tok/round (54.2%)
[2026-09-01 08:34:25.956] [info] ninfer-serve: throughput interval=5.000s prefill=0.0tok/s decode=59.4tok/s running=0 prefilling=0 decode_ready=0 waiting=0 materializing=0 capture_pending=0 terminal_pending=0 avg_decode_batch=1.00 host=3.05ms decode-host=28.6us/round wait=19728.4us/round boundary=0.07ms maintenance=0.04ms

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 193dab173a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Sha1rholder]

Sorry, forgot to update after modifying the code. I’ll fix it in a bit.

[Sha1rholder]

Resolved the Git conflict.

@Neroued 请问能review一下吗？看一下第一个commit就行，只是给Responses API加了个占位符，很小的改动

[Macasacker]

@Sha1rholder thanks, I'm trying to use Codex too.

Your branch is 151 commits behind master and has merge conflicts, so I rebased it and opened a PR as #295. The rebase was mostly routine. I did drop the log_compatibility_warning helper, since the serve/console_log.h it used is gone from master. The fix (accepting reasoning.summary and include: ["reasoning.encrypted_content"]) and the tests are unchanged.

Happy to close #295 if you're all over it.

[Sha1rholder]

@Macasacker Please proceed with #295.

I haven't been actively maintaining this PR recently, as I wasn't sure if it would get reviewed and merged. Thanks for taking over and cleaning this up. Feel free to continue with your PR.