Skip to content

fix(memory): add dedicated SGLang and vLLM thinking transports - #348

Open
LeslieWylie wants to merge 1 commit into
MemTensor:mainfrom
LeslieWylie:codex/sglang-vllm-thinking-providers-v2
Open

fix(memory): add dedicated SGLang and vLLM thinking transports#348
LeslieWylie wants to merge 1 commit into
MemTensor:mainfrom
LeslieWylie:codex/sglang-vllm-thinking-providers-v2

Conversation

@LeslieWylie

Copy link
Copy Markdown

Summary

Replaces #312 with a clean branch history.

Add dedicated sglang and vllm memory LLM providers for the Qwen chat_template_kwargs.enable_thinking transport.

The generic openai_compatible provider keeps its existing top-level request shape, so cloud-provider calls are unaffected.

Changes

  • accept sglang and vllm as explicit LLM provider values
  • route both providers through the existing OpenAI-compatible chat-completions client
  • move Qwen enable_thinking into chat_template_kwargs only for those dedicated providers
  • preserve caller-supplied chat_template_kwargs, including an explicit enable_thinking override
  • keep top-level enable_thinking behavior unchanged for DashScope and generic OpenAI-compatible endpoints

Validation

  • vitest run Memory/tests/llm-thinking.test.ts — 21 tests passed
  • coverage includes SGLang, vLLM, disabled/enabled thinking, preserved template kwargs, and generic-provider isolation
  • live SGLang replay previously confirmed that top-level enable_thinking=false was ignored while chat_template_kwargs.enable_thinking=false produced zero reasoning tokens and valid JSON

Route Qwen enable_thinking through chat_template_kwargs only for explicit sglang and vllm providers. Keep the generic OpenAI-compatible request body unchanged so cloud-provider calls retain their existing contract.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant