Skip to content

docs: clarify Qwen3 model metadata path - #2131

Open
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/qwen3-model-dir-chat-template
Open

docs: clarify Qwen3 model metadata path#2131
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/qwen3-model-dir-chat-template

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 4, 2026

Copy link
Copy Markdown

Summary

  • Clarify the Qwen3 AMX tutorial's <model_dir> requirement.
  • Explain that the directory still needs tokenizer/config metadata, including tokenizer_config.json and the chat template, even when large safetensor weights are not required for GGUF serving.

Refs #1226

Test Plan

  • python documentation guard check for the new Qwen3 model metadata guidance
  • git diff --check HEAD~1..HEAD

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a short clarifying paragraph to the Qwen3 AMX tutorial explaining the <model_dir> requirement: the directory must contain tokenizer and config metadata files even when serving from GGUF weights, because the large .safetensors weight files are not needed but the tokenizer files still are.

Confidence Score: 5/5

Documentation-only change with no runtime impact; safe to merge.

The change adds a single clarifying paragraph to a markdown tutorial. The listed files (config.json, tokenizer.json, tokenizer_config.json) are the correct minimal set needed by AutoTokenizer, and the explanation of why .safetensors weights can be omitted is accurate for GGUF-based serving. No code, config, or behavior is affected.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
doc/en/AMX.md Adds a clarifying paragraph explaining that <model_dir> must contain tokenizer/config metadata files (config.json, tokenizer.json, tokenizer_config.json) even when serving from GGUF weights, and that the large .safetensors files are not required.

Sequence Diagram

sequenceDiagram
    participant User
    participant KTransformers
    participant model_dir as model_dir (metadata only)
    participant gguf_dir as gguf_dir (weights)

    User->>KTransformers: start server (--model_path, --gguf_path)
    KTransformers->>model_dir: load config.json (architecture)
    KTransformers->>model_dir: load tokenizer.json + tokenizer_config.json (chat template)
    KTransformers->>gguf_dir: load BF16 GGUF weights
    Note over model_dir: .safetensors NOT required here
    User->>KTransformers: chat request
    KTransformers->>model_dir: resolve tokenizer.chat_template
    KTransformers-->>User: response
Loading

Reviews (2): Last reviewed commit: "[docs]: clarify Qwen3 metadata files" | Re-trigger Greptile

Comment thread doc/en/AMX.md Outdated
@zcxGGmu

zcxGGmu commented Aug 5, 2026

Copy link
Copy Markdown
Author

Thanks for the review — I pushed 84ec1899d007 addressing the feedback.

Changes made:

  • Folded the chat-template note into the tokenizer_config.json metadata description.

Validation:

  • git diff --check — passed.
  • Content guard — passed: Qwen3 metadata text folds chat template into tokenizer_config.json.
  • Diff scan — passed: no secret/debug-marker patterns found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants