docs: clarify Qwen3 model metadata path - #2131
Conversation
Greptile SummaryThis PR adds a short clarifying paragraph to the Qwen3 AMX tutorial explaining the
Confidence Score: 5/5Documentation-only change with no runtime impact; safe to merge. The change adds a single clarifying paragraph to a markdown tutorial. The listed files (config.json, tokenizer.json, tokenizer_config.json) are the correct minimal set needed by AutoTokenizer, and the explanation of why .safetensors weights can be omitted is accurate for GGUF-based serving. No code, config, or behavior is affected. Files Needing Attention: No files require special attention.
|
| Filename | Overview |
|---|---|
| doc/en/AMX.md | Adds a clarifying paragraph explaining that <model_dir> must contain tokenizer/config metadata files (config.json, tokenizer.json, tokenizer_config.json) even when serving from GGUF weights, and that the large .safetensors files are not required. |
Sequence Diagram
sequenceDiagram
participant User
participant KTransformers
participant model_dir as model_dir (metadata only)
participant gguf_dir as gguf_dir (weights)
User->>KTransformers: start server (--model_path, --gguf_path)
KTransformers->>model_dir: load config.json (architecture)
KTransformers->>model_dir: load tokenizer.json + tokenizer_config.json (chat template)
KTransformers->>gguf_dir: load BF16 GGUF weights
Note over model_dir: .safetensors NOT required here
User->>KTransformers: chat request
KTransformers->>model_dir: resolve tokenizer.chat_template
KTransformers-->>User: response
Reviews (2): Last reviewed commit: "[docs]: clarify Qwen3 metadata files" | Re-trigger Greptile
|
Thanks for the review — I pushed Changes made:
Validation:
|
Summary
<model_dir>requirement.tokenizer_config.jsonand the chat template, even when large safetensor weights are not required for GGUF serving.Refs #1226
Test Plan
pythondocumentation guard check for the new Qwen3 model metadata guidancegit diff --check HEAD~1..HEAD