Repair pinned Qwen3 bundle with its original tokenizer

[leehack]

Problem and change

The pinned Qwen3-0.6B bundle mixes decoded SentencePiece tokens with literal byte-level BPE spellings for incomplete UTF-8 tokens. Original IDs [9707, 0, 26525, 232] therefore produce Hello!ĠðŁĺĬ instead of Hello! 😊. Encoding text back through the converted tokenizer can hide the defect by choosing different token IDs.

Add an explicit, checksum-pinned artifact repair which replaces only this bundle's tokenizer section with the exact original Hugging Face tokenizer JSON. Existing v0.17.0-3 supports that section already. No runtime/output rewriting, token-ID remapping, automatic download changes, publication, or GPU gate changes.

Safety and validation

• Both latest-head GitHub checks passed (Validate and Publication lifecycle).
• Both exact input hashes and the known container layout are required. Preserve weights/other sections and source file; publish a new output atomically without overwriting existing files. Read header only from the verified staged copy.
• 21 diagnostics tests passed, including wrong inputs, layout mismatch, source mutation, output races, unchanged weights, and exact decompressed tokenizer content.
• Real macOS CPU and Metal native chat with repaired model: Hello! 😊; thinking/final and system/history recall pass. Original HF reference comparisons cover multilingual text, whitespace, normalization, literal BPE characters and split emoji IDs. C-string embedded-NUL limitation is explicitly excluded.
• Output SHA on local zlib: c5c3f100affb7be6abe99bc2d00228ee3552bc6e0b8adb047b1b6eb3ffbe5e57.
• Independent blocking review accepted 7026142f38b1eda1bd0a9357675e69742be23d4e against dd86d49, after fixing its source-header race finding. Zero remaining blocking findings.
• Other platforms and consumer adoption are not claimed. This model-specific utility does not fix arbitrary converted models. Follow documented exact inputs in docs/qwen3_tokenizer_repair.md.

Refs #48. The public original bundle/catalog remains unchanged; deployment/adoption is a separate step.

Additional desktop evidence

• Linux x64 / NVIDIA L4: repaired Qwen3 passes CPU full chat/thinking/history and Vulkan GPU streaming/cancellation/reuse with the separately corrected runtime from Fix shared-runtime linkage for desktop GPU builds #50. Original split emoji token IDs decode to Hello! 😊.
• Windows Server 2022 / NVIDIA L4: repaired Qwen3 passes CPU streaming/cancellation/reuse and exact split-emoji decoding. D3D12 GPU still produces incorrect generation with both original and repaired models; this is a separate Diagnose NVIDIA L4 desktop WebGPU initialization crashes before enabling consumer GPU #47 blocker and is not claimed fixed by this repair.
• The Linux and Windows test VMs and disks were deleted after evidence collection.