Skip to content

docs: clarify CPU-only runtime path - #2134

Open
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/cpu-only-runtime-note
Open

docs: clarify CPU-only runtime path#2134
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/cpu-only-runtime-note

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 4, 2026

Copy link
Copy Markdown

Summary

  • Clarify that the legacy local_chat / ktransformers.server examples in doc/en/install.md target CPU-GPU heterogeneous inference and require a CUDA-capable GPU.
  • Point CPU-only users toward the newer KT-Kernel CPU backends, such as LLAMAFILE with GGUF weights.

Refs #337

Test Plan

  • python documentation guard check for the new CPU-only runtime note
  • git diff --check HEAD~1..HEAD

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a single clarifying bullet to the "Preparation" section of doc/en/install.md, stating that local_chat and ktransformers.server require a CUDA-capable GPU and redirecting CPU-only users to the LLAMAFILE backend with a link to the llamafile operator guide.

  • Adds a GPU-requirement note as the first bullet in the Preparation list, pointing CPU-only users to ./operators/llamafile.md (which exists in the repo).
  • No code or configuration is changed; the diff is entirely documentation.

Confidence Score: 5/5

Documentation-only change with no impact on runtime behaviour; safe to merge.

The diff is five lines of Markdown inserted as a bullet point. The linked file (./operators/llamafile.md) is confirmed to exist in the repository, so the hyperlink is valid. No code, configuration, or build artefacts are touched.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
doc/en/install.md Adds a 5-line note clarifying the CUDA GPU requirement for local_chat / ktransformers.server and linking CPU-only users to the llamafile operator guide. The referenced file exists and the link is well-formed.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User reads install.md] --> B{Has a CUDA-capable GPU?}
    B -- Yes --> C[Follow CUDA setup steps]
    C --> D[local_chat / ktransformers.server\nCPU-GPU heterogeneous inference]
    B -- No --> E[CPU-only path]
    E --> F[Use KT-Kernel CPU backends\ne.g. LLAMAFILE with GGUF weights]
    F --> G[See llamafile operator guide]
Loading

Reviews (2): Last reviewed commit: "[docs]: clarify CPU-only install guidanc..." | Re-trigger Greptile

Comment thread doc/en/install.md Outdated
Comment thread doc/en/install.md Outdated
@zcxGGmu

zcxGGmu commented Aug 5, 2026

Copy link
Copy Markdown
Author

Thanks for the review — I pushed b97970419bbb addressing the feedback.

Changes made:

  • Softened the CPU-only runtime note and added a concrete link to the llamafile operator guide.

Validation:

  • git diff --check — passed.
  • Content guard — passed: CPU-only note links to llamafile docs and avoids unsupported legacy wording.
  • Diff scan — passed: no secret/debug-marker patterns found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants