LLM systems are often explained with large frameworks, diagrams, or pseudocode. This repository takes the opposite route: each concept is reduced to a small, dependency-free C++20 experiment with an executable invariant.
这里不是“抄一遍公式”，而是让每个问题都能运行、观察和验证。
| Experiment | Question answered | What the code verifies |
|---|---|---|
| Online Softmax | How can softmax stream over logits without losing numerical stability? | Online reduction matches stable softmax at large logits |
| RoPE | Why does rotary position embedding encode relative position? | q@7 · k@23 == q@0 · k@16 numerically |
| KV Cache | What is reused during autoregressive decoding? | Incremental keys and values produce causal attention context |
| MHA / GQA / MQA | Why do fewer KV heads reduce inference memory? | Cache size drops from 1x to 1/4x and 1/32x |
| Speculative Decoding | Why does rejection correction remain lossless? | Monte Carlo samples recover the target distribution |
| Tiled GEMM | How does cache-aware blocking reorganize matrix multiplication? | Tiled output matches a reference GEMM element by element |
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel
ctest --test-dir build --output-on-failure
./build/llmcore_cli allNo third-party libraries are required. Apple Clang, GCC, and recent MSVC toolchains with C++20 support are sufficient.
Run a single experiment:
./build/llmcore_cli rope
./build/llmcore_cli kv-cache
./build/llmcore_cli speculative[RoPE]
q@7 dot k@23 == q@0 dot k@16
relative-position error = 2.77556e-17
[MHA vs GQA vs MQA]
MHA: kv_heads=32, cache=2.000 GiB, ratio=1.000x
GQA: kv_heads= 8, cache=0.500 GiB, ratio=0.250x
MQA: kv_heads= 1, cache=0.062 GiB, ratio=0.031x
[Speculative Decoding]
acceptance rate = 0.8524
total variation = 0.0017
LLMCore-CPP/
├── include/llmcore/labs.hpp # Public experiment APIs
├── src/ # Six independent implementations
├── tests/test_labs.cpp # Numerical and behavioral checks
├── .github/workflows/ # Linux C++ CI
└── CMakeLists.txt # Portable C++20 build
- Small enough to read: each mechanism lives in one focused source file.
- Correct before fast: optimized variants are checked against clear references.
- Numbers over slogans: every demo prints a quantity that can falsify the explanation.
- Framework free: the core ideas stay visible without tensor-library abstractions.
- FlashAttention-style tiled attention and online normalization
- Tensor-parallel column/row linear layers with communication accounting
- Ring All-Reduce traffic simulation
- Paged KV cache allocation
- INT8 weight-only quantization
- Mixture-of-Experts routing and load balance
The learning prompts were inspired by systems questions curated in HaiPenglai/Treasure-Questions. All explanations, APIs, tests, and C++ implementations in this repository are original; no source code, images, or repository assets are redistributed.
Released under the MIT License.