Skip to content

docs: record SmolLM2 Transformers compatibility #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaBestCode merged 1 commit into from
Sep 8, 2026
Merged

docs: record SmolLM2 Transformers compatibility #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,25 @@ The initial adapter intentionally supports batch size one, greedy/sampling
generation, and full-attention decoder models only. Beam search and models with
sliding, chunked, or linear-attention layers fail explicitly.

### Transformers checkpoint compatibility

The adapter's compatibility boundary is recorded separately from performance
or model-quality claims. A checkpoint is **verified** only when the real smoke
test has been run and its raw result is checked into [`results/`](results/).

| Status | Checkpoint or family | Evidence / boundary |
| --- | --- | --- |
| Verified | `Qwen/Qwen3-0.6B` | Real Transformers smoke result on Apple M4; see [`results/README.md`](results/README.md). |
| Verified | `HuggingFaceTB/SmolLM2-135M` | Real full-attention decoder smoke result with two or more eviction rounds; see [`results/README.md`](results/README.md). |
| Expected, not individually verified | Other decoder-only Transformers checkpoints with full attention, batch size one, and a compatible `Cache` interface | This is an adapter-shape expectation, not a compatibility guarantee. Run a smoke test before relying on a checkpoint. |
| Unsupported | Sliding-window, chunked, linear, or other hybrid-attention layers | The adapter rejects these attention modes explicitly. |
| Unsupported | Beam search and batched generation | The initial adapter supports batch size one only. |

The **expected** row must not be read as a test result: model configuration,
attention layout, and Transformers integration details can still differ between
families. Compatibility evidence is deliberately kept separate from throughput
and quality evaluation.

Run a real-checkpoint smoke test (downloads the model from Hugging Face):

```bash
Expand Down
20 changes: 20 additions & 0 deletions results/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,23 @@ model-quality evaluation, and it does not validate the paper's throughput claim.
Within schema v1, additive optional fields remain compatible. Removing or
renaming fields, or changing their semantic meaning, requires a new schema
version.

## SmolLM2-135M on Windows CPU

- Result: [`smollm2-135m-smoke.json`](smollm2-135m-smoke.json)
- Model revision: `93efa2f097d58c2a74874c7e644dbc9b0cee75a2`
- Environment: Windows 11, AMD64, Python 3.13.5, PyTorch 2.14.0+cpu,
Transformers 5.16.1
- Hardware: 13th Gen Intel Core i7-13620H, 13.0 GB RAM
- Device: CPU
- Prompt tokens: 11
- Generated tokens: 128
- Persistent budget `K`: 64
- Buffer `r`: 8
- Eviction rounds: 9 per layer
- Final physical cache: 73 positions per layer

This smoke run verifies the Transformers adapter against the
`HuggingFaceTB/SmolLM2-135M` full-attention decoder configuration. It records
compatibility and eviction behavior only; it is not a throughput comparison,
quality evaluation, or claim about the model family as a whole.
87 changes: 87 additions & 0 deletions results/smollm2-135m-smoke.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
{
"model": "HuggingFaceTB/SmolLM2-135M",
"model_revision": "93efa2f097d58c2a74874c7e644dbc9b0cee75a2",
"device": "cpu",
"platform": "Windows-11-10.0.26200-SP0",
"machine": "AMD64",
"hardware": "13th Gen Intel Core i7-13620H; 13.0 GB RAM",
"python_version": "3.13.5",
"torch_version": "2.14.0+cpu",
"transformers_version": "5.16.1",
"prompt_tokens": 11,
"generated_tokens": 128,
"budget": 64,
"buffer_size": 8,
"seed": 0,
"elapsed_seconds": 4.05,
"tokens_per_second": 31.604,
"cache": {
"absolute_tokens_seen": 138,
"physical_tokens_per_layer": [
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73,
73
],
"eviction_rounds_per_layer": [
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9,
9
],
"head_evictions_total": 810,
"positions_evicted_total": 5850
}
}