fix(cli): save the trained model, default to fast, make decode compose

[Mirdula18]

Three CLI defects. No trainer, encoder or pre-tokenization code touched —
results/ and the README AUTOGEN block are byte-identical.

#	Defect	Fix
1	granule train never saved anything, so the next Quickstart line died with FileNotFoundError	-o/--output; without it, print that nothing was saved
2	CLI defaulted to the naive trainer while Tokenizer.train defaults to fast=True (54x penalty for following the docs)	fast is the default, --naive for the oracle, --fast kept as a hidden no-op
3	encode printed space-separated ids, --decode parsed only commas	decode accepts commas/whitespace/newlines; text is nargs="+" so xargs works

Plus Tokenizer.merges (returns a copy — _ranks is derived from _merges),
replacing the tok._merges reach-through in cli.py.

The Quickstart now runs start to finish

Verified verbatim in a fresh clone:

Trained tokenizer: vocab_size=4096 merges=3840
Saved model to model.json
72 433 111 44 32 1107 33
Hello
Hello, world!

Why the existing quickstart test didn't catch this

test_quickstart_granule_commands_are_real_subcommands appends --help to
every command, and argparse prints help and exits 0 before any handler
runs. It proved the flags parse and nothing else. Strengthened to check
Quickstart coherence (does the file train writes match the one encode
reads? does the corpus path exist?) and to actually execute the sequence,
pipeline included.

22 of the new tests were verified to fail against the pre-fix code.

One thing beyond the brief

ruff check src tests was already failing on main with 11 I001 errors,
before my change. Cause is toolchain skew, not the code: pyproject.toml
never set ruff's src, so ruff 0.5 (the floor in [dev]) treats granule as
third-party, while CI's newer ruff infers the src layout and accepts it. Set
src = ["src", "."] explicitly rather than reordering imports in 11 files to
satisfy the older version.

335 passed with -k "not slow"; ruff and mypy clean.

[Copilot]

Pull request overview

This PR fixes three CLI behavior defects so the README Quickstart can run end-to-end: training now persists a model when requested, CLI defaults match the Python API’s fast trainer behavior, and encode/--decode now compose cleanly (including xargs pipelines). It also exposes Tokenizer.merges as a public API to avoid CLI reach-through into tokenizer internals, and strengthens Quickstart-related tests to execute the documented flow rather than only parsing --help.

Changes:

• Add -o/--output to granule train and print an explicit “not saved” message when omitted.
• Make fast training the CLI default (--naive selects the reference trainer; legacy --fast remains accepted but hidden).
• Make encode accept nargs="+" and improve --decode parsing to accept commas/whitespace/newlines; expand CLI/Quickstart tests accordingly.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/test_cli.py	New focused regression tests covering train save behavior, trainer default, encode/decode composition, and avoiding private tokenizer access.
tests/test_b7_quickstart.py	Strengthens Quickstart validation by parsing pipelines, checking coherence (train output vs encode input), and executing the sequence.
src/granule/tokenizer.py	Adds a public merges property returning a copy to prevent external mutation from desyncing derived state.
src/granule/cli.py	Implements -o/--output, defaults training to fast, and improves decode parsing + xargs-friendly encode argument handling.
README.md	Updates Quickstart commands/documentation to reflect required -o, default fast training, and encode/decode composability.
pyproject.toml	Pins ruff src layout explicitly to avoid version-dependent import classification differences.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.