Skip to content

feat: compare cache reuse and prepare v0.7.0 - #46

Merged
kwisatzh merged 4 commits into
mainfrom
feat/cache-reuse-investigation
Oct 11, 2026
Merged

kwisatzh merged 4 commits into
mainfrom
feat/cache-reuse-investigation

Conversation

@kwisatzh

@kwisatzh kwisatzh commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Prepare v0.7.0: compare recorded cache reuse before and after a change, then rank keys, teams or sessions by uncached-input growth.

  • Include pinned LiteLLM cache-read, cache-write and reasoning aliases by default; usage_details=false opts out. Absent writer paths stay NULL.
  • Show per-model cache-read/write shares and human-readable request coverage. Missing/invalid detail is excluded and counted when nonzero; recorded zeros retain unknown origin.
  • Put significant cache changes immediately below the headline. Require complete cache-read coverage and either a five-point share change or an uncached-input change of at least 100 tokens and 20%. Incomplete call-type comparisons keep their coverage-first headline.
  • Point falling cache-read shares to provider diagnostics and gateway cache-marker checks. These are usage comparisons, not cause or cost attribution.
  • Extend summary comparisons with independently checked aggregate read/write coverage. Existing summaries lack per-model cache totals and cache-weighted contributor rankings; the report identifies that gap.
  • Ship a byte-checked synthetic OTLP capture for binary-only inspect.
  • Prepare v0.7.0 release notes, export upgrade instructions, installer defaults and README links. README: 165 lines.

Synthetic example:

Recorded tokens were unchanged (2,200 -> 2,200).
But cache reuse fell: cache-read share 92% -> 61% (uncached input +620).

The detail reads Cache reads: 1,840 of 2,000 input tokens -> 1,220 of 2,000 (all requests report cache detail). One key adds 620 uncached input tokens. Keyed rankings and bounds compose the existing accounting and sketch implementations.

Pinned Writer Evidence

The tests exercise LiteLLM v1.104.0's writer and native logging normalizers with synthetic responses, PostgreSQL, the read-only SQL projection and the importer. All eight supported stored call-type spellings are covered. Messages can normalize absent cache fields to zero; Responses leaves some detail paths absent. The guide records the observed table. These are offline writer/export tests, with no paid provider calls.

Verification

  • Race suite and vet pass.
  • Pinned LiteLLM/PostgreSQL and privacy suite: all 20 tests pass, no skips.
  • Headline boundaries, material growth/decrease, missing/partial coverage, clean exclusions, summary/spend wording and unchanged JSON accounting are covered by tests.
  • Fresh anonymous clone at eedaa5a2a0878054e5ac1f069ad87275ac9167f4: race tests, vet, all 20 pinned database/privacy tests (no skips), 1,000 spend-reader fuzz executions and 10,000 diagnose fuzz executions pass with fresh caches.
  • The cache excerpt matches; the shipped capture runs with Go absent from PATH; generated capture and session transcript match byte for byte; the clone stays clean.
  • Four development archives built and checksum-verified. Hosted checks exercise each platform's packaged binary before release; this update does not create release artifacts or tags.

Approved Release Sequence

The maintainer approved merging and publishing v0.7.0 after final-head hosted checks. A signed, signed-off documentation commit dates the changelog and finalizes the cache guide. The existing v0.6.1 release remains unchanged.

After tag workflow checks, independently verify the draft release's artifacts before publication, then run the public-download SQL export -> fresh install -> investigate outsider check. Update site guides to v0.7.0 (trace-check uses the shipped capture without Go), collector recipes to separately verified v0.3.2 images, then the cache-check guide. The release checklist records that order.

Signed-off-by: Vijay <evijay@gmail.com>
Signed-off-by: Vijay <evijay@gmail.com>
Signed-off-by: Vijay <evijay@gmail.com>
@kwisatzh kwisatzh changed the title feat: compare cache reuse and uncached input growth feat: compare cache reuse and prepare v0.7.0 Oct 11, 2026
Signed-off-by: Vijay <evijay@gmail.com>
@kwisatzh
kwisatzh marked this pull request as ready for review October 11, 2026 20:02
@kwisatzh
kwisatzh merged commit 25694f0 into main Oct 11, 2026
21 checks passed
@kwisatzh
kwisatzh deleted the feat/cache-reuse-investigation branch October 11, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant