Skip to content

Compare spend-log rows with daily request counts - #53

Merged
kwisatzh merged 2 commits into
mainfrom
feat/litellm-row-coverage
Oct 15, 2026
Merged

kwisatzh merged 2 commits into
mainfrom
feat/litellm-row-coverage

Conversation

@kwisatzh

@kwisatzh kwisatzh commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add optional investigate --litellm-daily-spend FILE coverage beside a
LiteLLM request export, for both two-period and session-review modes.

  • Compare selected UTC days' exported rows with daily API, successful and
    failed request counts. Keep unknown and partial-day coverage explicit.
  • Leave token, spend and ranking totals unchanged; omitting the option
    preserves existing report bytes.
  • Add a content-free daily SQL projection and a private paired-export helper
    using one repeatable-read, read-only transaction.
  • Bound the daily reader and extend privacy, fixture and offline checks.
  • Correct the export guide's installation label and link to v0.8.0. Row
    coverage itself is unreleased and is not a v0.8.0 capability.

Evidence And Limits

Pinned LiteLLM v1.104.0 and PostgreSQL 17.11 were exercised with a loopback
mock provider and network-disabled containers, without paid API calls or
direct insertion of the request/count evidence rows.

Two successful requests sharing a provider response ID produced one persisted
request row but two daily increments. The fixture reports 8 exported rows
versus 9 daily requests, one missing row, followed by a matching next day.
Token totals remain unchanged.

Success, failure, streaming, response-cache hits, absent user identity and UTC
day boundaries are covered. Disabling failure logging omits the failed request
from both tables. Matching counts cannot prove universal request completeness;
matching scope, retention and settled writers remain prerequisites.

Verification

  • Full Go 1.26.9 race tests and vet; 10,000 daily-reader fuzz cases.
  • Eight existing output goldens and five legacy identity/hash goldens.
  • 108 default CLI combinations byte-identical to the base.
  • Fixture, documentation, privacy and guarded export safety tests.
  • Pinned writer/database matrix and three proxy-persistence profiles.
  • Eight Linux/arm64 network-disabled equivalence cases.
  • Increase the packaged SQL-suite timeout from five to ten minutes. The first
    Linux amd64 hosted run exhausted the five-minute limit while running the
    expanded database suite; no assertion failed, and all tests remain enabled.

Both historical v0.7.0 reader-compatibility checks passed using the retained
release binary matched to its authenticated archive, published checksums and
recorded signed GitHub provenance verification. No new download was needed.
The checks cover additive-column synthetic exports and actual database exports.

Hosted checks on the submitted exact head remain required. No release,
deployment ranking, pricing adjustment or completeness certification is included.

Signed-off-by: Vijay <evijay@gmail.com>
Signed-off-by: Vijay <evijay@gmail.com>
@kwisatzh
kwisatzh marked this pull request as ready for review October 15, 2026 01:02
@kwisatzh
kwisatzh merged commit 77b6fdf into main Oct 15, 2026
21 checks passed
@kwisatzh
kwisatzh deleted the feat/litellm-row-coverage branch October 15, 2026 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant