Sync History shows 0 tokens and $0.0000 because memory inference spend is never metered

[YellowSnnowmann]

Summary

Brain › Sync › Sync History renders every run with 0 tokens and $0.0000, and the header totals read 0 in / 0 out · $0.0000 total. The reader and the UI are correct — no writer anywhere puts a non-zero token or cost figure on a source-sync audit row. Underneath it is a wider gap: memory inference spend never reaches the host cost tracker at all, so it is also missing from openhuman.cost_get_dashboard and from budget accounting.

Problem

What happened — 12 sync runs listed, Items correct (50, 50, 0, 50…), Tokens 0 and Cost $0.0000 on every row.

What was expected — real provider-reported token counts and cost for runs that embedded 506 chunks against cloud voyage-3-large.

Impact

• The panel states a sync was free. That is the one answer indistinguishable from the truth, and it is never true for a cloud embedder.
• AgentActivityPanel's monthly memory cost (openhuman.memory_sources_monthly_cost_summary) is $0.00 for the same reason.
• Memory spend is absent from the cost dashboard and from CostTracker::check_budget, which agent/tinyagents/host/budget_gate.rs uses for budget refusal and accounting. Memory inference is currently unbudgeted and unreported.

Steps to reproduce

1. Connect a Composio source (GitHub/Gmail/Notion/Slack) with a cloud embedder configured.
2. Brain › Sources › Sync on that source; wait for the run to finish.
3. Brain › Sync › Sync History → the new row shows Items: 50, Tokens: 0, Cost: $0.0000.
4. Settings › Agent Activity → monthly memory cost $0.00.

Environment — dev build on main @ 2475ea342, tinymemory pin v1.16.0, macOS. Staging profile ~/.openhuman-staging/users/69dc8c37935c9a8c539b43f2.

Root cause

Four independent layers. All four must be addressed or the symptom survives.

L1 — host writer hardcodes zeros. crates/openhuman-core/src/memory/sources/run_history.rs:96-100 — HostRun::into_entry writes batches: 0, input_tokens: 0, output_tokens: 0, estimated_cost_usd: 0.0, and HostRun has no token fields to carry. Covers every Sync click, Apply-all and Composio run (10 of the 12 rows).

L2 — driver writer hardcodes the same. tinymemory crates/tinymemory-core/src/sources/sync.rs:182-184 and :228-230, the periodic folder/GitHub/RSS/web loop (the other 2 rows). Zero since the file was created at tinymemory v1.5.0.

L3 — usage is produced, then dropped mid-chain.

stage	state
provider returns UsageInfo (tinymemory-api/src/host/usage.rs)	real tokens + charged_amount_usd
tree::summarise::summarise (tinymemory-core/src/tree/summarise.rs:169-188)	populates SummaryOutput.input_tokens
HostSummariser::summarise (tinymemory-core/src/engine/summariser.rs:52)	returns .output, drops usage
bucket_seal.rs:274 / document_seal.rs:196	call .summarise(), not summarise_with_usage()
audit row	zeros

summarise_with_usage() has exactly one production caller: tinycortex/memory/sync/rebuild.rs:208, the engine rebuild path. RealCostAccumulator exists and is tested — wired only to rebuild, never to source sync. Embeddings have no token accounting anywhere.

L4 — the host cost tracker is never called from memory. cost::record_provider_usage and cost::record_embedding_usage exist, persist durably, and feed openhuman.cost_get_dashboard. Neither has a single caller from modules/memory_host.rs or from inference/embeddings/.

L4c — tinyinference never surfaces embedding usage. EmbeddingModel::embed returns Result<Vec<Vec<f32>>> (tinyinference/src/embeddings/types.rs:51) and no file under tinyinference/src/embeddings/ parses a usage object. Every real embedding request routes here via TinyAgentsEmbeddingProvider (inference/embeddings/provider_trait.rs:86), including the managed cloud path, which is only a decorator over it (cloud_adapter.rs:153). The serde_json::Value decode in inference/embeddings/factory.rs belongs to DimensionAgnosticOpenAiProbe, the Test-connection probe — not production embedding traffic. So provider-reported embedding tokens cannot be obtained host-side.

L4b — UI drops the fields that could already be non-zero. SyncAuditPanel.tsx:216,297 sums and renders estimated_cost_usd only. Rust has SyncAuditEntry::effective_cost_usd() (actual_charged_usd.unwrap_or(estimated) + composio_cost_usd) and summarise_month already uses it; the panel does not. The TS interface at app/src/utils/tauriCommands/memoryTree.ts:1077-1099 does not even declare composio_cost_usd, actual_charged_usd or composio_actions_called, though the wire carries all three.

Why it regressed

• feat(memory): surface actual LLM usage/cost from inference API in sync audit #3110 / PR feat(memory): record real LLM cost in sync audit (#3110) #3150 (2026-06-01) built real-cost accounting, landing it on memory_sync/sources/{github,rebuild}.rs — the legacy paths.
• 96c370314 (refactor(memory): complete TinyCortex engine migration #4794, TinyCortex migration) gutted github.rs 861→few lines and rebuild.rs 836→few; the accounting moved into tinycortex, where only rebuild kept it.
• 4ce2a8c7c (refactor(memory): retire legacy sync compatibility paths #5246, "retire legacy sync compatibility paths") deleted the remainder.
• memory_sources/sync.rs — the Sources path that survived and is now the only path — always wrote zeros, even pre-migration. It never received feat(memory): surface actual LLM usage/cost from inference API in sync audit #3110's work.

#3110 is closed as completed against a code path that no longer exists.

Field proof

~/.openhuman-staging/users/69dc8c37935c9a8c539b43f2/workspace/state/memory_sync_runs.jsonl — 10 rows, plus 2 in memory_tree/sync_audit.jsonl = the 12 on screen.

{"timestamp":"2026-09-15T13:13:01Z","source_kind":"composio","scope":"github:ca_leUcIgnTw0DJ",
 "items_fetched":50,"batches":0,"input_tokens":0,"output_tokens":0,"estimated_cost_usd":0.0,
 "composio_actions_called":0,"composio_cost_usd":0.0,"actual_charged_usd":null,
 "duration_ms":35511,"success":true}

Every row identical in the cost half.

Solution

Measured numbers only — no item-count estimates. Both inference seams already execute in the openhuman host process: the module installs BusEmbeddingHost / BusChatHost (tinymemory-module/src/lib.rs:136-143) which call back over tinybus into EmbeddingCallbacks::embed and ChatCallbacks::complete in crates/openhuman-core/src/modules/memory_host.rs. ModelResponse already carries usage. So the host can meter 100% of memory inference spend with no contract change.

The chat half is reachable host-side; the embedding half is not, and embeddings are the dominant memory spend (a sync that embeds 506 chunks may summarise with the no-LLM fallback_summary and spend nothing on chat at all). So the embedding half sets the PR count.

Repo nesting that fixes the order: openhuman → vendor/tinyagents → vendor/tinyinference, with openhuman's root Cargo.toml patching tinyinference to vendor/tinyagents/vendor/tinyinference/crates/tinyinference. A tinyinference change therefore reaches openhuman only through a tinyagents pointer bump.

PR 1 — tinyinference

Add EmbeddingModel::embed_with_usage returning vectors alongside an optional provider-reported usage, with a default implementation delegating to embed and reporting None, so no existing implementor breaks. Parse the usage object in the OpenAI-compatible, Voyage and Cohere models (all three return it); Ollama and Noop keep the default. Release.

PR 2 — tinyagents

Bump the vendor/tinyinference submodule pointer to that release. No code.

PR 3 — openhuman

1. Bump vendor/tinyagents, taking the new tinyinference through the existing patch.
2. inference/embeddings/provider_trait.rs — call embed_with_usage and, when usage is reported, record_embedding_usage(provider, model, tokens, dims, vectors). embed()'s public signature is unchanged; a provider that reports nothing records nothing.
3. modules/memory_host.rs — ChatCallbacks::complete maps ModelResponse.usage → UsageInfo and calls record_provider_usage. build_token_usage already skips all-zero payloads and sets CostSource::ProviderCharged when the backend echoes a charge.
4. No new tag needed: ai.tinyhumans.tinymemory.EmbeddingHost is memory-only, and on the chat side role already discriminates ("summarization" is special-cased at memory_host.rs:129).
5. UI — Sync History header shows measured memory spend for the window; per-row Tokens/Cost render — with a tooltip rather than a fabricated $0.0000. TS type gains the three missing fields plus an effectiveCostUsd() mirroring the Rust accessor.

Outcome: real memory spend visible in Sync History and in the existing cost dashboard, and memory spend starts counting toward budget accounting. Per-row attribution still blank.

Separable now: steps 3–5 of PR 3 depend on nothing upstream and can ship first as an openhuman-only PR. That stops the $0.0000 claim immediately and meters summarisation, but on a profile using fallback_summary it will record nothing until the embedding half lands.

PR 4 — tinymemory (only if per-row attribution is wanted)

EmbeddingHost::embed and ChatHost::complete gain a trailing scope: Option<String>, appended at tail (MINOR — wire slots are append-only); bucket_seal.rs / document_seal.rs pass the source scope they already hold. No accounting in this PR — the host does the metering. Release.

PR 5 — openhuman

Re-pin rides inside this PR (precedent: #6269 pinned tinyconnectors v0.10.0 in the fix PR). TokenUsage gains a scope field (host-internal type, no contract); sync_audit_log_rpc joins the ledger by scope and fills the per-row columns.

Why per-row needs PR 4+5

The bus calls carry role, provider, model — no source id. Sealing is a queued async job (mem_tree_jobs), batched across sources and running after the run's row is written, so time-window attribution across concurrent syncs would be a guess rather than a measurement.

Acceptance criteria

• Repro gone — after a sync with a cloud embedder, memory spend is non-zero and provider-reported wherever it is shown; no surface claims $0.0000 for a run that cost money.
• No fabricated numbers — a provider that omits usage records nothing and logs at debug; no item-count estimate is presented as a measurement.
• Memory embedding and summarisation spend appears in openhuman.cost_get_dashboard.
• Memory spend reaches CostTracker, so check_budget accounts for it.
• SyncAuditPanel renders actual_charged_usd + composio_cost_usd when a row carries them, and — (not $0.0000) when a row genuinely has no cost recorded.
• Rows written before this change still parse and render — every added field serde(default).
• Regression safety — unit coverage on the two new metering call sites and on the UI's zero/absent distinction.
• Diff coverage ≥ 80% — Vitest + cargo-llvm-cov.

Related

• feat(memory): surface actual LLM usage/cost from inference API in sync audit #3110 — built real-cost accounting, landed on paths later deleted; closed completed.
• feat(memory): record real LLM cost in sync audit (#3110) #3150 — the implementing PR.
• refactor(memory): complete TinyCortex engine migration #4794 (96c370314), refactor(memory): retire legacy sync compatibility paths #5246 (4ce2a8c7c) — the migrations that removed the accounting.
• Memory sources: Sync History never records a manual sync, and "Queued" points at a queue no screen shows #6257 / fix(memory): record every sync run in Sync History and show live sync activity #6264 — made Sync History record runs at all; this issue is the cost half that stayed zero.
• Memory-tree corruption reported as "non-fatal" for 34min while the UI showed successful syncs; recovery then wiped the source registry #5820 — the tree_ingest_failures partial verdict on the same row shape.

[YellowSnnowmann]

Corrected the solution section after tracing the embedding path properly.

The first draft claimed openhuman could read the provider's usage object in inference/embeddings/factory.rs. That embed() is DimensionAgnosticOpenAiProbe — the Test-connection probe — not production traffic. Real embedding requests go through TinyAgentsEmbeddingProvider into tinyinference, whose EmbeddingModel::embed returns Result<Vec<Vec<f32>>> and parses no usage anywhere under src/embeddings/. The managed cloud provider is only a decorator over the same path.

Consequence: provider-reported embedding tokens are not reachable host-side, so the fix crosses three repos (openhuman → vendor/tinyagents → vendor/tinyinference, wired through the root Cargo.toml patch). Added as L4c, and the solution is re-cut as tinyinference → tinyagents → openhuman.

The chat/summarisation metering and the UI half stay host-only and can ship first, with the caveat that a profile summarising via fallback_summary spends nothing on chat, so those alone will still show no memory cost.