Connector syncs stopped feeding the memory tree after the TinyCortex migration

[yh928]

Summary

96c370314 (#4794) deleted the per-provider source.rs modules that batch-ingested each connector's synced items into the memory tree. The replacement pipeline writes to the base namespace store only, so the summary tree has received nothing from Gmail, Slack, Notion, GitHub, Linear, or ClickUp since. Reconnect the tree-ingest side on the new engine.

Problem

The migration replaced the connector sync pipeline and did not carry over its tree-ingest half. Deleted in that commit:

memory_sync/composio/providers/gmail/source.rs      ← "Batch-ingest the page's new
memory_sync/composio/providers/gmail/ingest.rs         messages into the memory tree"
                             slack/source.rs           (ingest_page_into_memory_tree_with_outcome)
                             notion/source.rs
                             github/source.rs
                             linear/source.rs
                             clickup/source.rs

Nothing replaced them. tree_summarizer_ingest has no production caller — only the RPC handler and the CLI — so on the current tree the only way to add tree content is an explicit operator call.

Observed on a live instance:

gmail base namespace	current — syncs run several times a day, documents and chunks landing continuously
gmail tree source	620 chunks, newest content dated 2026-07-14, nothing since
conversations:agent tree source	still growing (chat has its own path)

The sync audit log shows the connector itself is healthy: 6 syncs on the observation day, 13 items on a manually triggered one. The items reach skill-gmail; they never reach mem_tree_chunks.

Note this is the same commit that removed the only caller of the Composio response reshape (restored separately via ReshapingExecutor). Two halves of the sync path came off in the same migration; only one has been put back.

Solution (optional)

Reconnect tree ingestion in the new pipeline — memory::tinycortex::sync is where the connector run now lives — rather than reviving the deleted modules. The old code ingested per page with a partial-ingest outcome (item_ids_ingested) that drove the cursor, so the replacement needs the same property: a short ingest must not advance the cursor past items that were never stored.

Worth deciding as part of this:

• Scope. All six connectors regressed together. Doing one and leaving five is the state we are already in.
• Backfill. Existing installs have a gap between the migration and the fix. A re-ingest of already-synced documents would close it; doing nothing leaves a permanent hole in the tree.
• Cost. Tree ingest means sealing, which means summarization calls. On a large mailbox a naive backfill is expensive, and the summarizer is consent-gated when local AI is off.

Acceptance criteria

• Synced connector items reach the tree — a Gmail sync adds mem_tree_chunks rows for the newly ingested messages.
• All six connectors — gmail, slack, notion, github, linear, clickup.
• Partial ingest does not advance the cursor — the property the deleted item_ids_ingested outcome carried.
• Regression coverage — a test that fails if the sync path stops writing tree chunks, so this cannot come off silently a third time.
• Backfill decided — implemented, or explicitly declined with the gap documented.
• Diff coverage ≥ 80% — the implementing PR meets the changed-lines coverage gate.

Related

• 96c370314 (refactor(memory): complete TinyCortex engine migration #4794) — the migration that dropped it.
• src/openhuman/memory/tinycortex/sync.rs — where the connector run lives now.
• src/openhuman/memory/sources/status.rs — reports MAX(timestamp_ms) as "Last chunk", which is the content date, not the write time; that is why the freeze read as "22 days ago" rather than "stopped at the migration".

[tinysweeper]

Connector syncs stopped feeding the memory tree after a migration deleted the per-provider source.rs tree-ingest modules. The new pipeline writes only to the base namespace store, so synced items never reach the memory tree. A core path is broken with no practical workaround (manual RPC/CLI calls aside).

Labelled priority: p1.

This may be related to #5300. Left open either way — tinysweeper only closes an issue when the match is unambiguous.

[YellowSnnowmann]

Where the fix lives

The regression is entirely in the connector-sync engine, which now lives in the vendored tinymemory → tinycortex stack, so the fix is one PR there plus a follow-up pointer bump here:

Half	Covered by
Reconnect tree ingestion for Composio connector syncs (Gmail/Slack/Notion/GitHub/Linear/ClickUp)	tinymemory #13
Pull that fix into this app	follow-up submodule-pointer bump (this repo)

What #13 does — routes each synced item through the engine's document ingest at the Composio SkillDocSink seam (the same L0-chunk path local folder sources already use), so items land in mem_tree_chunks again and return to tree-backed recall. Tree scope is {toolkit}:{connection_id} so retrieval resolves it by platform prefix; each item keeps a distinct source_id so messages admit independently. Tree ingest is best-effort (a failure is logged and the sync continues; the skill store remains the source of truth), and the existing skill-store write is preserved. All six connectors are covered by the single shared seam.

Behavior note carried in #13 — items are stamped with ingest time (modified_at = now()), not the item's true source time, because SkillDocument carries no uniform timestamp at this seam. Threading real per-item timestamps needs the provider/orchestrator layer and is a tracked follow-up.

Backfill — declined by design. Composio writes to the skill store, not the raw archive, so a rebuild-from-raw can't see the gap items. Existing installs close the migration gap by re-syncing (or the operator rebuild RPC); documented rather than auto-backfilled.

Closing plan — #13 is approved and green on tinymemory. Once it lands on main, a follow-up openhuman PR bumps the vendor/tinymemory submodule pointer to pick it up and carries Closes #5473. That bump can't be raised earlier because a submodule can only point at a commit that already exists on tinymemory main.