feat(extractor): filter redundant ghost user_ids during cc_session sync - #1
Open
hrdAI3 wants to merge 1 commit into
Open
feat(extractor): filter redundant ghost user_ids during cc_session sync#1hrdAI3 wants to merge 1 commit into
hrdAI3 wants to merge 1 commit into
Conversation
Matrix-Riven's getUserId() in the realtime path falls back to
`${unix_user}@${hostname}` when `git config user.email` is unset on the
client machine. Pre-PR #3 (libz-renlab-ai/Matrix-Riven), that fallback
ALSO fired even when `~/.riven/digital-twin.json` carried a real
identity.user_id — producing same-machine pairs where the Stop hook
(transcript upload) tagged data with the real email and the realtime
hooks (cc-status snapshots) tagged data with the hostname form.
On the collector this surfaces as "ghost user_id" directories sitting
beside real-email directories. Two flavors:
1. Redundant ghost — same session_ids as a real-email peer on the
same machine; ghost only holds cc-status fragments, real peer
holds the full transcript. Ingesting the ghost double-counts.
2. Sole-identity ghost — that hostname form IS the person's only
upload identity (their git config user.email genuinely unset,
they never ran `riven digital-twin login`). Their transcripts
ONLY exist under the ghost id. Must be preserved.
Real-world snapshot taken from prod (192.168.22.88:8933):
19723@hut → redundant (paired with
horton2048@users.noreply.github.com)
lv@lvjiawendeMacBook-Air.local → sole identity (12MB+ transcripts)
blink@BlinkdeMacBook-Air.local → sole identity
zhangziyi@zhangziyideMacBook-Air-2.local → sole identity (16 transcripts)
alexpeng@pengchengdeMacBook-Air.local → sole identity
Implementation:
- `src/lib/ghost_user.ts` — pure shape predicate `isGhostUserId()`
(allowlist of email TLDs + `.local` hostname suffix) and a
`canonicalSessionId()` that strips the `.cc-status` listing
suffix the collector uses to distinguish snapshot entries from
transcripts.
- `syncCcSessions` adds a pre-pass scanning all non-ghost users'
transcript session_ids (NOT cc-status entries) into a global
Set. Main loop then skips any ghost session whose canonical id
is in that Set — emitting nothing to events.jsonl for it. The
summary surfaces a new `redundantGhostsSkipped` counter.
- Sole-identity ghosts pass through untouched.
Tests: 8 cases on `isGhostUserId` + `canonicalSessionId` covering
real emails (qq/outlook/163/nb-ai/gmail/github noreply), bare-hostname
fallback (`19723@hut`), `.local` mDNS, edge cases, and the unknown-TLD
conservative-lean.
All 8 new tests pass; pre-existing test failures on main
(bootstrap/extract, evolution/diff, lib/agents) are unrelated to this
change — confirmed by stash-and-rerun comparison.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Skip "redundant ghost" cc-session uploads from the Matrix-Riven collector so the workboard doesn't double-count the same physical CC session under two user_ids.
Background
Matrix-Riven's realtime path falls back to
\${unix_user}@\${hostname}whengit config user.emailis unset (and pre-PR #3 over there, also when the digital-twin config had a real identity — that PR fixes new sessions but historical data + clients that haven't reinstalled yet still produce ghosts). On the collector this surfaces as same-machine pairs:hrdai@qq.com(real)19723@hut(ghost)After cleanup the live
/api/userslist still has 4 sole-identity ghosts whose data only exists under the hostname form:These four MUST NOT be filtered — they ARE those people's only upload identity until they set
git config user.emailand reinstall the client.What this does
Two pure helpers in
src/lib/ghost_user.ts:isGhostUserId(userId)— shape predicate. Real-email TLD allowlist +.localhostname suffix + bare-hostname (no dot) classification. Conservative on unknown TLDs (leans real).canonicalSessionId(rawId)— strips the trailing.cc-statusthe collector appends to snapshot listing entries.syncCcSessionsgets a pre-pass:.cc-statussuffix) into a global Set.Main loop then for each ghost user's session: if
canonicalSessionId(sess.id)is in that Set → skip ingestion (don't fetch raw, don't emit events, don't write to events.jsonl), bump a newsummary.redundantGhostsSkippedcounter, advancelastMtimeso we don't re-evaluate the same entry next sync.Sole-identity ghosts (their session_ids have no non-ghost peer) pass through untouched and emit
cc.*events under their hostname-form user_id, same as before.Tests
8 cases in
tests/lib/ghost_user.test.ts— real email TLDs (qq.com / outlook.com / 163.com / nb-ai.com / gmail.com / GitHubusers.noreply.github.com), bare hostname (19723@hut),.localmDNS (blink@...,lv@...,zhangziyi@...,alexpeng@...), edge cases (empty, no-at, trailing-at), and the unknown-TLD conservative-lean (person@company.example→ not flagged).All 8 pass.
bun run testshows 5 pre-existing failures (bootstrap/extract.parseOrgChart, evolution/diff.path-allowlist, lib/agents.writeProfile+updateState) — confirmed unrelated by stash-and-rerun on clean main.Not done in this PR
Test plan
bun run typecheck— no new errors in changed files (pre-existing errors in vector_match + workboard are unrelated)bun run test tests/lib/ghost_user.test.ts— 8/8 passredundantGhostsSkipped > 0if any ghost-paired sessions land in window🤖 Generated with Claude Code