fix(db): stop v11 from clobbering non-claude harness labels (#19) - #47
Conversation
v11's canonical-session migration unconditionally set
`harness = COALESCE(harness, 'claude')` for every pre-existing
conversation, before `_backfill_exchange_provenance` (which infers
harness from source_path) ever ran. Since v11 always filled the column
first, the backfill's `WHERE harness IS NULL` guard never matched, so
grok/omp-pi/opencode/codex exchanges predating v11 stayed mislabeled
as claude forever.
Extract the source_path -> (harness, source_session_id) heuristic into
a single `_infer_harness_and_source_session_id` helper shared by v11
and `_backfill_exchange_provenance`, so both derive identical labels
and session_ids instead of one silently overriding the other.
Also fixes `file_renames.py`'s `git log --follow` call to pass
`-c core.quotepath=false`, so non-ASCII paths aren't returned as
octal-escaped quoted strings (`"caf\303\251.py"`), which broke rename
alias matching for such files.
Investigated the third checklist item (grok/omp-pi/opencode missing
git_branch) against tests/fixtures/harness_logs/README.md, which
documents key names/nesting verified against real logs (grok: 39,
omp-pi: 99, opencode: 1). None of the three carry a git-branch field
anywhere in their raw schema (grok's ACP session/update envelope,
omp-pi's `{type: "session", cwd}` envelope, opencode's
project/session/message/part tables) — unlike claude's `gitBranch` or
codex's `session_meta.git.branch`. Left `git_branch=None` for all
three with comments recording this finding instead of fabricating an
extraction path or a live `git branch` shell-out (which would record
the branch at index time, not at conversation time, diverging from
what the field means for claude/codex).
Closes #19
| 直後に走る _backfill_exchange_provenance の `WHERE harness IS NULL` が | ||
| ヒットせず、非claude 由来(grok/omp-pi/opencode/codex)の会話も claude 誤ラベルの | ||
| まま恒久化していた(issue #19)。ヒューリスティックを1箇所に集約し、 | ||
| 両者が常に同じ判定・同じ session_id を導出するようにする。 |
There was a problem hiding this comment.
Priority-1 gap: this only fixes the v11 migration going forward for databases whose PRAGMA user_version is still below 11. Databases that already ran the buggy v11 migration already have harness='claude' and a claude:-derived session ID written; _run_migrations skips v11 for them (already applied), and _backfill_exchange_provenance (~line 422) only selects rows where session_id IS NULL OR harness IS NULL" — which is none of them, since v11 already filled both. So every legacy grok/omp-pi/opencode/codex exchange that was mislabeled by the old buggy v11 stays permanently mislabeled after this fix. Please add an idempotent repair migration/backfill that identifies non-Claude source_path values among already-claude`-labeled legacy rows and rewrites their harness/session_id/canonical exchange id — not just fix the forward path.
There was a problem hiding this comment.
Good catch — fixed in 328f40a.
Added migration v13 (_migrate_v13_repair_legacy_claude_mislabel) that specifically targets DBs already past the buggy v11 (PRAGMA user_version >= 11). It walks every conversations row, re-derives the correct harness from source_path with the same _infer_harness_and_source_session_id heuristic used by v11/backfill, and for any exchange whose stored harness disagrees, rewrites harness, session_id, source_session_id, and — when the column already exists (i.e. _backfill_canonical_exchange_ids already ran against the stale values) — canonical_exchange_id. It also drops the stale claude-hashed sessions row for that conversation once nothing references it anymore (safe: that id is sha256("claude:" + source_path), unique per conversation since source_path is unique).
It's registered in _MIGRATIONS so it runs exactly once per DB via the normal PRAGMA user_version gate, and is independently idempotent (re-deriving already-correct rows is a no-op) if ever invoked again.
Regression test: test_migration_v13_repairs_exchanges_already_mislabeled_by_old_buggy_v11 builds a DB frozen at user_version=12 with harness='claude' and a claude:-derived session_id/canonical_exchange_id already persisted for a grok conversation — reproducing exactly the state a DB that already ran the old buggy v11 would be in. I verified it fails (assert 'claude' == 'grok') with v13 unregistered and passes with it registered.
Also had to fix test_migration_v12_adds_exchange_conversation_ply_index_to_existing_db, which hardcoded len(_MIGRATIONS) - 1 assuming v12 was the last migration; it now targets _MIGRATIONS.index(_migrate_v12_add_exchange_conversation_ply_index) so it still isolates v12 regardless of how many migrations exist.
make check run directly (not via git hooks, per issue #46): 664 passed, 0 lint issues, 0 typecheck errors.
Review feedback on #47: the previous fix only stops v11 from mislabeling *new* migrations (DBs still below user_version 11). Any DB that already ran the old buggy v11 already has harness='claude' and a claude-derived session_id/canonical_exchange_id written for non-claude conversations. _run_migrations skips v11 for such DBs (already applied per PRAGMA user_version), and _backfill_exchange_provenance only touches rows WHERE session_id IS NULL OR harness IS NULL — which is none of them, since the old v11 already filled both. Those rows stayed permanently mislabeled. Add migration v13 (_migrate_v13_repair_legacy_claude_mislabel): walks every conversation, re-derives the correct harness from source_path via the same _infer_harness_and_source_session_id heuristic, and for exchanges whose stored harness disagrees, rewrites harness/session_id/source_session_id and (when the column already exists) canonical_exchange_id. Also drops the stale claude-hashed sessions row for that conversation once nothing references it anymore (its id is unique per source_path, so this is safe). Naturally idempotent: re-deriving already-correct rows is a no-op, and as a versioned migration it only runs once per DB. Also fixed test_migration_v12_adds_exchange_conversation_ply_index_to_existing_db, which hardcoded `len(_MIGRATIONS) - 1` assuming v12 was the last migration; now targets `_MIGRATIONS.index(_migrate_v12_add_exchange_conversation_ply_index)` so it still isolates v12 regardless of migration count. Verification: - RED: added test_migration_v13_repairs_exchanges_already_mislabeled_by_old_buggy_v11, which builds a DB frozen at user_version=12 with harness='claude' and a claude-derived session_id/canonical_exchange_id already written for a grok conversation (simulating a DB that already ran the pre-fix v11). Confirmed it fails with `assert 'claude' == 'grok'` when v13 is not registered in _MIGRATIONS, and passes once it is. - make check (ruff + pyright + full pytest suite): 664 passed, 0 lint issues, 0 typecheck errors.
Closes #19
Root cause
v11 harness mislabeling (checklist item 1).
_migrate_v11_add_canonical_sessionsunconditionally ranharness = COALESCE(harness, 'claude')for every pre-existingconversationsrow, regardless of the actual harness. The very next step,_backfill_exchange_provenance(which infers harness fromsource_path—opencode.db#,rollout-,/.omp/,/.grok/), only touches rowsWHERE harness IS NULL. Since v11 always filled the column first, that guard never matched, so any grok/omp-pi/opencode/codex conversation indexed before v11 stayed permanently mislabeled asclaude.file_renames quotepath (checklist item 3).
_run_git_followcalledgit log --follow --name-status ...without-c core.quotepath=false. Git's defaultcore.quotepath=truerenders non-ASCII paths as octal-escaped, double-quoted strings (e.g. a literal"caf\303\251.py"), soresolve_aliasesreturned that mangled string instead of the real filename — rename tracking silently broke for any non-ASCII path.grok/omp-pi/opencode git_branch (checklist item 2) — investigated, not a fabrication candidate. Checked each harness's real raw schema against
tests/fixtures/harness_logs/README.md, which documents key names/nesting verified directly against real logs (grok: 39 sessions, omp-pi: 99 sessions, opencode: 1 realopencode.db, re-verified 2026-08-30 specifically for adapter implementation):session/updateenvelope (timestamp,method,params: {sessionId, update}) — no git field anywhere intool_call/tool_call_update/ message-chunk entries.{type: "session", version, id, timestamp, cwd}— only a working directory, no branch.project(id, worktree, vcs, name) /session(id, project_id, directory, time_created, model) /message/parttables — no branch column or key anywhere in thedataJSON blobs.None of the three carry a git-branch field anywhere in their data, unlike claude's
gitBranchor codex'ssession_meta.git.branch. I did not add a livegit branch --show-currentshell-out against the recordedcwd/worktreeas a substitute: that would record the branch at index time, not at conversation time — a different (and less accurate) semantic than whatgit_branchmeans for claude/codex, and it risks recording the wrong branch if the user has since switched. Leftgit_branch=Nonefor all three, with comments at each call site recording this finding so it isn't silently rediscovered later.Fix
source_path -> (harness, source_session_id)heuristic into a single_infer_harness_and_source_session_idhelper insrc/codeatrium/db.py, used by both_migrate_v11_add_canonical_sessionsand_backfill_exchange_provenance. They now always agree on harness and derive identicalsession_idhashes instead of one silently overriding the other.src/codeatrium/file_renames.py:_run_git_follownow passes-c core.quotepath=falsetogit log.src/codeatrium/indexer.py: documented the git_branch investigation at the threegit_branch=Nonesites (parse_grok_exchanges,parse_omp_pi_exchanges,parse_opencode_exchanges).Verification
test_migration_v11_infers_harness_from_source_path_for_non_claude_sessions(builds a pre-v11 DB with a/.grok/conversation, assertsinit_dbdoesn't mislabel itclaude) — failed withassert 'claude' == 'grok'before the fix.test_resolve_aliases_handles_non_ascii_renamed_path(renames acafé.pyfile in a real git repo) — failed with the octal-escaped path string instead ofcafé.pybefore the fix.make check(ruff + pyright + full pytest suite) passes clean: 663 passed, 0 lint issues, 0 typecheck errors.Checklist from the issue:
file_renamesgit calls use-c core.quotepath=false