Skip to content

fix(postgres): batch pg push comparison reads (#331)#754

Merged
wesm merged 2 commits into
kenn-io:mainfrom
rodboev:pr/pg-push-batch-comparison-reads
Jun 19, 2026
Merged

fix(postgres): batch pg push comparison reads (#331)#754
wesm merged 2 commits into
kenn-io:mainfrom
rodboev:pr/pg-push-batch-comparison-reads

Conversation

@rodboev

@rodboev rodboev commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Keep the candidate-window and boundary-session behavior in internal/postgres/push.go unchanged for this PR, and batch the PostgreSQL-side comparison reads used to decide whether a candidate session can be skipped.
  • Implement new batched loaders in internal/postgres/push_fingerprint.go for message aggregates, message content hashes, role/time fingerprints, message flags, message system ordinals, token fingerprints, tool-call aggregates, tool-call fingerprints, and usage fingerprints, with chunking inside the helper when session counts exceed ANY($1) practicality.
  • Use the preloaded message and tool-call aggregates on the hot no-op path, and retry any comparison-preload SQL failure in a fresh transaction without the batched preload instead of continuing inside an already-aborted transaction.
  • Add targeted regression tests in internal/postgres/push_test.go and internal/postgres/push_fingerprint_test.go to cover the new batch-driven skip decision path and helper behavior with empty inputs.

Scope

  • Files changed are internal/postgres/push.go, internal/postgres/push_fingerprint.go, internal/postgres/push_test.go, and internal/postgres/push_fingerprint_test.go.
  • No boundary/windowing semantics, no schema changes, and no changes to PR feat(sync): local-first multi-machine artifact sync #731 or broader sync-work areas.

Notes

  • A focused PG comparison query-count assertion was not added because the existing harness does not expose a stable helper-call/query metric for this exact path without adding brittle test-only instrumentation.
  • The review-driven follow-up keeps the existing non-batched fingerprint fallback, but now that fallback only runs from a clean transaction after preload failure instead of on the poisoned transaction that raised the preload error.

Fixes #331

@roborev-ci

roborev-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown

roborev: Combined Review (3fb4275)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (codex/default, done, 4m59s), codex_security (codex/security, done, 1m49s) | Total: 6m48s

@wesm

wesm commented Jun 19, 2026

Copy link
Copy Markdown
Member

awesome, thanks for working on this!

@wesm wesm merged commit 8557423 into kenn-io:main Jun 19, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

pg push: slow for large databases due to per-session PG round-trips

2 participants