Skip to content

[Fix] Accelerate and repair Brain backfills - #1365

Merged
mrubens merged 3 commits into
developfrom
codex/accelerate-brain-backfill
Aug 15, 2026
Merged

[Fix] Accelerate and repair Brain backfills#1365
mrubens merged 3 commits into
developfrom
codex/accelerate-brain-backfill

Conversation

@mrubens

@mrubens mrubens commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

What changed

  • continue bounded historical collector passes while they make progress, while preserving the 15-minute steady-state poll
  • stop fast continuation on Brain backpressure and persist collector cursors after each landed batch
  • make pull-request fact ingestion use a durable timestamp-plus-ID keyset
  • reset collector checkpoints and requeue completed task memories when gbrain OAuth clients are provisioned after a corpus rebuild
  • add a one-time state repair migration for deployments already affected
  • replay task memories newest-first

This includes the newest-first task replay from #1356.

Why

Roomote stored collector watermarks in its application database while gbrain corpus data lived separately. Rebuilding the Brain could therefore leave Roomote believing task and integration backfills were complete, producing an almost-empty corpus. Historical integrations also waited up to 15 minutes between bounded batches even when more work was immediately available.

Impact

A fresh or rebuilt Brain repopulates its historical task, Slack, Granola, GitHub issue, and pull-request context promptly. Per-pass limits and typed backpressure remain, normal steady-state polling does not become more aggressive, and recent task history becomes useful first.

Validation

  • focused BullMQ tests: 44 passed
  • focused real-database Brain tests: 14 passed
  • package typechecks for @roomote/db, @roomote/sdk, and @roomote/bullmq
  • repository pre-push checks, targeted lint, and formatting

@roomote-community

roomote-community Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

No new code issues found. See task

  • Fast continuation also runs all incremental integration polls every second (apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts:315).
  • PR-fact Brain backpressure is swallowed and can be retried every second during another backfill (apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts:612).
  • PR-fact keyset cursor can permanently skip rows written at its current timestamp (apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts:634).

Reviewed f12751f

Comment thread apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts Outdated
Comment thread apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts Outdated
Comment thread apps/bullmq/src/scheduled-jobs/brain-outbox-drain.ts
@mrubens
mrubens marked this pull request as ready for review August 15, 2026 15:59
@mrubens
mrubens merged commit bedf186 into develop Aug 15, 2026
17 checks passed
@mrubens
mrubens deleted the codex/accelerate-brain-backfill branch August 15, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant