Skip to content

Scale tenant reconciliation - #699

Draft
behinddwalls wants to merge 1 commit into
mainfrom
preetam/mq-tenant-reconciliation
Draft

Scale tenant reconciliation#699
behinddwalls wants to merge 1 commit into
mainfrom
preetam/mq-tenant-reconciliation

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Reconciling every configured tenant on every subscription tick multiplies idle MySQL work when one process serves many tenants.

What?

  • Schedule tenant discovery, heartbeats, lease maintenance, purges, and orphan sweeps independently by tenant.
  • Share recent tenant activity across pipeline topics while preserving tenant-scoped Vitess queries.
  • Add deterministic scheduler coverage and retain the configured gateway integration tenant correction.

Test Plan

./tool/bazel test //platform/extension/messagequeue/... //service/messagequeue:go_default_test
./tool/bazel test //test/integration/extension/messagequeue/mysql:go_default_test //test/integration/extension/messagequeue/mysql/vitess:go_default_test --strategy=TestRunner=local
./tool/bazel test //test/integration/submitqueue/gateway:go_default_test --strategy=TestRunner=local
./tool/bazel test //test/e2e/submitqueue:go_default_test --strategy=TestRunner=local
make lint && make check-tidy && make check-gazelle

Co-authored-by: Cursor cursoragent@cursor.com

@behinddwalls behinddwalls changed the title feat(messagequeue): scale tenant reconciliation Scale tenant reconciliation for sharded queues Sep 10, 2026
@behinddwalls
behinddwalls force-pushed the preetam/mq-tenant-reconciliation branch from ab68ac3 to 8139485 Compare September 10, 2026 21:14
@behinddwalls
behinddwalls changed the base branch from preetam/mq-tenant-vitess to main September 10, 2026 21:14
@behinddwalls
behinddwalls force-pushed the preetam/mq-tenant-reconciliation branch from 8139485 to 2b99e30 Compare September 10, 2026 21:59
@behinddwalls behinddwalls changed the title Scale tenant reconciliation for sharded queues Schedule tenant reconciliation by due time Sep 10, 2026
## Summary

### Why?

Reconciling every configured tenant on every subscription tick multiplies idle MySQL work when one process serves many tenants.

### What?

- Schedule tenant discovery, heartbeats, lease maintenance, purges, and orphan sweeps independently by tenant.
- Share recent tenant activity across pipeline topics while preserving tenant-scoped Vitess queries.
- Add deterministic scheduler coverage and retain the configured gateway integration tenant correction.

## Test Plan

✅ `./tool/bazel test //platform/extension/messagequeue/... //service/messagequeue:go_default_test`
✅ `./tool/bazel test //test/integration/extension/messagequeue/mysql:go_default_test //test/integration/extension/messagequeue/mysql/vitess:go_default_test --strategy=TestRunner=local`
✅ `./tool/bazel test //test/integration/submitqueue/gateway:go_default_test --strategy=TestRunner=local`
✅ `./tool/bazel test //test/e2e/submitqueue:go_default_test --strategy=TestRunner=local`
✅ `make lint && make check-tidy && make check-gazelle`

Co-authored-by: Cursor <cursoragent@cursor.com>
@behinddwalls
behinddwalls force-pushed the preetam/mq-tenant-reconciliation branch from 2b99e30 to a4e627f Compare September 10, 2026 23:56
@behinddwalls behinddwalls changed the title Schedule tenant reconciliation by due time Scale tenant reconciliation Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant