feat(datastream-mongodb-to-firestore): implement high-throughput shadowless migration - #4128

Open

michaeltle-goog wants to merge 16 commits into

GoogleCloudPlatform:mainfrom

michaeltle-goog:datastream-shadowless-firestore

Commits on Aug 10, 2026

feat(datastream-mongodb-to-firestore): implement high-throughput shadowless migration
michaeltle-goog
committed
a0c7bec

Commits on Aug 11, 2026

fix(datastream-mongodb-to-firestore): fix timestamp sort key ordering and address PR 4128 review comments
michaeltle-goog
committed
83ce0eb
style: apply spotless formatting
michaeltle-goog
committed
8324c3f
fix(utils): add bool_ prefix to documentIdToString and add distinctness tests for 123 string vs 123L
michaeltle-goog
committed
a4c6f78
refactor: remove redundant read_method and _metadata_dlq_reconsumed checks in MongoDbChangeEventContext
michaeltle-goog
committed
6983c6f
fix(review): address performance, thread-safety, defensive casting and test coverage review findings
michaeltle-goog
committed
0137ad1
perf(datastream-mongodb-to-firestore): eliminate CombineHotKeys window bottleneck and stream directly to StatefulDeduplication
michaeltle-goog
committed
440496b
perf(datastream-mongodb-to-firestore): implement deterministic binary coders to eliminate GC overhead in StatefulDeduplication
michaeltle-goog
committed
cd1c08b
perf(datastream-mongodb-to-firestore): decouple processing timestamps before keyed shuffle to eliminate watermark stalls
michaeltle-goog
committed
72bdcd7
Revert "perf(datastream-mongodb-to-firestore): decouple processing timestamps before keyed shuffle to eliminate watermark stalls"
michaeltle-goog
committed
f4d5a8f
refactor(datastream-mongodb-to-firestore): remove orderingStrategy param and make stateful deduplication default shadowless implementation
michaeltle-goog
committed
ec9a813

Commits on Aug 14, 2026

Commits on Aug 18, 2026

fix(datastream-mongodb-to-firestore): preserve customer document 'data' field and isolate shadow metadata stripping
michaeltle-goog
committed
add53a5

Commits on Aug 20, 2026