Skip OpenLogReplicator cached events when Debezium has empty offsets #227
Description
Environment
Debezium 3.1.0-SNAPSHOT
OpenLogReplicator 1.8.3
Oracle 19.3 Standalone
Steps
- Start OpenLogReplicator and Debezium, capturing one table.
- Stop Debezium and leave it down for an extended period, in my case, 48 hours.
- Clear Debezium's offsets, but leave OpenLogReplicator's state without restarting.
- Restart Debezium, but it fails to begin streaming.
In Debezium logs it reports:
2025-03-24 08:14:41,342 INFO Oracle|server1|streaming Starting streaming [io.debezium.pipeline.ChangeEventSourceCoordinator]
2025-03-24 08:14:41,343 INFO Oracle|server1|streaming Streaming will start at SCN 14405744. [io.debezium.connector.oracle.olr.client.OlrNetworkClient]
2025-03-24 08:14:41,459 INFO Oracle|server1|streaming OpenLogReplicator has already started, continue from SCN 14405744
This indicates that Debezium sent the INFO packet and received the REPLICATE packet. But since we've cleared the offsets, Debezium responds with a CONTINUE packet containing only the scn.
On the OpenLogReplicator side, the log shows:
2025-03-24 12:14:41 INFO 00000 client requested scn: 13978392
2025-03-24 12:14:41 INFO 00000 streaming to client
I am assuming OpenLogReplicator is streaming from the last offset from where Debezium stopped in step 2. While OpenLogReplicator does send data since that point to Debezium, this can take some time depending on how long Debezium was down for and how much activity happened on the database.
If I recall, START is not a valid response to REPLICATE.
So would it be possible to avoid replaying all those events to Debezium if its offsets were cleared and instead would allow CONTINUE with scn to act like START and only if a cscn and cidx are provided that OpenLogReplicator continue from that offset position?
If START is the way to handle that use case, that would work as long as REPLICATE supports a START command as a response to indicate this distinction between continuing from position X or starting from position X.