Skip to content

[#4643] AggregateBasedJpaEventStorageEngine#stream loops until StreamingCondition matching events are retrieved#4684

Merged
smcvb merged 3 commits into
mainfrom
bug/4625/second-approach
Jun 23, 2026
Merged

[#4643] AggregateBasedJpaEventStorageEngine#stream loops until StreamingCondition matching events are retrieved#4684
smcvb merged 3 commits into
mainfrom
bug/4625/second-approach

Conversation

@smcvb

@smcvb smcvb commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This pull request adjusts the AggregateBasedJpaEventStorageEngine#streaming method, to keep querying batches of events until some match the given StreamingCondition.
If we don't do this, there's a chance that the batch is empty, stalling the ContinuousMessageStream until it's nudged (e.g. when a new event is appended).
Thus, the old approach essentially blocks a consumer like the PooledStreamingEventProcessor, potentially indefinitely.

This PR is a second try to resolve issue #4643, with the original PR being #4681.
This PR differs from #4681 as it's an AggregateBasedJpaEventStorageEngine only solution.
This is by design, as Axon Server and our PostgreSQL EventStorageEngine implementations ensure the same: they loop through the event store until a StreamingCondition match is found.
As all EventStorageEngines support this, and should support this, a test has been added to the StorageEngineBackedEventStoreTestSuite to validate the behavior that an EventStorageEngine#stream operation will progress until a first StreamingCondition match has been found.

By doing the above, this PR works towards resolving #4643.
A port to axon-5.1.x is still required, though.

smcvb added 2 commits June 23, 2026 10:43
Adjust the AggregateBasedJpaEventStorageEngine#streaming to keep querying events until some match the given StreamingCondition. If we don't do this, there's a chance that the batch is empty, stalling the ContinuesMessageStream until a new event is appended. Thus, essentially blocking a consumer like the PooledStreamingEventProcessor, potentially indefinitely.

#4625
@smcvb smcvb added this to the Release 5.2.0 milestone Jun 23, 2026
@smcvb smcvb self-assigned this Jun 23, 2026
@smcvb smcvb requested a review from a team as a code owner June 23, 2026 08:52
@smcvb smcvb requested review from MateuszNaKodach, jangalinski and laura-devriendt-lemon and removed request for a team June 23, 2026 08:52
@smcvb smcvb added Type: Bug Use to signal issues that describe a bug within the system. Priority 1: Must Highest priority. A release cannot be made if this issue isn’t resolved. labels Jun 23, 2026
@smcvb smcvb changed the title [#4625] AggregateBasedJpaEventStorageEngine#stream loops until StreamingCondition matching events are retrieved [#4643] AggregateBasedJpaEventStorageEngine#stream loops until StreamingCondition matching events are retrieved Jun 23, 2026
@vcanuel

vcanuel commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Tested the same way as #4681 — confirms it works !.
Setup: AggregateBasedJpaEventStorageEngine + PostgreSQL, PooledStreamingEventProcessor (TagAndTypeFilteredEventCriteria, 4 segments).

  • 5.1.1: processor caught up at token 203483, injected ~200K events of an unhandled type + a real NodeCreatedEvent after them → token frozen at 203483, entity never projected.
  • This PR (built locally, verified by jar sha): on first start it streamed through the ~200K non-matching events, token reached the head, entity projected immediately.
    The engine-only approach works perfectly and feels like the cleaner fix. Happy to validate the axon-5.1.x backport too. Thanks!

@smcvb smcvb requested a review from hjohn June 23, 2026 10:05

@hatzlj hatzlj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one suggestion to improve the readability, otherwise looks good to me

@hatzlj

hatzlj commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

would be worth to run the StorageEngineBackedEventStoreTestSuite tests living in Axoniq Framework once to check

@smcvb

smcvb commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

would be worth to run the StorageEngineBackedEventStoreTestSuite tests living in Axoniq Framework once to check

Will do!

Simplify loop by using a do-while i.o. while(true)

#4643

@hjohn hjohn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that will do the trick :) LGTM

@smcvb

smcvb commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

would be worth to run the StorageEngineBackedEventStoreTestSuite tests living in Axoniq Framework once to check

Validated the new test to work for the AxonServerEventStorageEngine and PostgresqlEventStorageEngine, so I think we're good!

@smcvb smcvb merged commit 9d438e1 into main Jun 23, 2026
7 checks passed
@smcvb smcvb deleted the bug/4625/second-approach branch June 23, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority 1: Must Highest priority. A release cannot be made if this issue isn’t resolved. Type: Bug Use to signal issues that describe a bug within the system.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants