draft: Universal stream join#59
Draft
wzru wants to merge 96 commits into
Draft
Conversation
2. abstract the join method and creat a STL named AbstractJoinMethod 3. change the celljoin by extending the AbstractJoinMethod 4. try handshake join (not work fine yet)
2. Add the MicroDataSet.h into Utils
2. Add the ThreadPerf into Utils 3. Add the MultiThreadHashTable into Common 4. Add the NPJ into JoinAlgo
* 1. add the branch for lazy window slider * 1. add the branch for lazy window slider * issue #29 * issue #29: 1. Add a JoinAlgoTable in JoinAlgo, to access all supported algos 2. Rewrite the existing classes of eager join (just briefly done) 3. First version of lazy window join slider and processor (No sharing) 4. Upgrade the NPJ accordingly * issue #29: 1. Add a JoinAlgoTable in JoinAlgo, to access all supported algos 2. Rewrite the existing classes of eager join (just briefly done) 3. First version of lazy window join slider and processor (No sharing) 4. Upgrade the NPJ accordingly 5. Remove the build folder * issue #29: 1. Add a JoinAlgoTable in JoinAlgo, to access all supported algos 2. Rewrite the existing classes of eager join (just briefly done) 3. First version of lazy window join slider and processor (No sharing) 4. Upgrade the NPJ accordingly 5. Remove the build folder * remove the xxx build * issue #29: 1. Add a JoinAlgoTable in JoinAlgo, to access all supported algos 2. Rewrite the existing classes of eager join (just briefly done) 3. First version of lazy window join slider and processor (No sharing) 4. Upgrade the NPJ accordingly * delete the commit scripts Co-authored-by: tony <292224750@.com>
- add Test0 dataset (100 tuples) to debug
* SplitJoin(with bug) Co-authored-by: whyy7777 <yinyhuang7@gmail.com>
* take spliting window and making new joiner to engine level * modify make joiner and free joiner * clang-format * amend new JoinCore policy * no elimination of joiner * index of begin joiner
* take spliting window and making new joiner to engine level * rebase universal_stream_join * debugging * fix: SplitJoin segfault * no exception * only some concurrency bug * completion of SplitJoin * fix: SystemTest.Verify * fix: ci --------- Co-authored-by: wzru <harryshaunwang@outlook.com>
wzru
force-pushed
the
universal_stream_join
branch
from
1f7274b to
ba5d312
Compare
* refactor: context & Param * feat: add FATAL
* SplitJoinOrigin and correct EmitAllWindow * origin logger * format code * format src * format include * modify * no DS_Store
.. Many issues in the code structure and main logic. 1) it needs to be changed to pull-based rather than push-based. 2) the structure of engine-algo-joincore is not cleared enough. Need a major refactor. 3) the time measurement is not precise, I have moved the related part to ctx, to be updated later to force more fair evaluation.
.. Many issues in the code structure and main logic. 1) it needs to be changed to pull-based rather than push-based. 2) the structure of engine-algo-joincore is not cleared enough. Need a major refactor. 3) the time measurement is not precise, I have moved the related part to ctx, to be updated later to force more fair evaluation. 4) some code clean.
discovered some bugs in the existing code: the algorithm is not producing correct results when setting different slide_size.
discovered some bugs in the existing code: the algorithm is not producing correct results when setting different slide_size.
- Revise RelationCouple implementation. - Add code skeleton. TO-Do List: - Remove Template, use inheritance instead as we don't need to worry virtual dispatch for different window sliders. - Need to use template for join algorithms -- however, we may simply rely on one algorithm at the current stage. - Implement algorithm selector in the future.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.