Skip to content

Add in-memory type index for query_by_type_prefix#735

Open
iamnbutler wants to merge 1 commit intomainfrom
tasks/gh-iamnbutler-tasks-issue-507--c8d0e0ab
Open

Add in-memory type index for query_by_type_prefix#735
iamnbutler wants to merge 1 commit intomainfrom
tasks/gh-iamnbutler-tasks-issue-507--c8d0e0ab

Conversation

@iamnbutler
Copy link
Copy Markdown
Owner

Summary

  • Adds a lazily-initialized in-memory secondary index (HashMap<event_type, Vec<Event>>) to EventStore that eliminates the O(n*m) full-scan in query_by_type_prefix()
  • The index is built from disk on first query, then kept in sync on every append() call — subsequent queries only iterate over matching type keys
  • Compaction and orphan cleanup invalidate the index so it rebuilds on next query
  • Adds two new tests: index sync after append, and index rebuild after compaction

Closes #507

Test plan

  • All 55 existing + new unit tests pass (cargo test --package events)
  • Full project builds clean (cargo check)
  • Verify orchestrator and API endpoints still work correctly with the indexed path

🤖 Generated with Claude Code

query_by_type_prefix previously scanned every event file for every task
on each call — O(n*m) where n=tasks and m=events per task. This adds a
lazily-initialized in-memory secondary index (HashMap<event_type, Vec<Event>>)
that is built once from disk on first query and kept in sync on every
append(). Compaction and orphan cleanup invalidate the index so it gets
rebuilt on next query.

Closes #507

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

query_by_type_prefix scans all task event files — O(n*m) on every call

1 participant