Skip to content

⚡ Bolt: Optimize JSON history scanning with os.scandir - #97

Open
Jandir wants to merge 1 commit into
mainfrom
bolt-optimize-history-scandir-15574138100101399888
Open

⚡ Bolt: Optimize JSON history scanning with os.scandir#97
Jandir wants to merge 1 commit into
mainfrom
bolt-optimize-history-scandir-15574138100101399888

Conversation

@Jandir

@Jandir Jandir commented Aug 8, 2026

Copy link
Copy Markdown
Owner

💡 What:
Replaced Path.glob("*.json") in _scan_directory_for_history with a single os.scandir pass. The files are now dynamically grouped into master_jsons and info_jsons using string matching on the cached DirEntry.name, eliminating the need to iterate through the entire directory's JSON results twice.

🎯 Why:
Previously, the code generated a full list of Path objects via glob and iterated over it twice, repeatedly doing .name string checks for every file in both loops. As the historical JSON backup folder grows (potentially hundreds or thousands of JSON files), this repetitive traversal and Path object instantiation degrades load times unnecessarily.

📊 Impact:

  • Reduces file list traversals from 2 passes to 1.
  • Avoids instantiating numerous Path objects unnecessarily.
  • Reduces repetitive .name string extractions and string condition checking overhead.

🔬 Measurement:
Run the test suite using export PYTHONPATH=$PWD && source .venv/bin/activate && pytest tests/ --ignore=tests/test_escriba.py and observe test_history.py continuing to pass 100%, indicating all historical aggregation operates correctly with the underlying scanning optimization.


PR created automatically by Jules for task 15574138100101399888 started by @Jandir

- Replaced `Path.glob` and double list iteration in `_scan_directory_for_history` with a single `os.scandir` loop.
- Group files into `master_jsons` and `info_jsons` in one pass without extra syscalls or regex matching.
- Documented performance learnings in `.jules/bolt.md`.

Co-authored-by: Jandir <3695656+Jandir@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant