⚡ Bolt: Optimize JSON history scanning with os.scandir - #97
Conversation
- Replaced `Path.glob` and double list iteration in `_scan_directory_for_history` with a single `os.scandir` loop. - Group files into `master_jsons` and `info_jsons` in one pass without extra syscalls or regex matching. - Documented performance learnings in `.jules/bolt.md`. Co-authored-by: Jandir <3695656+Jandir@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
💡 What:
Replaced
Path.glob("*.json")in_scan_directory_for_historywith a singleos.scandirpass. The files are now dynamically grouped intomaster_jsonsandinfo_jsonsusing string matching on the cachedDirEntry.name, eliminating the need to iterate through the entire directory's JSON results twice.🎯 Why:
Previously, the code generated a full list of
Pathobjects viagloband iterated over it twice, repeatedly doing.namestring checks for every file in both loops. As the historical JSON backup folder grows (potentially hundreds or thousands of JSON files), this repetitive traversal and Path object instantiation degrades load times unnecessarily.📊 Impact:
Pathobjects unnecessarily..namestring extractions and string condition checking overhead.🔬 Measurement:
Run the test suite using
export PYTHONPATH=$PWD && source .venv/bin/activate && pytest tests/ --ignore=tests/test_escriba.pyand observetest_history.pycontinuing to pass 100%, indicating all historical aggregation operates correctly with the underlying scanning optimization.PR created automatically by Jules for task 15574138100101399888 started by @Jandir