Improve performance

docs/adr/0006-pool-based-index-layout.md

@@ -0,0 +1,64 @@
+Status: Accepted
+
+## Context
+
+Profiling on a 624 K-entry index showed two root causes of slow search:
+
+1. **Cache thrashing from heap-scattered strings.** Each `FileEntry` owns three
+   heap-allocated `std::wstring` objects (`name`, `nameLower`, `path`). Searching
+   624 K entries requires chasing ~1.9 M scattered heap pointers, producing near-zero
+   L2/L3 cache reuse and dominating search latency.
+
+2. **No early exit across parallel search threads.** Threads continue scanning after
+   `maxResults` hits are found collectively, wasting CPU time on every keystroke.
+
+A third issue — `GetEntries()` returning a raw pointer with no lock while ChangeWatcher
+threads can concurrently call `ApplyAdd()` and reallocate the vector — is an active data
+race. See also ADR-0003 (binary format) which this supersedes for the in-memory layout.
+
+## Options
+
+| Option | Fits when | Cost now | Extension path | Trade-off |
+|--------|-----------|----------|----------------|-----------|
+| a. Keep `vector<FileEntry>` with per-entry `wstring` | Index is small (<100 K entries) | None | None | Cache-hostile at 600 K+ entries; scales poorly |
+| b. Flat pool: single `vector<wchar_t>` with offsets | Sequential scan is the primary access pattern | Medium refactor | Add sorted/trigram index later | O(n) scan but L3-resident; correct for 600 K scale |
+| c. Sorted array + binary search | Prefix-match queries dominate | High refactor | Replace with trie later | Only fast for prefix queries; substring still O(n) |
+| d. Trigram inverted index | Arbitrary substring at >10 M entries | High complexity | Standard IR approach | Overkill at 600 K; large memory overhead |
+
+## Decision
+
+Adopt **option b** — separate flat string pools:
+
+- **`nameLower` pool** (`vector<wchar_t>`): all lowercased filenames concatenated,
+  ~18 MB for 624 K entries. Fits in L3 cache. This is the default search target.
+- **`path` pool** (`vector<wchar_t>`): all full paths original-case concatenated,
+  ~75 MB. Used only when `matchPath = true` (opt-in).
+- **Metadata array** (`vector<EntryMeta>`): fixed-size structs (32 bytes each) holding
+  offsets and lengths into both pools, plus `size`, `lastModified`, `attributes`.
+
+`SearchResult::entry` changes from `const FileEntry*` to `uint32_t entryIndex`. The UI
+looks up display fields via a thin `IndexPool::GetEntry(index)` call.
+
+Concurrency is managed by `std::shared_mutex` (backed by Windows SRWLOCK — no heavier
+than a CRITICAL_SECTION). Search acquires a shared read lock for the duration of the
+scan (~5-10 ms). ChangeWatcher acquires an exclusive write lock to append entries.
+
+A `pathLower` pool is deferred: path search (`matchPath = true`) is opt-in and not the
+latency-sensitive default. It can be added as a fourth pool if path-search performance
+becomes a concern.
+
+On-disk format is bumped to **version 2** (see ADR-0003). Version mismatch triggers a
+one-time silent re-index on first launch after the upgrade.
+
+## Consequences
+
+- Name-only search scans ~18 MB of contiguous memory instead of chasing ~1.9 M heap
+  pointers — expected 3-5x throughput improvement.
+- Data race on `GetEntries()` is eliminated by `shared_mutex`.
+- `SearchResult` is a breaking API change: callers must use `entryIndex` + pool lookup
+  rather than a `FileEntry*`.
+- `nameLower` is not persisted on disk; it is recomputed from `name` at load time
+  (one-time cost at startup, O(n) `towlower` pass).
+- `pathLower` is not stored; path search uses a per-thread lowercase buffer as before.
+- On-disk format version 2 is incompatible with version 1; existing `.idx` files are
+  discarded and rebuilt on first launch.

src/core/CMakeLists.txt

@@ -7,7 +7,9 @@ add_library(winindex_core STATIC
     indexer/Indexer.cpp
     search/SearchEngine.cpp
     search/SimdSearch.cpp
+    search/SimdSearchAvx2.cpp
     search/TokenMatcher.cpp
+    storage/IndexPool.cpp
     storage/IndexStore.cpp
     storage/IndexSerializer.cpp
     settings/Settings.cpp
@@ -25,6 +27,14 @@ target_link_libraries(winindex_core
     PRIVATE ntdll
 )
 
+# SimdSearchAvx2.cpp must be compiled with AVX2 enabled so intrinsics are available.
+# Runtime detection in SimdFindSubstring() guards actual execution on non-AVX2 CPUs.
+if(MSVC)
+    set_source_files_properties(search/SimdSearchAvx2.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX2")
+else()
+    set_source_files_properties(search/SimdSearchAvx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2")
+endif()
+
 # Expose interfaces for mocking in tests
 target_sources(winindex_core PUBLIC
     indexer/IFileSystemScanner.h

src/core/indexer/Indexer.cpp

@@ -8,6 +8,7 @@
 
 #include <algorithm>
 #include <iterator>
+#include <thread>
 #include <utility>
 
 namespace winindex {
@@ -102,10 +103,16 @@ void Indexer::IndexPaths(std::vector<std::wstring> paths) {
 
 void Indexer::IndexPathsThread(const std::vector<std::wstring>& paths) {
     EmitStatus(IndexerState::Scanning, L"Indexing new paths...");
+    std::vector<std::wstring> excludedPaths = m_settings->GetExcludedPaths();
     for (const auto& root : paths) {
         if (m_cancel.load(std::memory_order_relaxed))
             break;
-        ScanDrive(root);
+        std::vector<FileEntry> localEntries;
+        ScanDriveInto(root, excludedPaths, localEntries);
+        for (const auto& fe : localEntries) {
+            m_indexStore->ApplyAdd(fe);
+            ++m_filesIndexed;
+        }
     }
     if (!m_cancel.load(std::memory_order_relaxed)) {
         m_indexStore->Save();
@@ -206,7 +213,6 @@ void Indexer::EmitStatusDone(uint64_t filesIndexed, std::vector<std::wstring> lo
 
 void Indexer::IndexingThread() {
     EmitStatus(IndexerState::Scanning, L"Starting index build...");
-    m_indexStore->BeginWrite();
 
     auto selectedDrives = m_settings->GetSelectedDrives();
     if (selectedDrives.empty()) {
@@ -216,45 +222,75 @@ void Indexer::IndexingThread() {
         std::transform(fixed.begin(), fixed.end(), std::back_inserter(selectedDrives),
                        [](const DriveInfo& d) { return d.root; });
     }
-    for (const auto& root : selectedDrives) {
-        if (m_cancel.load(std::memory_order_relaxed))
-            break;
-        ScanDrive(root);
+
+    // Snapshot settings before spawning threads; Settings isn't thread-safe.
+    std::vector<std::wstring> excludedPaths = m_settings->GetExcludedPaths();
+
+    // Scan each drive on its own thread into a per-drive local buffer.
+    const size_t n = selectedDrives.size();
+    std::vector<std::vector<FileEntry>> perDrive(n);
+    std::vector<std::thread> scanThreads;
+    scanThreads.reserve(n);
+
+    for (size_t i = 0; i < n; ++i) {
+        scanThreads.emplace_back([this, &selectedDrives, &excludedPaths, &perDrive, i]() {
+            ScanDriveInto(selectedDrives[i], excludedPaths, perDrive[i]);
+        });
     }
+    for (auto& t : scanThreads) t.join();
 
-    if (!m_cancel.load(std::memory_order_relaxed)) {
-        m_indexStore->EndWrite();
-        m_indexStore->Save();
-        EmitStatusDone(m_filesIndexed, m_settings->GetSelectedDrives());
+    if (m_cancel.load(std::memory_order_relaxed)) {
+        SetEvent(m_completionEvent);
+        return;
     }
 
-    if (!m_cancel.load(std::memory_order_relaxed))
-        StartLiveMonitoring();
+    m_indexStore->BeginWrite();
+    for (const auto& entries : perDrive) {
+        for (const auto& fe : entries) {
+            m_indexStore->AddEntry(fe);
+            ++m_filesIndexed;
+        }
+    }
+    m_indexStore->EndWrite();
+    m_indexStore->Save();
+    EmitStatusDone(m_filesIndexed, m_settings->GetSelectedDrives());
+    StartLiveMonitoring();
     SetEvent(m_completionEvent);
 }
 
+bool Indexer::BuildAndApplyAdd(const std::wstring& path) {
+    WIN32_FILE_ATTRIBUTE_DATA fad{};
+    if (!GetFileAttributesExW(path.c_str(), GetFileExInfoStandard, &fad))
+        return false;
+    if (fad.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
+        return false;
+    FileEntry fe;
+    size_t slash = path.rfind(L'\\');
+    fe.name = (slash != std::wstring::npos) ? path.substr(slash + 1) : path;
+    fe.nameLower = fe.name;
+    std::transform(fe.nameLower.begin(), fe.nameLower.end(), fe.nameLower.begin(), ::towlower);
+    fe.path = path;
+    fe.size = (static_cast<uint64_t>(fad.nFileSizeHigh) << 32) | fad.nFileSizeLow;
+    fe.lastModified = (static_cast<uint64_t>(fad.ftLastWriteTime.dwHighDateTime) << 32) |
+                      fad.ftLastWriteTime.dwLowDateTime;
+    fe.attributes = fad.dwFileAttributes;
+    m_indexStore->ApplyAdd(fe);
+    return true;
+}
+
 void Indexer::ApplyChange(const FileChangeEvent& evt) {
-    if (evt.type == FileChangeType::Added || evt.type == FileChangeType::Modified) {
-        WIN32_FILE_ATTRIBUTE_DATA fad{};
-        if (!GetFileAttributesExW(evt.path.c_str(), GetFileExInfoStandard, &fad))
-            return;
-        if (fad.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
-            return;
-        if (evt.type == FileChangeType::Modified)
-            m_indexStore->ApplyRemove(evt.path);
-        FileEntry fe;
-        size_t slash = evt.path.rfind(L'\\');
-        fe.name = (slash != std::wstring::npos) ? evt.path.substr(slash + 1) : evt.path;
-        fe.nameLower = fe.name;
-        std::transform(fe.nameLower.begin(), fe.nameLower.end(), fe.nameLower.begin(), ::towlower);
-        fe.path = evt.path;
-        fe.size = (static_cast<uint64_t>(fad.nFileSizeHigh) << 32) | fad.nFileSizeLow;
-        fe.lastModified = (static_cast<uint64_t>(fad.ftLastWriteTime.dwHighDateTime) << 32) |
-                          fad.ftLastWriteTime.dwLowDateTime;
-        fe.attributes = fad.dwFileAttributes;
-        m_indexStore->ApplyAdd(fe);
+    if (evt.type == FileChangeType::Added) {
+        BuildAndApplyAdd(evt.path);
+    } else if (evt.type == FileChangeType::Modified) {
+        m_indexStore->ApplyRemove(evt.path);
+        BuildAndApplyAdd(evt.path);
     } else if (evt.type == FileChangeType::Removed) {
         m_indexStore->ApplyRemove(evt.path);
+    } else if (evt.type == FileChangeType::Renamed) {
+        if (!evt.oldPath.empty())
+            m_indexStore->ApplyRename(evt.oldPath, evt.path);
+        else
+            BuildAndApplyAdd(evt.path);
     }
 }
 
@@ -284,34 +320,31 @@ void Indexer::StartWatchersForRoots(const std::vector<std::wstring>& roots) {
     }
 }
 
-void Indexer::ScanDrive(const std::wstring& root) {
+void Indexer::ScanDriveInto(const std::wstring& root,
+                            const std::vector<std::wstring>& excludedPaths,
+                            std::vector<FileEntry>& out) {
     DriveFilesystem fs = GetFilesystem(root);
 
-    // Select scanner: MFT for NTFS if admin available, FindFile otherwise
+    // Select scanner: MFT for NTFS if admin available, FindFile otherwise.
     IFileSystemScanner* scanner = m_findScanner.get();
 
     if (fs == DriveFilesystem::NTFS && m_mftScanner->IsMftAvailable(root)) {
         scanner = m_mftScanner.get();
-        EmitStatus(IndexerState::Scanning, L"Indexing " + root + L" (MFT mode)...", m_filesIndexed);
+        EmitStatus(IndexerState::Scanning, L"Indexing " + root + L" (MFT mode)...");
     } else if (fs == DriveFilesystem::NTFS) {
-        EmitStatus(
-            IndexerState::Scanning,
-            L"Indexing " + root + L" (standard mode - run as administrator for faster MFT scan)...",
-            m_filesIndexed);
+        EmitStatus(IndexerState::Scanning,
+                   L"Indexing " + root +
+                       L" (standard mode - run as administrator for faster MFT scan)...");
     } else {
-        EmitStatus(IndexerState::Scanning, L"Indexing " + root + L" (FAT32)...", m_filesIndexed);
+        EmitStatus(IndexerState::Scanning, L"Indexing " + root + L" (FAT32)...");
     }
 
     ScanOptions opts;
     opts.rootPaths = {root};
-    opts.excludedPaths = m_settings->GetExcludedPaths();
+    opts.excludedPaths = excludedPaths;
 
     scanner->Scan(
-        opts,
-        [this](const FileEntry& fe) {
-            m_indexStore->AddEntry(fe);
-            ++m_filesIndexed;
-        },
+        opts, [&out](const FileEntry& fe) { out.push_back(fe); },
         [this, &root](uint64_t count, const std::wstring& /*dir*/) {
             EmitStatus(IndexerState::Scanning,
                        L"Indexing " + root + L"... " + std::to_wstring(count) + L" files", count);

src/core/indexer/Indexer.h

@@ -64,7 +64,9 @@ class Indexer {
     void IndexingThread();
     void IndexPathsThread(const std::vector<std::wstring>& paths);
     void RemovePathsThread(const std::vector<std::wstring>& paths);
-    void ScanDrive(const std::wstring& root);
+    void ScanDriveInto(const std::wstring& root, const std::vector<std::wstring>& excludedPaths,
+                       std::vector<FileEntry>& out);
+    bool BuildAndApplyAdd(const std::wstring& path);
     void ApplyChange(const FileChangeEvent& evt);
     void StartLiveMonitoring();
     void StartWatchersForRoots(const std::vector<std::wstring>& roots);

src/core/indexer/UsnJournalMonitor.cpp

@@ -76,8 +76,10 @@ uint64_t UsnJournalMonitor::ReplaySince(const std::wstring& root, uint64_t saved
                 evt.type = FileChangeType::Added;
             else if (record->Reason & USN_REASON_FILE_DELETE)
                 evt.type = FileChangeType::Removed;
+            else if (record->Reason & USN_REASON_RENAME_OLD_NAME)
+                evt.type = FileChangeType::Removed;
             else if (record->Reason & USN_REASON_RENAME_NEW_NAME)
-                evt.type = FileChangeType::Renamed;
+                evt.type = FileChangeType::Added;
             else
                 evt.type = FileChangeType::Modified;
 
@@ -143,8 +145,10 @@ void UsnJournalMonitor::StartMonitoring(const std::wstring& root, uint64_t start
                 evt.type = FileChangeType::Added;
             else if (record->Reason & USN_REASON_FILE_DELETE)
                 evt.type = FileChangeType::Removed;
+            else if (record->Reason & USN_REASON_RENAME_OLD_NAME)
+                evt.type = FileChangeType::Removed;
             else if (record->Reason & USN_REASON_RENAME_NEW_NAME)
-                evt.type = FileChangeType::Renamed;
+                evt.type = FileChangeType::Added;
             else
                 evt.type = FileChangeType::Modified;
 

src/core/search/ISearchEngine.h

@@ -1,7 +1,8 @@
 #pragma once
 #include "../indexer/IFileSystemScanner.h"
+#include "../storage/IndexPool.h"
+#include <atomic>
 #include <cstdint>
-#include <string>
 #include <vector>
 
 namespace winindex {
@@ -17,30 +18,34 @@ struct SearchOptions {
 
 /// @brief A single match returned by a search query.
 struct SearchResult {
-    const FileEntry*
-        entry;            ///< Pointer into the live index — valid for the lifetime of IIndexStore.
+    uint32_t entryIndex;  ///< Index into the IndexPool that produced this result.
     uint32_t matchStart;  ///< Character offset within the matched string (for highlight rendering).
-    uint32_t matchLen;    ///< Length of the matched substring in characters.
+    uint32_t
+        matchLen;  ///< Length of the matched substring in characters (0 for regex/token matches).
 };
 
 /// @brief Interface for searching the in-memory file index.
-///
-/// Implementations: ScalarSearchEngine (simple substring), SimdSearchEngine
-/// (AVX2/SSE4.2 accelerated), and RegexSearchEngine (RE2-based).
 class ISearchEngine {
 public:
     virtual ~ISearchEngine() = default;
 
-    /// @brief Searches @p entries for files matching @p query.
+    /// @brief Searches the pool for files matching @p query.
+    ///
+    /// Callers must hold a shared read lock on IndexPool for the duration of
+    /// this call (IndexStore::GetSearchMutex()).
+    ///
     /// @param query       Search string or regular expression.
-    /// @param entries     Pointer to the flat array of FileEntry objects in the index.
+    /// @param meta        Pointer to the flat EntryMeta array.
     /// @param entryCount  Number of entries in the array.
+    /// @param nameLowerPool  Flat pool of lowercased filenames (searched by default).
+    /// @param pathPool       Flat pool of full paths (searched when matchPath is true).
     /// @param options     Match mode flags.
     /// @param maxResults  Maximum number of results to return.
     /// @param cancelToken Set to true externally to abort a long-running search.
-    /// @return Matched entries with highlight offsets, capped at @p maxResults.
-    virtual std::vector<SearchResult> Search(const std::wstring& query, const FileEntry* entries,
-                                             uint64_t entryCount, const SearchOptions& options,
+    /// @return Matched entry indices with highlight offsets, capped at @p maxResults.
+    virtual std::vector<SearchResult> Search(const std::wstring& query, const EntryMeta* meta,
+                                             uint64_t entryCount, const wchar_t* nameLowerPool,
+                                             const wchar_t* pathPool, const SearchOptions& options,
                                              uint32_t maxResults,
                                              const std::atomic<bool>& cancelToken) = 0;
 };