Skip to content

perf(security): memoized token scanner in PiiMasker.detectPiiColumn eliminating dynamic RegExp and split/join #1540

Open

Description

@ZhuchkaTriplesix

Problem

In PiiMasker.detectPiiColumn (lib/core/security/pii_masker.dart), every column of every executed query triggers:

  • Two dynamic regexes in _tokens (r'([a-z0-9])([A-Z])' and r'[^A-Za-z0-9]+').
  • Multiple string transformations: replaceAllMapped -> replaceAll -> toLowerCase -> split(' ') -> where -> toList -> join('_').
  • A third dynamic regex (RegExp(r'(^|_)(api|private|access|secret|auth)_key($|_)')).
    For 30 columns, this generates over 250 temporary heap objects per query execution.

Solution

  • Add a static thread-safe LRU/cache map (_piiCache) since standard database column names (id, name, created_at, status, etc.) are highly repetitive.
  • Replace regex transformations with a simple character scanner on ASCII code units (codeUnitAt).

Benefits

  • Instant O(1) cache hits for recurring column names.
  • Zero regex compilations and zero split/join allocations during PII masking.

Activity

  1. added
    performanceTheme parser epic label: performance
    enhancementNew feature or request
    coreCore library logic and services
    P2Medium priority / Parity & Refactoring
    on Oct 12, 2026
  2. added this to the 0.5.0 milestone on Oct 12, 2026
  3. added a commit that references this issue on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority / Parity & RefactoringcoreCore library logic and servicesenhancementNew feature or requestperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions