Skip to content

perf(mongodb): single-pass tokenizer in MqlParser._normalizeJsObject eliminating repeated string copying #1539

Open

Description

@ZhuchkaTriplesix

Problem

In MqlParser._normalizeJsObject (lib/features/mongodb/mql_parser.dart):

  1. Runs three replaceAllMapped passes with dynamic RegExps.
  2. Replaces string literals with placeholders (@@@STR_0@@@, @@@STR_1@@@).
  3. Iterates over placeholders with processed = processed.replaceAll('@@@STR_$i@@@', strings[i]).
    For a query containing 50 strings, replaceAll creates 50 complete copies of the entire query string in heap.

Solution

  • Replace the multi-pass placeholder mechanism with a single-pass tokenizer/scanner streaming normalized JSON into a StringBuffer.
  • Quote keys and preserve string literals in one linear scan (N)$ without intermediate query re-allocations.

Benefits

  • Single linear pass (N)$ with zero repeated string cloning.
  • Significantly faster MQL parsing for batch inserts and complex queries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority / Parity & RefactoringbackendBackend database driver execution and queriesenhancementNew feature or requestparserTheme parser epic label: parserperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions