Skip to content

perf(query): enforce limit parameter and reduce compact row parsing allocations - #72

Merged
ZhuchkaTriplesix merged 1 commit into
devfrom
issue/47-enforce-query-limit-and-reduce-parse-allocations
Sep 29, 2026
Merged

ZhuchkaTriplesix merged 1 commit into
devfrom
issue/47-enforce-query-limit-and-reduce-parse-allocations

Conversation

@ZhuchkaTriplesix

Copy link
Copy Markdown
Member

Summary

QueryParams.limit (sent by Querya Desktop as AppSettings.sqlResultMaxRows, typically 1,000–5,000) was deserialized in src/rpc/handlers/query.rs but never used anywhere. db.query always parsed and materialized the entire result set, which risks OOM on the 256 MB ClickHouse Sandbox ceiling for large tables.

  • parse_compact_output (src/mapper/row_compact.rs) now takes an Option<usize> limit and stops parsing/allocating further data rows once it's reached, instead of parsing everything and discarding the excess. handle_query passes QueryParams.limit through on both the mock and real ClickHouse paths.
  • Replaced the Vec<&str> line collection with direct iteration over output_text.lines(), dropping the intermediate allocation of the whole line list before any parsing starts.
  • Added query_id directly to QueryResult instead of round-tripping through serde_json::to_value(...) once just to mutate the resulting Value via as_object_mut() to splice in a queryId key.

All other parse_compact_output call sites (schema introspection in schema.rs, tree building in tree.rs) pass None — they read small, bounded system tables and don't need truncation.

Closes #47

Test plan

  • cargo test — 98/98 passing, including new regression tests: parse_compact_output limit enforcement (under/over/zero/no limit) and its query_id field, plus an end-to-end handle_query mock test asserting a limit: 1 request actually returns 1 row
  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features -- -D warnings

…llocations. Closes #47

QueryParams.limit was deserialized but never used, so db.query with a
limit still fully parsed and materialized unbounded result sets in
memory (every cell heap-allocated as a serde_json::Value), risking OOM
under ClickHouse Sandbox's 256 MB ceiling on large tables.

- parse_compact_output now takes an Option<usize> limit and stops
  parsing (and allocating) further data rows once it's reached, instead
  of parsing everything and discarding the excess. handle_query passes
  QueryParams.limit through on both the mock and real ClickHouse paths.
- Replace the Vec<&str> line collection with direct iteration over
  output_text.lines(), removing an intermediate allocation of the whole
  line list before any parsing starts.
- Add query_id directly to QueryResult instead of round-tripping through
  serde_json::to_value(...) once to get a Value, then mutating it via
  as_object_mut() to splice in a queryId key.
@ZhuchkaTriplesix
ZhuchkaTriplesix merged commit 01250a8 into dev Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant