Skip to content

Fix lake cold startup, restart cache persistence, and pgwire deadlines - #1019

Merged
ajroetker merged 2 commits into
mainfrom
fix/lake-cold-start-pgwire-deadlines
Oct 8, 2026
Merged

ajroetker merged 2 commits into
mainfrom
fix/lake-cold-start-pgwire-deadlines

Conversation

@ajroetker

Copy link
Copy Markdown
Contributor

Cold remote lake searches were opening segment readers serially and scanning every dictionary block to populate diagnostic statistics. Disk-cache writes were also rejected on macOS because capacity freshness compared native monotonic timestamps with Io awake timestamps. Pgwire passed the same mismatched absolute clock through native catalog/storage boundaries, causing six lake SQL tests to return QueryCanceled.

Defer range-backed dictionary diagnostics, admit readers in bounded parallel jobs and transfer them to the writer without reopening, overlap bounded required header reads, and fetch complete immutable artifacts with one bounded GET plus exact length/SHA-256 verification. Normalize cache capacity and pgwire deadlines into the native clock domain. Catalog reader protocols 24–33 remain parseable for reconciliation; serving still requires the current protocol.

Raw BigQuery HN export in antfly-dev-01, local Debug binary, 10,000 rows:

Measurement Prior main run Two fixed runs
First text search 16.5 s 7.6–10.2 s
Warm text search 369 ms 379–400 ms
First search after process restart 18.1 s 418–432 ms
Persisted cache files 0 82 (about 5 MiB)

The new runs copied catalog state into directories with no cache files before startup; the harness SQL count precedes text search. Empty-cache search still pays required remote metadata/posting/hydration I/O. These are smoke measurements, not archive-scale or deployment-region benchmarks. Update the Hacker News example and qualification record with the results.

Validation:

  • antfly build plus lake-integration-test and lake-test: all build steps passed; 695 tests passed with no leaks.
  • Final test-only addition exercises OOM after prepared-reader transfer: all 695 lake tests passed again with no leaks.
  • Full test_lake_sql.py: 20 passed, including all six previous pgwire cancellation failures, restart/ranking/highlighting and indexed predicates above 100,000 rows.
  • Focused pgwire offset-clock/expiry/explicit cancellation regression: passed.
  • Two raw GCS smoke runs: score order, highlights, exact filters, pagination and process restart passed.
  • Regression checks cover offset-clock cache writes/reopen, bounded index admission, prepared-reader transfer/OOM, and appended/truncated/same-length artifact corruption.
  • License headers and git diff --check passed.

@ajroetker
ajroetker merged commit 272db51 into main Oct 8, 2026
2 checks passed
@ajroetker
ajroetker deleted the fix/lake-cold-start-pgwire-deadlines branch October 8, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant