Skip to content

perf(process-storms): rolling-window AORC scan, read each year once (~3-5x, opt-out)#16

Open
nghiemv wants to merge 1 commit into
USACE-Cloud-Compute:mainfrom
nghiemv:perf/rolling-scan
Open

perf(process-storms): rolling-window AORC scan, read each year once (~3-5x, opt-out)#16
nghiemv wants to merge 1 commit into
USACE-Cloud-Compute:mainfrom
nghiemv:perf/rolling-scan

Conversation

@nghiemv

@nghiemv nghiemv commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What

The upstream scan re-reads each AORC hour once per overlapping storm window
(storm_duration / check_every_n_hours times — 3× at 72h/24h, ~12× at 72h/6h).
The rolling-window scan sweeps each year once: keep a running cumulative
sum, snapshot at the indices each storm-date needs, so a window sum is
snapshots[end] - snapshots[start]. Complexity O(area × D × num_dates)
O(area × T); peak memory bounded to ~one time-chunk + the snapshot pool. The
speedup scales with cadence: ~3× at 72h/24h, ~5× measured at 72h/6h.

Ported from the fork's cumsum_scan, renamed rolling. Monkey-patches
collect_event_stats via install()/restore() around new_collection, gated
on CC_ROLLING_SCAN.

Default-on: yes, with confidence

Shipped default-on (CC_ROLLING_SCAN=1). This is justified, not aspirational:
end-to-end parity is confirmed bit-identical on a real production domain (see
below), so default-on changes speed, not results. CC_ROLLING_SCAN=0 remains a
one-flag escape hatch to the exact upstream per-date path.

Validation (end-to-end, real AORC) — ✅ confirmed

Real indian-creek production domain (fetched geometry), fast
aorc-cache-conus, 2-year window, full pipeline (scan → DSS → grid → upload):

  • Parity — bit-identical. rolling vs baseline (CC_ROLLING_SCAN=0)
    storm-stats.csv: 2925 / 2925 rows byte-exact (min/mean/max/x/y), 0 diffs.
  • Perf — ~5.4×. rolling 585.8s vs baseline 3137.7s total (72h/6h).
  • Unit: test/test_rolling_scan.py parity-tests the numerical core against
    naive window sums (chunk-size invariant, NaN-as-zero, toggle). 4 pass.

CI gates (verified locally on this rebased branch)

ruff check src/ ✅ · ruff format --check src/ ✅ · docker compose build ✅ ·
built image imports rolling_scan, enabled()True.

Reviewer notes

env default effect
CC_ROLLING_SCAN 1 rolling-window scan; 0 = upstream per-date loop
CC_ROLLING_CHUNK_HOURS 720 streaming time-chunk (memory vs overhead)

@nghiemv
nghiemv marked this pull request as draft July 14, 2026 21:24
@nghiemv
nghiemv marked this pull request as ready for review July 15, 2026 18:46
@nghiemv
nghiemv force-pushed the perf/rolling-scan branch from 82c6daa to 3a20e63 Compare July 16, 2026 14:40
@nghiemv nghiemv changed the title perf(process-storms): rolling-window AORC scan (~3x, opt-out) perf(process-storms): rolling-window AORC scan, read each year once (~3-5x, opt-out) Jul 16, 2026
@nghiemv
nghiemv force-pushed the perf/rolling-scan branch 2 times, most recently from 2c8d4e3 to 7b620a9 Compare July 16, 2026 15:09
The upstream scan re-reads each AORC hour once per overlapping storm window
(storm_duration / check_every_n_hours times: 3x at 72h/24h, ~12x at 72h/6h).
The rolling-window scan sweeps each year ONCE: keep a running cumulative sum,
snapshot it at the indices each storm-date needs, so a window sum is
snapshots[end] - snapshots[start]. Complexity O(area x D x num_dates) ->
O(area x T); peak memory bounded to ~one time-chunk + the snapshot pool.

Ported from the fork's cumsum_scan, renamed rolling (CC_ROLLING_SCAN,
CC_ROLLING_CHUNK_HOURS). Monkey-patches collect_event_stats via
install()/restore() around new_collection, gated on CC_ROLLING_SCAN (default
on; 0 = upstream per-date path).

Validated end-to-end on real AORC against the real indian-creek domain
(fetched geometry, fast aorc-cache-conus, 2-year window, full pipeline):
  - Parity: rolling vs baseline storm-stats.csv = 2925/2925 rows byte-exact
    (min/mean/max/x/y), 0 diffs. Numerical core also unit-tested against naive
    window sums (test/test_rolling_scan.py).
  - Perf: rolling 585.8s vs baseline 3137.7s total (~5.4x at 72h/6h).

Default-on is safe: parity is bit-identical, so it changes speed, not results.
@nghiemv
nghiemv force-pushed the perf/rolling-scan branch from 7b620a9 to 6daa0fb Compare July 16, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant