Skip reparsing canonical CoordRange coords in CoordManager update/select - #768
Conversation
CoordManager.update / update_coords / select funnel every coordinate through _get_coord (in _get_coord_dim_map), which dumps each coord via model_dump and rebuilds it with get_coord -- re-running full validation (and, for array coords, the monotonic/even-sampling scan) even when the coord was already valid. Short-circuit the one case that is provably a no-op: a CoordRange is the canonical evenly-sampled representation and round-trips identically, so return it directly. Other coord types are still re-inferred, because slicing (e.g. __getitem__ during sample selection) can leave an array coord non-canonical, and a fully-specified CoordPartial should canonicalize to a CoordRange. Micro-benchmark on master-equivalent code: update_coords with CoordRange dims ~393us -> ~22us; value-based select ~596us -> ~180us. Add identity, equivalence, and canonicalization regression tests plus an update_coords benchmark.
|
Warning Review limit reached
Next review available in: 1 minute Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #768 +/- ##
=======================================
Coverage 99.93% 99.93%
=======================================
Files 145 145
Lines 12857 12859 +2
=======================================
+ Hits 12849 12851 +2
Misses 8 8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Why the guard is
|
| Type | Stores | Used when |
|---|---|---|
CoordRange |
start, stop, step |
values are evenly spaced |
CoordMonotonicArray |
the full array | increasing but uneven (e.g. [0, 1, 3, 9]) |
CoordArray |
the full array | not even sorted |
CoordPartial |
just a length | a dim with a size but no values |
[0, 10, 20, 30] is evenly spaced, so its canonical form is CoordRange(start=0, stop=40, step=10) — compact, and it reports evenly_sampled == True. The same values as a CoordMonotonicArray would be correct-but-non-canonical (bulkier, and reports evenly_sampled == False).
get_coord(data=...) is what performs this inference ("evenly spaced → CoordRange, monotonic-uneven → CoordMonotonicArray, empty → CoordPartial"), so anything built through it comes out canonical.
The redundancy this PR removes. Every update/select funneled each coord through _get_coord, which did:
coord = coord.model_dump() # dump a valid coord object to a plain dict
coord = get_coord(**dict) # ...and rebuild it from scratcheven when coord was already a valid, canonical CoordRange. That round-trip re-runs the full even-spacing scan over the array for nothing — the ~393 µs → ~22 µs (update_coords) and ~596 µs → ~180 µs (value select) cost.
Why we can't skip the rebuild for every coord. That rebuild wasn't only wasted work — for some coords it was silently canonicalizing a coord that had gone stale. Coords go non-canonical via slicing: sample-selection does array indexing (coord[indices]) which keeps the same type and does not re-infer spacing. So:
[0, 1, 3]→CoordMonotonicArray(uneven, correct)- sample-select indices
0,1→ subset[0, 1] - the slice stays a
CoordMonotonicArray([0, 1])— but[0, 1]is now evenly spaced and should be aCoordRange.
The old rebuild fixed that (get_coord([0,1]) → CoordRange). A blanket "skip all BaseCoords" guard would drop the fix-up, leaving a CoordMonotonicArray that wrongly reports evenly_sampled == False — same values, wrong representation. (A second variant: a fully-specified CoordPartial should expand into a CoordRange, else value-based selection on it raises.)
Why CoordRange alone is safe. A CoordRange can never be non-canonical — it is the canonical even form, and slicing a CoordRange yields another CoordRange. So rebuilding one is always pure waste. Every other type might be sitting in a non-canonical state, so those keep going through the rebuild:
if isinstance(coord, CoordRange):
return coord # already canonical — no rebuild needed
# all other types fall through to the rebuild, which canonicalizes themThe two regression tests pin exactly the traps that ruled out a broader guard: an even sample-select subset must become a CoordRange, and a fully-specified CoordPartial must become a CoordRange.
TL;DR: the old code rebuilt every coordinate partly to waste time and partly to canonicalize coords that slicing had left in the wrong representation. This skips the rebuild only for CoordRange — the one type guaranteed already-canonical — dropping the wasted work without dropping the fix-up.
* Improve draft-release skill effectiveness (#745) * Raise CoordError instead of assert for non-1D coord operations (#747) * Raise CoordError instead of assert for non-1D coord operations Several coordinate operations that only support 1D coords guarded their input with `assert`, which raises a bare AssertionError and is stripped entirely under `python -O` (so the check silently vanishes in optimized runs). Convert the user-reachable ones -- select-by-sample-array, align_to, get_sample_count, CoordPartial.change_length, and CoordRange construction -- to raise CoordError. Genuine internal invariants that are impossible by construction (CoordRange.change_length) stay as asserts. Add tests covering each new error path. * Extend assert->raise cleanup to proc, viz, and wav IO Apply the same treatment repo-wide to user-reachable asserts that validate caller input, converting them to ParameterError: - proc/taper: taper window must be a length-2 sequence - proc/detrend: dim must be in the patch - proc/correlate: patch must be 2D - viz/map_fiber: x/y/color must be existing coords; scale_type and scale validated - io/wav: only single-patch spools can be written to wav Internal invariants (impossible-by-construction shape/postcondition checks, binary-format parser consistency) are left as asserts. Adds tests for every new error path; the five changed modules keep 100% line coverage. * Fix/fbe decibel (#755) * fixed decibel scaling factore to 20 (was 10) * fixed test to match new decibel factor * implemented gap-sensistive waterfall plot (#753) * implemented gap-sensistive waterfall plot * added 3 more tests * refactor gap_detection and mesh-coordinates; handle datetime64 natively * Add no-op fast paths for transpose/squeeze and idempotent coordinate snapping (#765) * CI: bump actions off deprecated Node.js 20 runtime (#766) * Skip reparsing canonical CoordRange coords in CoordManager update/select (#768) --------- Co-authored-by: Andreas Wuestefeld <115324323+andreas-wuestefeld@users.noreply.github.com>
What
CoordManager.update/update_coords/selectfunnel every coordinatethrough
_get_coord(nested in_get_coord_dim_map), which dumps each coord viamodel_dump()and rebuilds it withget_coord(**dict)— re-running fullvalidation and, for array coords, the monotonic / even-sampling scan — even
when the coordinate was already a valid, canonical object. Because
BaseCoordis a pydantic model, this bypassed
get_coord's ownif isinstance(data, BaseCoord): return datafast path.This adds a single short-circuit for the one type where returning the coord
unchanged is provably a no-op:
A
CoordRangeis the canonical evenly-sampled representation and round-tripsidentically through
model_dump()→get_coord().Why only CoordRange (not all BaseCoords)
The reparse also canonicalizes non-canonical coordinates, so it cannot be
blanket-skipped:
CoordArray/CoordMonotonicArray) can be leftnon-canonical by slicing — e.g. an evenly spaced subset produced by
__getitem__during sample selection should collapse to aCoordRange.CoordPartialshould canonicalize to aCoordRange(otherwise value-based selection on it wrongly raises).
Both cases are covered by regression tests.
CoordRangewas verified (andindependently confirmed in review) to round-trip identically for forward,
reversed, singleton, datetime, unit-bearing, and dtype-bearing variants.
Performance (micro-benchmark on master-equivalent code)
update_coords, CoordRange dimsselect, CoordRangeArray-coord paths are intentionally unchanged (they still reparse for
canonicalization). Output is bit-for-bit identical: coordinate arrays, dim maps,
dtypes, data, attrs, and history are unchanged.
Tests / benchmarks
CoordRangeupdate/select; value-equivalence forarray coords; the select result is unchanged.
CoordRange; fully-specifiedCoordPartial→CoordRangewith workingvalue-select.
test_update_existing_coordsCodSpeed benchmark.Verification
pytest tests→ 6318 passed, 84 skipped, 4 xfailed.coordmanager.pypass;pre-commitclean.(array-slice and fully-specified-CoordPartial canonicalization) were fixed and
now have regression tests.
Follow-up to #765 / #757; see
.scratch/optimization_ideas.md(PR 3).