Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
8674936
WIP: FixedInterpCoordinate
atrabattoni May 12, 2026
e3a71d5
Merge remote-tracking branch 'origin/dev' into feature/fixed-interp-c…
atrabattoni May 12, 2026
341fd32
Change parse, parse_tolerance -> parse_data_dim, parse_scalar_delta.
atrabattoni May 13, 2026
ca3d7c7
WIP: sampling_interval/tolerance checking + some tests
atrabattoni May 14, 2026
6c899c5
Merge branch 'dev' into feature/fixed-interp-coords
atrabattoni May 14, 2026
09a8b5b
Merge branch 'dev' into feature/fixed-interp-coords
atrabattoni May 16, 2026
0f7ddbd
Merge branch 'dev' into feature/fixed-interp-coords
atrabattoni Jun 19, 2026
524d9ee
Refactor FixedInterpCoordinate to align with private method conventions
atrabattoni Jun 19, 2026
2165f53
Implement RegularInterpCoordinate and split coordinate mixins
atrabattoni Jun 20, 2026
6b4f14c
Document RegularInterpCoordinate and the get_sampling_interval change
atrabattoni Jun 20, 2026
a7bd6f1
Fix get_sampling_interval returning None incorrectly and from_block f…
atrabattoni Jun 20, 2026
7bf383e
Collapse RegularInterpCoordinate into InterpCoordinate
atrabattoni Jun 20, 2026
b34431f
Fix from_block producing an endpoint inconsistent with sampling_interval
atrabattoni Jun 20, 2026
7e166fe
Introduce AxisCoordinate intermediate ABC
atrabattoni Jun 20, 2026
2038de8
Clean cross-submodule imports to go through __init__.py
atrabattoni Jun 20, 2026
68e9cf5
Merge PiecewiseMixin into AxisCoordinate
atrabattoni Jun 20, 2026
f8ef624
Compare split candidates against the left segment sampling interval
atrabattoni Jun 20, 2026
daf8a90
Infer regular sampling interval by minimising worst-case drift
atrabattoni Jun 22, 2026
e5b509f
Track running reference rate in DenseCoordinate split detection
atrabattoni Jun 22, 2026
56e35d5
Document InterpCoordinate CF semantics and centralise continuous-area…
atrabattoni Jun 22, 2026
0e2c6a9
Find Chebyshev-center spacing pair in O(n log n) via numba
atrabattoni Jun 22, 2026
f642bce
Document InterpCoordinate.simplify and cover discontinuity cases
atrabattoni Jun 22, 2026
ed4c141
Tighten InterpCoordinate helper internals
atrabattoni Jun 22, 2026
84e78aa
Widen simplify tolerance to absorb fused discontinuities
atrabattoni Jun 22, 2026
9678e41
Honour explicit to_regular arguments on an already-regular coord
atrabattoni Jun 22, 2026
17fe290
Make InterpCoordinate._concat strict, reconcile rates in concat_coords
atrabattoni Jun 22, 2026
460cb72
Drop to_regular auto tolerance mode
atrabattoni Jun 22, 2026
6f7a6cf
Add InterpCoordinate.infer_regular diagnostic helper
atrabattoni Jun 22, 2026
f6d984b
Add reduce/regularize stages to simplify, thread through concat
atrabattoni Jun 26, 2026
3b38913
Preserve input coordinate type in stft output coordinates
atrabattoni Jun 28, 2026
12c6079
Add xdas.testing.dummy fixture helper
atrabattoni Jun 28, 2026
0b5df0f
Migrate signal functions to coordinate-level get_sampling_interval
atrabattoni Jun 28, 2026
17a7f0c
Make coordinate regularity an explicit declaration
atrabattoni Jul 30, 2026
c03c320
Carry declared jitter through simplify and concat
atrabattoni Jul 30, 2026
4d9a946
Emit regular coordinates from IO engines and FFT
atrabattoni Jul 30, 2026
7ba04e7
Document regular coordinates and cover xdas.testing
atrabattoni Jul 30, 2026
bbdad06
Use xdas.testing.dummy for signal-agnostic test fixtures
atrabattoni Jul 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 55 additions & 18 deletions docs/api/coordinates.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ Methods
:toctree: ../_autosummary

Coordinates.isdim
Coordinates.get_query
Coordinates.to_index
Coordinates.equals
Coordinates.copy
Expand All @@ -43,15 +42,10 @@ Attributes
:toctree: ../_autosummary

Coordinate.dtype
Coordinate.ndim
Coordinate.shape
Coordinate.size
Coordinate.empty
Coordinate.dim
Coordinate.indices
Coordinate.values
Coordinate.start
Coordinate.end
Coordinate.name
```

Expand All @@ -61,14 +55,49 @@ Methods
.. autosummary::
:toctree: ../_autosummary

Coordinate.isscalar
Coordinate.isdim
Coordinate.isregular
Coordinate.equals
Coordinate.to_index
Coordinate.format_index
Coordinate.slice_indexer
Coordinate.copy
Coordinate.to_dataarray
```

## AxisCoordinate

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

AxisCoordinate
```

Attributes

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

AxisCoordinate.ndim
AxisCoordinate.empty
AxisCoordinate.indices
AxisCoordinate.start
AxisCoordinate.end
```

Methods

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

AxisCoordinate.isregular
AxisCoordinate.get_sampling_interval
AxisCoordinate.to_regular
AxisCoordinate.get_split_indices
AxisCoordinate.get_discontinuities
AxisCoordinate.get_availabilities
AxisCoordinate.simplify
AxisCoordinate.to_index
AxisCoordinate.to_dataarray
```

## ScalarCoordinate
Expand Down Expand Up @@ -106,7 +135,8 @@ Methods

DenseCoordinate.from_block
DenseCoordinate.get_sampling_interval
DenseCoordinate.get_div_points
DenseCoordinate.to_regular
DenseCoordinate.simplify
```

## InterpCoordinate
Expand All @@ -126,6 +156,8 @@ Attributes

InterpCoordinate.tie_indices
InterpCoordinate.tie_values
InterpCoordinate.sampling_interval
InterpCoordinate.tolerance
```

Methods
Expand All @@ -135,10 +167,8 @@ Methods
:toctree: ../_autosummary

InterpCoordinate.from_block
InterpCoordinate.to_regular
InterpCoordinate.get_sampling_interval
InterpCoordinate.get_split_indices
InterpCoordinate.get_discontinuities
InterpCoordinate.get_availabilities
InterpCoordinate.simplify
```

Expand Down Expand Up @@ -171,8 +201,15 @@ Methods

SampledCoordinate.from_block
SampledCoordinate.get_sampling_interval
SampledCoordinate.get_split_indices
SampledCoordinate.get_discontinuities
SampledCoordinate.get_availabilities
SampledCoordinate.to_regular
SampledCoordinate.simplify
```

## Functions

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

get_sampling_interval
```
1 change: 1 addition & 0 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ picking
processing
signal
synthetics
testing
virtual
```
1 change: 0 additions & 1 deletion docs/api/synthetics.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@

wavelet_wavefronts
randn_wavefronts
dummy
```
12 changes: 12 additions & 0 deletions docs/api/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
```{eval-rst}
.. currentmodule:: xdas.testing
```

# xdas.testing

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

dummy
```
1 change: 1 addition & 0 deletions docs/api/xdas.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
concat
concatenate
concat_coords
get_sampling_interval
split
plot_availability
```
Expand Down
202 changes: 202 additions & 0 deletions docs/plan_regular_coordinates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
---
orphan: true
---

# Design: regular coordinates — settling the open questions

Status: implemented on this branch (2026-07-29).
Branch: `feature/fixed-interp-coords`, targeting a PR to `dev` (0.2.8, untagged).

This document settles the four design questions left open by the regular-
coordinate work, so the remaining implementation (engine emission, docs pass,
failing tests) has a fixed contract to build against. It supersedes the
never-written `docs/plan_propagate_simplify_kwargs.md` referenced by
`concat`'s docstring.

## Background: the model as implemented

An `InterpCoordinate` may carry two optional metadata entries:

- `sampling_interval` — the nominal sample spacing. Its presence is what makes
the coordinate *regular* (`isregular()`).
- `tolerance` — the allowed jitter around that spacing. The validity invariant,
checked at construction (`_is_valid_sampling_interval`), is per continuous
segment: `|num - si * den| <= 2 * tolerance`, evaluated at the dtype
resolution (integer division for datetime64, so sub-resolution drift is
always absorbed).

`from_block` produces regular coordinates; `_to_regular` enforces or infers a
spacing (raising when it cannot); `simplify(tolerance, reduce, regularize)`
spends an accuracy budget on tie-point reduction and optional promotion to
regular; `_concat` is strict (keeps the spacing only when both sides agree
exactly, takes `max` of tolerances, otherwise drops to irregular).

## D1. Public API surface: `to_regular` public, `infer_regular` private

**Decision.** Promote `_to_regular` to public `to_regular`, defined on
`AxisCoordinate` (not just `InterpCoordinate`), honouring the rule that a
public coordinate method exists on the whole axis hierarchy or not at all:

- `InterpCoordinate.to_regular(sampling_interval=None, tolerance=None)` —
current `_to_regular` behaviour: enforce the given spacing, inferring it when
omitted, raising `ValueError` when the tie points cannot be described by a
single spacing within `tolerance`.
- `SampledCoordinate.to_regular(...)` — regular by construction: with no
arguments return a copy; with explicit arguments validate them against the
stored interval and raise on mismatch.
- `DenseCoordinate.to_regular(...)` — *conversion*: return a regular
`InterpCoordinate` built from the dense values (reduce within `tolerance`,
then enforce the spacing), raising when the values are genuinely irregular.
Returning a different subclass is acceptable: the `to_` prefix already
signals a conversion, and this is the natural "make this axis usable by
signal processing" entry point.

`_infer_regular` stays private. It is an implementation detail of
`to_regular`/`simplify` (the Chebyshev-center fit); exposing it publicly on
only one subclass would recreate the partial-interface problem, and its
diagnostic value is available through `to_regular`'s behaviour and error
message. `docs/api/coordinates.md` must drop the `infer_regular` entry and the
release notes keep advertising `to_regular` (now truthfully).

Consequence: `get_sampling_interval` (module level, `core.py:1244`) loses its
`hasattr(coord, "_to_regular")` duck-typing — see D3.

## D2. What "regular" means per subclass (the Dense question)

**Decision.** *Regular* means "carries an explicit nominal sampling interval",
uniformly:

- `InterpCoordinate`: regular iff `sampling_interval` metadata is present.
- `SampledCoordinate`: always regular (the interval is part of its data).
- `DenseCoordinate`: **never regular**. `get_sampling_interval` returns `None`
unconditionally, dropping the current end-to-end average. The average makes
`isregular()` vacuously true for any dense axis and silently hands a
meaningless rate to signal routines on jittery data — the exact failure mode
this branch exists to eliminate. A dense axis that really is evenly sampled
becomes regular explicitly, via `to_regular` (D1) or
`simplify(regularize=True)`.
- `ScalarCoordinate`: `isregular()` moves to the `Coordinate` base and returns
`False` there; `AxisCoordinate` overrides it with the current
`get_sampling_interval() is not None`. This makes the release-notes claim
("on the base ABC") true and removes the `AttributeError` on scalar coords.

## D3. The `get_sampling_interval` contract: strict, one choke point

Three layers, each with a single behaviour:

1. **Primitive** — `coord.get_sampling_interval(cast=True)`: return the
nominal interval, or `None` when the coordinate is not regular. Never
raises, never infers, O(1).
2. **Conversion** — `coord.to_regular(...)`: the only place inference and
enforcement happen. Raises with an actionable message on genuinely
irregular axes.
3. **Convenience** — `xdas.get_sampling_interval(da, dim)`: return the nominal
interval when the coordinate is regular, otherwise **raise** `ValueError`
telling the user how to fix it (open the files with a `tolerance`, or
`da[dim] = da[dim].to_regular(tolerance=...)`). The current silent
`_to_regular()` fallback is removed: it hides an O(n log n) inference in
every FFT/filter call and only ever succeeds on exactly-uniform axes anyway
(the implicit epsilon tolerance rejects any real jitter), so its benefit is
marginal and its implicitness is not.

*Amendment (2026-07-30):* data saved by earlier versions carries no
`sampling_interval` metadata, so raising immediately would break every
signal-processing call on existing archives. For one deprecation cycle the
helper therefore falls back to inference on irregular coordinates: it infers
the spacing (and, for `InterpCoordinate`, the minimal tolerance that
validates it via the Chebyshev fit), emits a `FutureWarning` stating both
values and the migration path, and returns the inferred spacing. Dense
coordinates go through the strict `to_regular()` (uniform axes work, jittery
ones still raise — the old end-to-end average was a silent wrong answer not
worth preserving). Raising remains only where no spacing can be inferred at
all. The strict behaviour described above becomes the default when the
deprecation completes.

**Migration.** All signal-consuming code goes through layer 3 — including
`xdas/signal.py`, which currently open-codes the strict check six times
(`d = coords[dim].get_sampling_interval(); if d is None: raise ...`). Revert
those to the module-level helper so the error message and the policy live in
one place, and keep `fft.py`, `spectral.py`, `atoms/`, `picking.py`,
`miniseed.py` on the helper. Net user-visible behaviour: every signal routine
raises the *same* error on irregular axes, and none of them raise on data
opened through the engines once D5 lands.

Also fix `DataArrayList`-style compatibility checking
(`routines.py:919-922`): `get_sampling_interval` returning `None` for the
incoming chunk must produce a `CompatibilityError`, not a `TypeError` inside
`np.isclose`.

## D4. Tolerance semantics and propagation

**Meaning.** `tolerance` is a *declared jitter bound carried by the
coordinate*: the promise that every continuous segment satisfies
`|num - si * den| <= 2 * tolerance` at the dtype resolution. It is data, not a
processing parameter — processing functions take a *budget* argument that may
default to it.

**Propagation rules** (R1–R2 already implemented, kept as-is):

- **R1 — slicing/striding** (`_slice`): spacing scales by the step, tolerance
is preserved.
- **R2 — raw concatenation** (`_concat`): strict; equal spacings are kept with
`max` of tolerances, anything else drops to irregular. Reconciliation is the
job of user-facing routines via `simplify`.
- **R3 — derived rates must carry their quantization error.** Any operation
that synthesizes a new nominal spacing that is not exactly representable in
the coordinate dtype must record the representation error in `tolerance`
instead of claiming `0`. Concretely for `Upsample(factor)` on datetime axes:
`new_delta = delta // factor` truncates, so the coordinate must carry
`tolerance >= (delta - factor * new_delta)` (2 ns in the failing test) on
top of the inherited tolerance. This is what makes chunk seams land within
tolerance of the nominal grid.
- **R4 — `simplify(tolerance=None)` defaults to the coordinate's own stored
tolerance** (falling back to the current zero-like default when the
coordinate has none). Rationale: the coordinate has already declared "my
values are only meaningful to within `tolerance`"; a canonicalisation pass
that refuses to spend that declared slack is pointless strictness. This
applies to `concat(tolerance=None)` too, per-coordinate. `tolerance=False`
keeps its "no simplification" meaning; an explicit scalar overrides.
- **R5 — no unconditional widening.** `InterpCoordinate.simplify` on a regular
coordinate currently stores `self.tolerance + tolerance` whenever `reduce`
runs. Replace with: after reduction, keep the original tolerance if it still
validates, and only widen (to the smallest valid value, bounded by
`self.tolerance + budget`) when it does not. Without this, chunked and
unchunked pipelines can never produce `equals()` coordinates because the
chunked path concatenates and re-simplifies.

**Why this fixes `test_upsample`.** Each upsampled chunk carries
`sampling_interval = 6_666_666 ns, tolerance = 2 ns` (R3). `_concat` keeps the
spacing (R2). `concat`'s simplify defaults its budget to the stored 2 ns (R4),
Douglas-Peucker drops the seam tie points (they deviate ≤ 2 ns from the global
line), and R5 keeps `tolerance = 2 ns` — identical to the unchunked result.

**Defaults alignment.** `concat` and `concat_coords` currently disagree
(`regularize=False, tolerance=None` vs `regularize=True, tolerance=False`).
Align `concat_coords` to `concat`: `reduce=True, regularize=False,
tolerance=None` (with R4's meaning). `regularize` stays opt-in for this PR —
with engines emitting regular coordinates (D5) and R2 preserving them,
multi-file opens stay regular without promotion, so the conservative default
costs nothing; flipping it can be revisited once propagation has soaked.

## D5. IO emission (scope confirmed, design only sketched here)

Engines construct per-file time/space coordinates with
`InterpCoordinate.from_block(start, size, step)` (the existing `# TODO: use
from_block` sites in `prodml`, `terra15`, `asn`, plus `miniseed.read_stream`
and ObsPy `from_stream`, which must also build at ns resolution to round-trip
`to_stream`). Per-file tolerance is `0`: within one file the grid is exact by
construction. Cross-file jitter is reconciled where it appears — at
`concat`/`open_mfdataarray` time via the user-supplied `tolerance` (R4/R2).
`from_stream` uses `stats.delta`; engines use the file's metadata rate.

## Acceptance criteria

- `tests/test_atoms.py::TestFilters::test_upsample` and
`tests/test_dataarray.py::TestIO::test_stream` pass without weakening the
assertions.
- `xd.signal.*`, `xd.fft.*`, `xd.spectral.*`, and the atoms raise one uniform,
actionable error on irregular axes, and raise nothing on engine-opened data.
- Release notes, `docs/api/coordinates.md`, and the user guide describe only
APIs that exist (`to_regular` public, `infer_regular` gone from docs).
- `concat`'s docstring no longer references this document's missing
predecessor.
14 changes: 11 additions & 3 deletions docs/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@

## 0.2.8

### Breaking Changes
- Removed `DefaultCoordinate`, the `Coordinate.is*` predicate properties (`isdense`, `isdefault`, `isinterp`, `issampled`), and `to_dict`/`from_dict` from `DataArray` and coordinate types. These were internal APIs not intended for public use, so this should not affect user code (@atrabattoni).
### New Features
- **Regular coordinates.** A coordinate can now declare a nominal `sampling_interval` (with a `tolerance` bounding the allowed jitter). Query it with `isregular()` / `get_sampling_interval()`; promote an irregular coordinate with `to_regular()`. File engines, `from_block`, and the `fft`/`stft` outputs produce regular coordinates out of the box (@atrabattoni).
- Chunked and unchunked processing now yield identical coordinates: operations that derive a new rate record their rounding error in `tolerance`, and `simplify`/`concat` spend the declared tolerance by default, fusing chunk seams away (@atrabattoni).
- `simplify` gained `reduce` and `regularize` keywords, and the gaps/overlaps API now works on every axis coordinate, including dense ones (@atrabattoni).

### Deprecations
- The sampling interval is now declared metadata rather than a computed end-to-end average (which was silently wrong on jittery or gappy axes). Data saved by earlier versions carries no declared rate: querying it — e.g. through any signal-processing routine — still works for now, but the rate is inferred and a `FutureWarning` explains how to make the coordinate regular (`da[dim] = da[dim].to_regular(tolerance=...)`). A future release will raise instead (@atrabattoni).

### Refactoring
- `Coordinate` is now a proper ABC with an explicit abstract interface; shared ordered-coordinate logic is consolidated in `SampledMixin`; NumPy 2.0 `copy` keyword compliance (@atrabattoni).
- Reworked the coordinate class hierarchy: `Coordinate` is now a proper ABC and the new `AxisCoordinate` ABC holds the axis-mapping contract shared by dense, interpolated, and sampled coordinates. Use `isinstance(coord, AxisCoordinate)` instead of the removed `is*` predicates (@atrabattoni).
- Cleaned up internal-leaning APIs: removed `DefaultCoordinate`, `to_dict`/`from_dict`, `get_div_points`, `decimate`, and `from_array`; made underscore-private `concat`, `get_indexer`, `get_value`, `format_index`, `slice_index(er)`, `isvalid`, and `get_query`; NumPy 2.0 `copy` keyword compliance (@atrabattoni).
- `concat_coords` now simplifies its result by default, like `concat`; values are unchanged, only redundant tie points are dropped (@atrabattoni).
- Added `xdas.testing.dummy`, a configurable fixture generator replacing `xdas.synthetics.dummy` (@atrabattoni).

## 0.2.7

Expand Down
Loading
Loading