Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
74a0411
Add minimal CI: offline pytest matrix on ubuntu/macos, py3.10/3.12
mdenolle Aug 3, 2026
b72f898
Phase 1: parse fast path, segment-aware API, deterministic S3, window…
mdenolle Aug 3, 2026
6d7da61
Phase 2+3+4ab: benchmarks with persisted results, obspy-free NoisePy …
mdenolle Aug 3, 2026
34254da
Phase 4c + 5: CCF bit-equivalence through real noisepy, lazy imports,…
mdenolle Aug 3, 2026
b614a2b
Machine matrix results + final report
mdenolle Aug 3, 2026
72b824a
Response removal without obspy + container parse fix
mdenolle Aug 3, 2026
a732d2a
Refresh benchmarks after owned-buffer parse fix; warmup in timing loops
mdenolle Aug 3, 2026
5f6d176
Notebook 05: response removal without obspy — methods, residuals, spe…
mdenolle Aug 4, 2026
2e4f100
One-to-one method match in the response notebook + module refinements
mdenolle Aug 4, 2026
8b9c820
Three-persona external critique (obspy RSE, EarthScope cloud RSE, res…
mdenolle Aug 4, 2026
ab7b4d3
Resolve critique B1: compound MIT AND LGPL-3.0-only license
mdenolle Aug 4, 2026
862f0bc
Fix critique B2+B3: typed fetch errors, working wildcards, half-open …
mdenolle Aug 4, 2026
689e063
Critique resolution log: B2+B3 fixed, live-verified
mdenolle Aug 4, 2026
ce451a8
Fix critique B4: response module fails loudly on dirty metadata
mdenolle Aug 4, 2026
640c901
Fix critique correctness majors: merge policy, mixed rates, path pari…
mdenolle Aug 4, 2026
7380f72
Fix critique operations pass: hardened clients for campaign-scale use
mdenolle Aug 4, 2026
8a079f9
Project shell: 0.3.0, CHANGELOG, CI legs, pymseed 0.9, hygiene, plott…
mdenolle Aug 4, 2026
46d1e60
Critique resolution log: project shell resolved — full critique closed
mdenolle Aug 4, 2026
40b30eb
Fix CI lint failure: pin ruff to one version everywhere (0.15.9)
mdenolle Aug 4, 2026
6bf3370
Release plumbing: PyPI publish workflow (trusted publishing) + lean s…
mdenolle Aug 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .DS_Store
Binary file not shown.
39 changes: 39 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: publish

# Publishes to PyPI when a version tag is pushed (v*). Uses PyPI Trusted
# Publishing (OIDC) — configure once at pypi.org: project seisfetch →
# Publishing → add GitHub publisher (owner Denolle-Lab, repo seisfetch,
# workflow publish.yml, environment pypi). No token secrets needed.

on:
push:
tags: ["v*"]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install build twine
- run: python -m build
- run: twine check dist/*
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

publish:
needs: build
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write # OIDC for trusted publishing
steps:
- uses: actions/download-artifact@v4
with:
name: dist
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1
62 changes: 62 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: test

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: "3.12"
# pinned to the same ruff as pre-commit/pixi — unpinned CI ruff
# drifted ahead and failed format --check on style changes
- run: pip install ruff==0.15.9
- run: ruff check seisfetch tests benchmarks
- run: ruff format --check seisfetch tests benchmarks

pytest:
strategy:
fail-fast: false
matrix:
include:
- { os: ubuntu-latest, python: "3.9" }
- { os: ubuntu-latest, python: "3.10" }
- { os: ubuntu-latest, python: "3.12" }
- { os: ubuntu-latest, python: "3.13" }
- { os: macos-latest, python: "3.12" }
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Install with dev extras
run: pip install -e ".[dev]"
- name: Unit tests (offline)
run: pytest tests -m "not integration" -q

# informational: does the NEXT pymseed break us? Never blocks merges.
pymseed-next:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install with latest pymseed (ignore our pin)
run: |
pip install -e ".[dev]"
pip install --upgrade pymseed
python -c "import pymseed; print('pymseed', pymseed.__version__)"
- name: Parse-path tests against latest pymseed
run: >
pytest tests/test_convert.py tests/test_segments_api.py
tests/test_correctness_majors.py
tests/precision/test_parse_identity.py -q
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.6
rev: v0.15.9
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
Expand Down
95 changes: 95 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Changelog

All notable changes to seisfetch are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
[Semantic Versioning](https://semver.org).

## 0.3.0 — 2026-08-04

The obspy-replacement evaluation release: parse rewrite, gap-aware API,
typed failure contract, instrument-response removal, and the full
three-persona external critique with every blocker, correctness major, and
operations finding resolved (`docs/reviews/2026-08-external-critique.md`).

### Changed (breaking or behavior-visible)

- **License** is now the compound expression `MIT AND LGPL-3.0-only`:
ObsPy-derived numerical translations live in
`seisfetch/contrib/obspy_ports.py` under LGPL-3.0-only; everything else
stays MIT. See THIRD_PARTY_NOTICES.md.
- **Failure contract**: fetch paths no longer swallow errors into empty
bytes. Clean 404s are tolerated per key; any other failure raises
`FetchError` (`on_error="warn"` opts out); nothing-found raises
`NoDataError` unless `missing_ok=True`. FDSN maps 204/404 to no-data and
raises `FDSNError` on real HTTP errors.
- **`location="*"` (the default) now works**: resolved by paginated LIST
discovery on SCEDC/NCEDC (finds location-coded channels instead of
guessing blank-location keys); FDSN passes `*` through and spells blank
as `--`.
- **Day windows are half-open** `[start, end)`: a request ending at
midnight no longer fetches the following day's objects.
- **`get_numpy`/`get_xarray` trim to the requested window by default**
(`trim=False` restores whole-object behavior) and filter station-day
objects to the requested channel/location after parse.
- **`bundle_to_obspy`/`get_waveforms` return one Trace per segment**
(obspy-read parity; no masked arrays). `merge=1` restores the old
force-merge.
- **`to_dict()` warns on gappy channels** (the legacy concatenation has a
wrong time axis after the first gap); `to_dict(fill_value=)` places
segments at true offsets with obspy `merge(method=1)` overlap semantics.
- **Mixed sampling rates under one NSLC raise `MixedSamplingRateError`**
from `to_dict`/`metadata` (`segments()` is the per-rate escape hatch).
- **`FDSNMultiClient` defaults to failover** (first non-empty provider
wins); the old broadcast is `strategy="broadcast"`.
- **`fetch_bulk_numpy` drops raw bytes after parsing** (`keep_raw=True`
restores; byte accounting preserved).
- **`parse_mseed(collect_flags=False)`**: per-record quality flags are now
opt-in; `TraceBundle.traces` holds true continuous segments (one per
contiguous run, not one per miniSEED record) and `num_segments` counts
real segments.
- pymseed pinned `>=0.6,<0.10` (verified against 0.9.3); the private-API
sid fallback is isolated behind a guard.

### Added

- **Parse fast path** on libmseed's `MS3TraceList` with decode into
numpy-owned memory: 11 MB Steim2 channel-day 217 ms (v0.2.0) → ~21 ms,
faster than `obspy.read` in every measured environment, at ~1/3 the
parse memory.
- **Gap-aware API**: `TraceBundle.segments()`, `trim()`, `overlaps()`,
`to_dict(fill_value=)`.
- **`seisfetch.exceptions`**: `SeisfetchError`, `FetchError`,
`NoDataError`, `FDSNError`, `MixedSamplingRateError`.
- **Instrument response removal without obspy**
(`seisfetch/contrib/response.py`): evalresp-equivalent evaluator
(1.6e-10 vs compiled evalresp, including its conditional A0 rule),
obspy-`remove_response` port (machine precision on real data, ~2x
faster), SeisIO.jl-style translation, stdlib StationXML parsing, loud
failures on defective metadata.
- **NoisePy adapter** (`seisfetch/contrib/noisepy_adapter.py`): numpy
ports of NoisePy's `rm_resp=NO` preprocessing chain, validated exact
against obspy; end-to-end CCFs through real NoisePy are bit-identical.
- **Operations hardening**: adaptive retries, connect/read timeouts,
pool sized to fan-out, one shared executor per client, paginated
listings, EarthScope credential refresh, `iter_bulk_raw` streaming.
- **Benchmarks with committed results** (`benchmarks/results/*.json`,
`RESULTS.md`), docker machine matrix (Fargate/Lambda-class), CI
(offline pytest matrix + lint), tutorial notebook
`notebooks/05_response_removal.ipynb`.

### Fixed

- Silent gap concatenation in `to_dict`/`bundle_to_xarray` time axes.
- Non-deterministic multi-day/multi-provider byte order (`as_completed`
joins → submission order).
- Contained-segment crash in `to_dict(fill_value=)`.
- Out-of-order contiguous records now heal identically on the fast and
fallback parse paths; truncated buffers warn about unparsed bytes.
- BG (The Geysers) routed to NCEDC (was SCEDC).
- Timezone-offset timestamps in response epoch selection.

## 0.2.0 — 2026-04-27

Initial public state: S3 clients for EarthScope/SCEDC/NCEDC, FDSN HTTP
client, pymseed-based `parse_mseed`, bulk engine, xarray/zarr/obspy
exporters, Earth2Studio adapters, CLI.
13 changes: 13 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Keep the sdist lean: source + licenses + docs metadata only.
# Tests carry an 11 MB real-data fixture and notebooks carry executed
# outputs — they live in the repo, not the distribution.
include LICENSE THIRD_PARTY_NOTICES.md CHANGELOG.md README.md
recursive-include seisfetch *.py
prune tests
prune benchmarks
prune notebooks
prune tools
prune docs
prune .github
prune .jupyter-kernels
exclude .pre-commit-config.yaml pixi.lock environment.yml .gitignore
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -592,6 +592,11 @@ pixi run test-int

See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for attribution and licenses.

Benchmarks with plots: [benchmarks/RESULTS.md](benchmarks/RESULTS.md) (or the
self-contained [RESULTS.html](benchmarks/RESULTS.html)). Changes:
[CHANGELOG.md](CHANGELOG.md). Response-removal tutorial:
[notebooks/05_response_removal.ipynb](notebooks/05_response_removal.ipynb).

## Citation

When using data accessed through `seisfetch`:
Expand All @@ -609,4 +614,10 @@ Software references:

## License

MIT. See [LICENSE](LICENSE).
`MIT AND LGPL-3.0-only`. The package is MIT ([LICENSE](LICENSE)) with one
exception: [`seisfetch/contrib/obspy_ports.py`](seisfetch/contrib/obspy_ports.py)
contains numerically exact translations of ObsPy routines and is
LGPL-3.0-only (ObsPy is © The ObsPy Development Team, LGPL v3). Using
seisfetch as a library is unaffected; redistributors of modified versions of
that one file must honor LGPL terms. Details in
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
27 changes: 27 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,30 @@ decoding) draws on the architecture established by:

- **Clements & Denolle (2019)** — *Cactus to Clouds: Processing The SCEDC
Open Data Set on AWS.* 2019 SCEC Annual Meeting.

---

## Derived code (license-bearing)

- **`seisfetch/contrib/obspy_ports.py` — LGPL-3.0-only.** Contains Python
translations of ObsPy routines (`Trace.resample`, `Trace.taper`,
`obspy.signal.invsim.cosine_taper` / `cosine_sac_taper` /
`invert_spectrum`, `obspy.signal.util._npts2nfft`), preserving their exact
numerical behavior including float-operation order. ObsPy is
Copyright (C) The ObsPy Development Team, licensed under the GNU Lesser
General Public License v3 (https://github.com/obspy/obspy). These
translations are works based on the Library and are distributed under
LGPL-3.0-only; the file carries an SPDX header. The project license
expression is therefore `MIT AND LGPL-3.0-only`.
- **Parts of `seisfetch/contrib/noisepy_adapter.py` — MIT.** The
`check_sample_gaps_np` / `segment_interpolate_np` / `preprocess_raw_np`
chain reimplements the preprocessing semantics of NoisePy
(`noisepy-seis`, MIT, Copyright (c) Marine Denolle & Chengxin Jiang;
https://github.com/noisepy/NoisePy).
- **Parts of `seisfetch/contrib/response.py` — MIT.**
`translate_resp_np` and `damped_oscillator_response` reimplement the
response-translation approach of SeisIO.jl (`translate_resp!`,
`fctoresp`; MIT, https://github.com/jpjones76/SeisIO.jl). The
evalresp-equivalent evaluator in the same file was built by black-box
empirical verification against evalresp's observed behavior (clean-room;
no evalresp or ObsPy source translated).
Loading
Loading