diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 7f079c5..c27c741 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -41,7 +41,7 @@ jobs: shell: bash -l {0} run: | set -e - micromamba run -n oceanarray jupyter nbconvert --to notebook --execute notebooks/demo_mooring.ipynb --output=demo_mooring-output.ipynb --ExecutePreprocessor.allow_errors=True + micromamba run -n oceanarray jupyter nbconvert --to notebook --execute notebooks/demo_mooring.ipynb --output=demo_mooring-output.ipynb mv notebooks/*output.ipynb docs/source/ pushd docs micromamba run -n oceanarray make clean html diff --git a/.github/workflows/docs_deploy.yml b/.github/workflows/docs_deploy.yml index fb4da87..b6e6892 100644 --- a/.github/workflows/docs_deploy.yml +++ b/.github/workflows/docs_deploy.yml @@ -40,7 +40,7 @@ jobs: shell: bash -l {0} run: | set -e - micromamba run -n oceanarray jupyter nbconvert --to notebook --execute notebooks/demo_mooring.ipynb --output=demo_mooring-output.ipynb --ExecutePreprocessor.allow_errors=True + micromamba run -n oceanarray jupyter nbconvert --to notebook --execute notebooks/demo_mooring.ipynb --output=demo_mooring-output.ipynb mv notebooks/*output.ipynb docs/source/ pushd docs micromamba run -n oceanarray make clean html diff --git a/.github/workflows/pypi.yml b/.github/workflows/pypi.yml index ee4d176..1add1d0 100644 --- a/.github/workflows/pypi.yml +++ b/.github/workflows/pypi.yml @@ -37,7 +37,7 @@ jobs: - name: Set up Python uses: actions/setup-python@v7 with: - python-version: "3.x" + python-version: "3.13" - name: Install build tools run: | diff --git a/CHANGELOG.md b/CHANGELOG.md index e3a77d2..a2e583e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,8 +6,16 @@ All notable changes to oceanarray are documented here. ## [Unreleased] +## [0.5.0] — 2026-10-12 + +### Added + +- **Serial-uniqueness validation** ([#101](https://github.com/ocean-uhh/oceanarray/pull/101)): `validate` refuses duplicate serials within a mooring, a blank serial, and the reserved names `stack`/`grid` (which would collide the per-instrument report filenames `{mooring}_{serial}_report.html`). A trailing `*` is stripped before every check — `1234` and `1234*` on one mooring are a duplicate, a serial that is only `*` is blank — and stays a diagram-only "pay attention to this sensor" annotation, never carried into filenames or nav labels. + ### Changed +- **Report NetCDF-inventory appendix restyled** ([#103](https://github.com/ocean-uhh/oceanarray/pull/103)): the instrument, stack and grid reports share one inventory appendix — a Coordinates table (one row per coordinate variable: type, units, standard/long name, min/max, N valid, attributes), variables grouped by dimension set, QC companions shown inline as a green `(_QC)`, and global attributes split into named groups with a Type column; the per-page variable/scalar/globals templates collapse to one shared set, and the grid page gains a coordinates table it never had. The dune2 test fixture units were canonicalised in place (14 attribute edits, data byte-identical) and the hosted dune2 example suite regenerated. `demo_mooring.ipynb` is ported to `oceanarray.processors.process` and runs on the committed fixture from a clean checkout; `demo_step1.ipynb` is removed, and five old-API demo notebooks move to `notebooks/legacy/`. +- **Shared masthead nav** ([#100](https://github.com/ocean-uhh/oceanarray/pull/100)): the report masthead navigation is rebuilt from the shared vendored macro, and the six report-design files are re-vendored from ctdcast with a vendoring guard (`scripts/check_vendored.py`). - **Canonical units at every stage** ([#102](https://github.com/ocean-uhh/oceanarray/pull/102)): one canonical spelling per quantity is now written to every file — `degree_Celsius`, `mS cm-1`, `dbar`, `m`, `m s-1`, `umol kg-1`, `g kg-1`, `kg m-3`, `1` (practical salinity and turbidity). A registry-scoped pass (`oceanarray.processors.units.canonicalise_units`) runs on the way out of every stage, rewriting the reader spellings (`degC`, `mS/cm`, `meters`, `db`, `S/m`→×10) to the canonical and recording any value conversion in `units_converted_from` / `units_conversion_factor`. Temperature variables carry CF-1.11 `units_metadata = "temperature: on_scale"`. Plot labels are derived from the canonical string by `format_units` (slash form: `mS/cm`, `m/s`, `kg/m³`, `µmol/kg`, `°C`, `PSU`, `NTU`). - **Single netCDF writer** ([#98](https://github.com/ocean-uhh/oceanarray/pull/98)): the six write paths the processing stages used before now go through one `oceanarray.writers.write` seam. The files that were written uncompressed — the seasenselib-written stage files, whose compression parameters its writer silently discarded, and the bare-`to_netcdf` stage3 output — shrink 58–66% on the dune2 fixtures. The two that were already zlib level-5 compressed — the pressure-gridded and stacked outputs — grow about 1% as they move to the common level-4 setting. Decoded values, dtypes, attributes, and time encoding are unchanged; raw values are byte-identical, verified per stage against the old writer. The private `_get_netcdf_writer_params` methods are removed, including the `quantize` key, which was never a valid netCDF4 encoding parameter and never took effect. @@ -27,6 +35,11 @@ All notable changes to oceanarray are documented here. - **`None` attributes dropped, not blanked** ([#99](https://github.com/ocean-uhh/oceanarray/pull/99)): a `None`-valued attribute (global or per-variable) is now dropped with a warning instead of written as `""`. `UNK` belongs where a value is determined, not the write seam. On the dune2 fixtures no attribute is affected (0/84 globals are empty); this is a guardrail. - **Time encoding pinned** ([#99](https://github.com/ocean-uhh/oceanarray/pull/99)): every datetime variable is written as `seconds since 1970-01-01` (float64, `proleptic_gregorian`) rather than xarray's per-file guess. Decoded instants change by at most 0.13 µs (float64 seconds have ~238 ns spacing at the 2026 epoch, so ≤119 ns rounding; ~0.5 µs by 2050) — negligible against clock drift, which is corrected in whole seconds. **Raw stored values change**, so a reader using `decode_times=False`/`decode_cf=False` sees different numbers. This also removes a pandas-version dependence in the units string (space vs `T`). +### Known issues + +- **Empty-units turbidity files**: a registered variable with an empty `units` fails that one instrument at stage 1 — the handler logs `ERROR: : `, skips the instrument, and the mooring completes with the others and a non-zero exit (the family rule). Six turbidity files on the drive are affected; the reader fix lands in 0.5.1. +- **dune2 test fixture stack/grid**: the committed `_stack.nc`/`_grid.nc` fixtures reference instruments not present in the committed raw and YAML — they were built from the full mooring before the fixture was trimmed to two instruments — so running `oceanarray run` on the fixture collapses the stack to those two. A known fixture inconsistency, rebuilt in 0.5.1. + ## [0.4.0] — 2026-09-08 ### Added diff --git a/CITATION.cff b/CITATION.cff index b43d62f..f99de29 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -7,8 +7,8 @@ authors: affiliation: University of Hamburg, Bundesstraße 53, 20146 Hamburg, Germany title: oceanarray doi: 10.5281/zenodo.21890564 -version: 0.4.0 -date-released: 2026-09-08 +version: 0.5.0 +date-released: 2026-10-12 url: https://github.com/ocean-uhh/oceanarray repository-code: https://github.com/ocean-uhh/oceanarray license: MIT diff --git a/MIGRATION-BASEDIR.md b/MIGRATION-BASEDIR.md deleted file mode 100644 index 350ec9c..0000000 --- a/MIGRATION-BASEDIR.md +++ /dev/null @@ -1,188 +0,0 @@ -# Migrating from --basedir to --raw-dir / --proc-dir - -The `--basedir` flag has been **removed**. All subcommands now require -`--raw-dir` and `--proc-dir` instead. Passing `--basedir` prints a migration -message naming the replacement and exits without processing. This document -explains how to convert an old `--basedir` layout to the current one. - ---- - -## New canonical directory layout - -``` -{raw_dir}/ # --raw-dir (cruise level) - {mooring}/ - {mooring}.mooring.yaml - {instrument}/ - raw_file.asc - raw_file.aqd - ... - -{proc_dir}/ # --proc-dir (cruise level) - {mooring}/ - {mooring}.mooring.yaml # copy or symlink from raw_dir - {instrument}/ - {mooring}_{serial}_stage1.nc - {mooring}_{serial}_stage2.nc - {mooring}_{serial}_stage3.nc - {mooring}_stack.nc - {mooring}_grid.nc - processing_logs/ - {mooring}_{timestamp}_stage1.mooring.log - ... - report/ - {mooring}_report.html - {mooring}_stack_report.html - {mooring}_grid_report.html - instrument/ - {mooring}_{serial}_report.html -``` - -### Concrete example - -``` -/Volumes/T9/odb2026/raw/ # --raw-dir - dsG3_1_2026/ - dsG3_1_2026.mooring.yaml - microcat/ - 26261_recovery.asc - 26262_recovery.asc - aquadopp/ - A400118_dsG3.aqd - A400118_dsG3.hdr - -/local/proc/ # --proc-dir - dsG3_1_2026/ - dsG3_1_2026.mooring.yaml # place a copy here before running - microcat/ - dsG3_1_2026_26261_stage1.nc - dsG3_1_2026_26261_stage2.nc - dsG3_1_2026_26261_stage3.nc - aquadopp/ - dsG3_1_2026_400118_stage1.nc - dsG3_1_2026_stack.nc - dsG3_1_2026_grid.nc - report/ - dsG3_1_2026_report.html - instrument/ - dsG3_1_2026_26261_report.html -``` - ---- - -## New shell script pattern - -```bash -#!/usr/bin/env bash -RAW=/Volumes/T9/odb2026/raw -PROC=/local/proc -MOORING=dsG3_1_2026 - -oceanarray run "$MOORING" \ - --raw-dir "$RAW" \ - --proc-dir "$PROC" \ - --dp 10 \ - --force -``` - -Individual steps: - -```bash -oceanarray process dsG3_1_2026 --raw-dir $RAW --proc-dir $PROC --stage 1 2 3 stack grid --dp 10 -oceanarray report dsG3_1_2026 --raw-dir $RAW --proc-dir $PROC --instruments --stack --grid -``` - ---- - -## Migration steps - -### Step 1: reorganise your raw files (mooring-first layout) - -``` -# Old instrument-first layout: -raw/{instrument}/{mooring}/filename - -# New mooring-first layout: -raw/{mooring}/{instrument}/filename -``` - -Example: - -```bash -OLD_RAW=/Volumes/T9/odb2026 -NEW_RAW=/Volumes/T9/odb2026/raw - -mkdir -p $NEW_RAW/dsG3_1_2026/microcat -mv $OLD_RAW/microcat/dsG3_1_2026/*.asc $NEW_RAW/dsG3_1_2026/microcat/ - -mkdir -p $NEW_RAW/dsG3_1_2026/aquadopp -mv $OLD_RAW/aquadopp/dsG3_1_2026/*.aqd $NEW_RAW/dsG3_1_2026/aquadopp/ -``` - -### Step 2: place the YAML in the proc directory - -The YAML must be at `{proc_dir}/{mooring}/{mooring}.mooring.yaml` -(same location as today under `--basedir`). - -```bash -PROC=/local/proc -MOORING=dsG3_1_2026 - -cp $NEW_RAW/$MOORING/$MOORING.mooring.yaml $PROC/$MOORING/ -``` - -### Step 3: remove `directory:` from the YAML (or keep as override) - -If your YAML has a `directory:` key, you can remove it — `--raw-dir` -takes over. If you leave it in and it is an absolute path, it acts as -an override for the raw root for that mooring (and a WARNING is printed). - -### Step 4: update your shell scripts - -```bash -# Before -oceanarray run $MOORING --basedir $DATA - -# After -oceanarray run $MOORING --raw-dir $RAW --proc-dir $PROC -``` - ---- - -## What `--basedir` used to mean - -The old `--basedir DIR` resolved paths as: - -| File | Old path | -|------|----------| -| YAML | `DIR/proc/{mooring}/{mooring}.mooring.yaml` | -| Raw | `DIR/{directory}/{instrument}/{mooring}/filename` | -| NC output | `DIR/proc/{mooring}/{instrument}/` | -| Reports | `DIR/proc/{mooring}/report/` | -| Logs | `DIR/proc/{mooring}/` (flat) | - -The new `--raw-dir`/`--proc-dir` flags give you explicit control over -each root, support mooring-first raw layout, and put logs in a -`processing_logs/` subdirectory. - ---- - -## If you call the Python API directly (notebooks / scripts) - -The same removal touched the Python entry points: - -- **Processor constructors are keyword-only and required.** Replace - `MooringProcessor(basedir)` with - `MooringProcessor(raw_dir=RAW, proc_dir=PROC)`, and - `Stage2Processor(basedir)` / `Stage3Processor` / `MooringStacker` / - `MooringGridder` / `MooringReport` / `TimeGriddingProcessor` with - `proc_dir=PROC` (add `raw_dir=RAW` for stage 1 and the report). A positional - or `base_dir=` call now raises `TypeError`. -- **Removed convenience functions:** `stage1_mooring`, - `process_multiple_moorings`, `stage2_mooring`, - `process_multiple_moorings_stage2`. Use the `oceanarray` CLI, or loop over - moorings yourself constructing the processor directly. -- **`time_gridding_mooring(mooring, proc_dir, ...)`** and - `process_multiple_moorings_time_gridding` — the second positional argument is - now the **proc root**, not a cruise base dir. Update call sites; a positional - `basedir` value will be silently misused as `proc_dir`. diff --git a/docs/source/methods/concatenation.rst b/docs/source/methods/concatenation.rst index af45d73..ef74049 100644 --- a/docs/source/methods/concatenation.rst +++ b/docs/source/methods/concatenation.rst @@ -6,7 +6,7 @@ into a single continuous time series on a regular time-pressure grid. Purpose ------- -- Interpolate individual deployment periods onto a unified pressure grid (e.g., every 20 dbar from 0–4820 dbar). +- Interpolate individual deployment periods onto a common pressure grid (e.g., every 20 dbar from 0–4820 dbar). - Construct a consistent time axis (e.g., half-daily `jg` from April 1, 2004 to October 11, 2015). - Interpolate all segments onto this unified time grid, filling short gaps between deployments. - Output gridded `.mat` files for later use in AMOC transport code. diff --git a/docs/source/methods/standardisation.rst b/docs/source/methods/standardisation.rst index a097164..843a247 100644 --- a/docs/source/methods/standardisation.rst +++ b/docs/source/methods/standardisation.rst @@ -155,7 +155,7 @@ The standardised output is a **raw-equivalent** `xarray.Dataset` with: - **Dimensions**: `time` (primary coordinate) - **Variables**: Standardised names (e.g., `temperature`, `salinity`, `pressure`) - **Coordinates**: `time`, plus instrument metadata variables -- **Attributes**: Comprehensive metadata including instrument, mooring, and deployment information +- **Attributes**: Metadata including instrument, mooring, and deployment information Example output structure: @@ -186,7 +186,7 @@ Example output structure: 5. Quality Control and Error Handling -------------------------------------- -The standardisation process includes robust error handling: +The standardisation process includes error handling: - **Missing Files**: Graceful handling with detailed logging - **Format Errors**: Reader-specific error catching and reporting @@ -217,7 +217,7 @@ This standardisation step evolved from the RAPID programme's RDB format, which p 2021 01 05 21.00056 3.9389 33.2389 1725.4 2021 01 05 22.00056 3.9389 33.2405 1725.3 -The modern NetCDF-based approach provides several advantages over the historical RDB format: +The NetCDF-based approach provides several advantages over the historical RDB format: - **Self-describing metadata**: CF-compliant attributes and coordinate information - **Efficient storage**: Compression and chunking for large datasets @@ -239,9 +239,9 @@ This script performed similar functions to the current Python implementation: :linenos: :caption: Excerpt from `microcat2rodb_3.m` -The modern Python implementation in :mod:`oceanarray.processors.stage1` provides equivalent functionality with improved: +The Python implementation in :mod:`oceanarray.processors.stage1` provides equivalent functionality with improved: -- **Error handling**: Comprehensive logging and graceful failure modes +- **Error handling**: Logging and graceful failure modes - **Format support**: Multiple instrument types through pluggable readers - **Metadata management**: YAML-based configuration with validation - **Output formats**: NetCDF with CF conventions for interoperability diff --git a/docs/source/migration.rst b/docs/source/migration.rst index f55edf6..3f164fe 100644 --- a/docs/source/migration.rst +++ b/docs/source/migration.rst @@ -4,6 +4,32 @@ Migrating from the ``--basedir`` Layout ========================================= +Upgrading from 0.4 to 0.5 +------------------------- + +0.5.0 is a breaking release. If you are upgrading an existing install: + +- **Reprocess from stage 1 for files you distribute.** Old stage files still + load — the canonical-units pass rewrites the reader spellings (``degC`` → + ``degree_Celsius``, ``mS/cm`` → ``mS cm-1``, …) on the way in and logs one line + per variable — but files you *share* should be regenerated so they carry the + canonical strings + (`#102 `__). +- **``save_dataset`` / ``save_OS_instrument`` are gone.** Call + ``oceanarray.writers.write`` instead + (`#99 `__). +- **Time encoding is pinned** to ``seconds since 1970-01-01`` (float64). + Re-encoding introduces at most 0.13 µs of rounding in the decoded instants + (≤ 119 ns at the 2026 epoch), but the on-disk representation changes, so a + reader using ``decode_times=False`` / ``decode_cf=False`` sees different raw + units and numbers + (`#99 `__). +- **``oceanarray process`` runs stages 1–3 by default.** Pass ``--stage 1 2`` + for the old raw→trim-only behaviour + (`#102 `__). + +---- + This page describes how to move from the older instrument-first raw directory layout (used with the ``--basedir`` flag) to the current mooring-first layout (used with ``--raw-dir`` and ``--proc-dir``). diff --git a/oceanarray/cli.py b/oceanarray/cli.py index b89bd45..e0896e1 100644 --- a/oceanarray/cli.py +++ b/oceanarray/cli.py @@ -38,8 +38,8 @@ def _parse_dirs( raise SystemExit( "ERROR: --basedir has been removed. Use --raw-dir + --proc-dir " "(the mooring directory is now /, not " - "/moor/proc/). See MIGRATION-BASEDIR.md at the " - "repository root." + "/moor/proc/). See the migration guide: " + f"docs/source/migration.rst or {paths.DOCS_URL}migration.html" ) if not proc_dir: diff --git a/oceanarray/config/parameters.py b/oceanarray/config/parameters.py index 7f84e55..f86ac14 100644 --- a/oceanarray/config/parameters.py +++ b/oceanarray/config/parameters.py @@ -173,6 +173,10 @@ "fail_span": (0.0, 500.0), "suspect_span": (0.0, 450.0), }, # umol kg-1 (per-mass numbers differ from per-volume by ~2.5%, noise for a gross-range span) + "dissolved_oxygen_2": { # second per-mass oxygen sensor — same span as dissolved_oxygen + "fail_span": (0.0, 500.0), + "suspect_span": (0.0, 450.0), + }, # umol kg-1 # O2 % saturation: >200 % would indicate severe bubble entrainment or sensor fault. "oxygen_saturation_pct": {"fail_span": (0.0, 200.0), "suspect_span": (0.0, 150.0)}, } @@ -508,6 +512,15 @@ "standard_name": "moles_of_oxygen_per_unit_mass_in_sea_water", "cmap": "RdYlGn", }, + "dissolved_oxygen_2": { + # Second per-mass oxygen sensor (SBE ``sbeox1Mm/Kg``); same as + # dissolved_oxygen, registered so the units pass canonicalises it. + "label": "Dissolved oxygen (sensor 2)", + "label_units": "µmol/kg", + "units": "umol kg-1", + "standard_name": "moles_of_oxygen_per_unit_mass_in_sea_water", + "cmap": "RdYlGn", + }, "dissolved_oxygen_ml_l": { "label": "Dissolved oxygen", "label_units": "mL/L", @@ -518,6 +531,16 @@ "standard_name": None, "cmap": "RdYlGn", }, + "dissolved_oxygen_ml_l_2": { + # Second oxygen sensor's mL/L reading (SBE ``sbeox1ML/L``); same units as + # dissolved_oxygen_ml_l, registered so the units pass canonicalises it + # instead of leaving it untouched. + "label": "Dissolved oxygen (sensor 2)", + "label_units": "mL/L", + "units": "mL L-1", + "standard_name": None, + "cmap": "RdYlGn", + }, "oxygen_saturation_pct": { "label": "O₂ percent saturation", "label_units": "%", @@ -603,7 +626,9 @@ "speed": "#D55E00", # Biogeochemistry — Paul Tol "dissolved_oxygen": "#332288", # indigo + "dissolved_oxygen_2": "#332288", "dissolved_oxygen_ml_l": "#332288", + "dissolved_oxygen_ml_l_2": "#332288", "oxygen_saturation_pct": "#332288", "turbidity": "#661100", # dark red } diff --git a/oceanarray/paths.py b/oceanarray/paths.py index 55dfa49..6edc6af 100644 --- a/oceanarray/paths.py +++ b/oceanarray/paths.py @@ -13,6 +13,9 @@ _PathLike = str | Path +#: Base URL of the hosted documentation, used in user-facing upgrade messages. +DOCS_URL = "https://ocean-uhh.github.io/oceanarray/" + class LegacyLayoutError(RuntimeError): """Raised when a directory uses the removed ``moor/proc`` (``basedir``) layout.""" @@ -215,7 +218,8 @@ def require_current_layout(proc_root: _PathLike, mooring: str) -> None: raise LegacyLayoutError( f"--proc-dir points at an old-layout directory (found " f"'{legacy_root / mooring}'). Use --proc-dir {legacy_root} " - f"instead (see MIGRATION-BASEDIR.md)." + "instead (see the migration guide: docs/source/migration.rst " + f"or {DOCS_URL}migration.html)." ) diff --git a/oceanarray/reports/_array.py b/oceanarray/reports/_array.py index 28443ee..1f54adb 100644 --- a/oceanarray/reports/_array.py +++ b/oceanarray/reports/_array.py @@ -10,6 +10,7 @@ from __future__ import annotations import warnings +from datetime import UTC from pathlib import Path from typing import Any @@ -342,7 +343,7 @@ def generate_array_report( Path to the written HTML file, or None on failure. """ - from datetime import datetime, timezone + from datetime import datetime array_yaml_path = Path(array_yaml_path) proc_dir = Path(proc_dir) @@ -472,7 +473,7 @@ def generate_array_report( "fig_map_b64": fig_map_b64, "type_summary": _build_type_summary(all_instruments), "mooring_completeness": mooring_completeness, - "generated": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC"), + "generated": datetime.now(UTC).strftime("%Y-%m-%d %H:%M UTC"), } html = render_template("array.html", **ctx) diff --git a/oceanarray/reports/_html_helpers.py b/oceanarray/reports/_html_helpers.py index 34fa9ba..7b0204b 100644 --- a/oceanarray/reports/_html_helpers.py +++ b/oceanarray/reports/_html_helpers.py @@ -879,7 +879,7 @@ def _file_info(path: Path) -> dict[str, str]: import datetime as _dt st = path.stat() - mtime = _dt.datetime.fromtimestamp(st.st_mtime, tz=_dt.timezone.utc) + mtime = _dt.datetime.fromtimestamp(st.st_mtime, tz=_dt.UTC) return { "exists": True, "name": path.name, diff --git a/oceanarray/reports/_instrument.py b/oceanarray/reports/_instrument.py index 9cdc733..f3644b0 100644 --- a/oceanarray/reports/_instrument.py +++ b/oceanarray/reports/_instrument.py @@ -4,7 +4,7 @@ import socket from collections.abc import Callable -from datetime import datetime, timezone +from datetime import UTC, datetime from pathlib import Path from typing import Any @@ -392,7 +392,7 @@ def generate_instrument_pages( """ mooring_report_link = f"../{mooring_name}_report.html" - generated = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") + generated = datetime.now(UTC).strftime("%Y-%m-%d %H:%M UTC") proc_machine = socket.gethostname().split(".")[0] stack_exists = (proc_dir / f"{mooring_name}_stack.nc").exists() grid_exists = (proc_dir / f"{mooring_name}_grid.nc").exists() diff --git a/oceanarray/reports/_mooring.py b/oceanarray/reports/_mooring.py index 3f0e8a9..197ec39 100644 --- a/oceanarray/reports/_mooring.py +++ b/oceanarray/reports/_mooring.py @@ -4,7 +4,7 @@ import contextlib import socket -from datetime import datetime, timezone +from datetime import UTC, datetime from pathlib import Path from typing import Any @@ -850,7 +850,7 @@ def _combined(deploy_key: str, recover_key: str, legacy_key: str) -> str: current="summary", array_href=_find_array_report_href(out_dir), ), - "generated": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC"), + "generated": datetime.now(UTC).strftime("%Y-%m-%d %H:%M UTC"), "proc_machine": socket.gethostname().split(".")[0], "yaml_path": self._rel(yaml_path), "diagram_b64": _load_pdf_b64(_resolve_diagram_pdf(proc_dir, mooring_name)), diff --git a/oceanarray/reports/_recovery_table.py b/oceanarray/reports/_recovery_table.py index 4944c72..0a234c8 100644 --- a/oceanarray/reports/_recovery_table.py +++ b/oceanarray/reports/_recovery_table.py @@ -12,6 +12,7 @@ from __future__ import annotations +from datetime import UTC from pathlib import Path from typing import Any @@ -292,7 +293,7 @@ def generate_recovery_table( Path to the written HTML file, or ``None`` on failure. """ - from datetime import datetime, timezone + from datetime import datetime from ..utilities import format_latlon, parse_latlon_with_source from ._html_helpers import _duration_str @@ -343,7 +344,7 @@ def generate_recovery_table( "duration": _duration_str(deploy_dt, recover_dt), "rows": rows, "comments": comments, - "generated": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC"), + "generated": datetime.now(UTC).strftime("%Y-%m-%d %H:%M UTC"), } html = render_template("recovery_table.html", **ctx) diff --git a/pyproject.toml b/pyproject.toml index e87f4c6..dfa5e2c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -148,7 +148,7 @@ omit = [ [tool.ruff] line-length = 88 -target-version = "py310" +target-version = "py311" exclude = [ "oceanarray/legacy", # legacy modules — not maintained to current standards "notebooks", # notebooks are exploratory; not held to production lint standards diff --git a/tests/unit/test_stage1.py b/tests/unit/test_stage1.py index a792a3b..17de20a 100644 --- a/tests/unit/test_stage1.py +++ b/tests/unit/test_stage1.py @@ -229,6 +229,36 @@ def test_sanitize_slash_vars_ml_l_oxygen_to_ml_l_variable(self, processor): assert "dissolved_oxygen_ml_l" in out assert "dissolved_oxygen" not in out # not the per-mass variable + def test_sanitize_slash_vars_second_oxygen_sensors_registered(self, processor): + """Both second-sensor oxygen columns map to registered canonical variables. + + ``sbeox1ML/L`` → ``dissolved_oxygen_ml_l_2`` (``mL L-1``) and ``sbeox1Mm/Kg`` + → ``dissolved_oxygen_2`` (``umol kg-1``). Both must be registered so the + units pass canonicalises them rather than leaving the second sensor + untouched; the per-mass ``dissolved_oxygen`` would reject a mL/L value. + """ + from oceanarray.config import parameters as params + + out = processor._sanitize_slash_vars( + xr.Dataset( + {"sbeox1ML/L": (["time"], np.array([5.3, 5.4]))}, + coords={"time": (["time"], np.arange(2))}, + ) + ) + assert "sbeox1ML/L" not in out + assert "dissolved_oxygen_ml_l_2" in out + assert params.VARIABLES["dissolved_oxygen_ml_l_2"]["units"] == "mL L-1" + + out = processor._sanitize_slash_vars( + xr.Dataset( + {"sbeox1Mm/Kg": (["time"], np.array([250.0, 260.0]))}, + coords={"time": (["time"], np.arange(2))}, + ) + ) + assert "sbeox1Mm/Kg" not in out + assert "dissolved_oxygen_2" in out + assert params.VARIABLES["dissolved_oxygen_2"]["units"] == "umol kg-1" + def test_clean_dataset_variables_unknown_type(self, processor): """Test cleaning dataset variables for unknown file type.""" ds = xr.Dataset(