diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2f1e27f..adb845f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,54 @@ this file is about the package, whose version moves independently.
## Unreleased
+- **Breaking: the profile axis is milliseconds.** §5.1 of
+ [the specification](https://github.com/divejson/divejson/blob/main/spec/divejson.md) puts a
+ Series' `times`, a profile's `duration` and an event's `time` in milliseconds, where a
+ dive's own `duration` and the `ndl` and `tts` readings stay seconds. Every reader multiplies
+ its source's seconds by a thousand before rounding, so a fraction the source states keeps
+ its place — a Suunto app export's first depth lands at 160 rather than 0 — and two readings
+ collide only on one millisecond: `30` and `30.4`
+ are two samples now. The members keep their names and types, so a document written in
+ seconds still validates and reads a thousand times short; this package's converters wrote
+ every such document up to this release. The UDDF writer puts `` back in seconds
+ with a fraction only where the millisecond is not a whole second. Report lines still speak
+ seconds (`converter.in_seconds`), and `converter.milliseconds` is the one factor.
+
+- **Breaking: a device's readouts and its salinity sit on the recording.**
+ `surface_pressure`, `cns_start`, `cns_end`, `otu_start` and `otu_end` move from the dive to
+ §6.4a's recording, and `divejson validate` refuses them on a dive as undefined members.
+ Each reader puts them on the recording its file produced; a figure a source states once
+ for the whole dive — UDDF's ``, Subsurface's `@cns` and `@otu` — goes to
+ the primary recording, reported `resolved` where the dive has more than one, and is a
+ recording of its own where it has none, a readout now satisfying §3 rule 4 on its own. FIT's
+ `dive_settings.water_type` is the recording's `salinity`, and the dive's `water_type` loses
+ `en13319`. The UDDF writer takes `` from the primary recording and reports
+ a recording's salinity and oxygen clocks dropped. `converter.recording` takes `salinity` and
+ `readouts`; `converter.onto_primary` is the dive-level rule.
+
+- **Breaking: a dive may start on a date alone.** A UDDF `2002-06-18` or
+ `2002-06-18T`, and a `.ssrf` `@date` with no `@time`, read as the bare date with the time of
+ day reported absent, where they read as midnight. `divejson validate` accepts a date in a
+ dive's `started_at` and nowhere else. The UDDF writer writes it back as the bare date with
+ no report, which the UDDF XSD's `xs:dateTime` refuses; the suite's XSD pass widens that one
+ element's spelling for itself alone.
+
+- **Breaking: `po2_limit` is `ppo2_limit`**, the name of the quantity §6.4's `ppo2` channel
+ samples. The readers write the new name, the writer reads it, and `divejson validate`
+ refuses the old one as an undefined member.
+
+- **Breaking: `notes` has no length cap.** A converter carries a note whole, where it cut one
+ at 10 000 characters and reported the rest dropped; `converter.MAX_NOTES` is gone. The
+ strings that stay bounded are names, numbers, labels and identifiers.
+
+- **`divejson validate` no longer checks member order.** §4 makes `format` first and
+ `version` second a SHOULD, so a document a generic re-serialisation sorted is conforming.
+ §3's `gf_low ≤ gf_high` is rule 6, and the converter's report says so.
+
+- **The certification and course agencies gain twenty values**, AIDA among them, arriving
+ with the schema. §5.5 now reserves the `divejson` producer key for converters following
+ `docs/converting.md`, which is where this package's converters already write.
+
## 0.12.0
- **A diver carries a portrait.** §6.1 of
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 6c4698c..2793953 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -73,8 +73,13 @@ this arrangement exists to prevent:
- **Parse XML through `xmlsource.parse_xml`**, which refuses a `` before expat
expands anything (spec §9).
- **Build a profile through `series.SampleAxis`**, which owns the ordering, the sample with
- no time, the two samples on one second, and the dive whose samples carry nothing this
- format can hold. `noun` and `time_member` keep the report speaking the source's language.
+ no time, the two samples on one millisecond, and the dive whose samples carry nothing this
+ format can hold. Offer it `converter.milliseconds` of the seconds the source states, which
+ keeps a fraction the source wrote. `noun` and `time_member` keep the report speaking the
+ source's language.
+- **Put a device's readouts on its recording.** A figure the source states once for the
+ whole dive goes to the primary recording, reported where the dive has more than one —
+ `docs/converting.md`'s rule, which `converter.onto_primary` implements.
- **Ask `converter.recorded` which way a zero reads.** The member's own schema constraint
decides, and the answer differs between members that look alike.
- **Validate the document before returning it, unless `scope.validates_alone` is false.**
diff --git a/README.md b/README.md
index a875944..56dc4b1 100644
--- a/README.md
+++ b/README.md
@@ -30,9 +30,11 @@ divejson validate my-logbook.divejson
The JSON Schema, and then the requirements the specification states in prose and a schema
cannot — identifier uniqueness, referential closure, profile-series integrity in **every**
-recording of a dive, a recording carrying at least one of a device, a profile and its
-stored files, the member order, the UTC offset on `exported_at`. Exit status is non-zero if
-any file fails, with one line per violation.
+recording of a dive, a recording carrying at least one of a device, a profile, its stored
+files and a readout, the UTC offset on `exported_at`. The order of a document's members is
+not among them: §4 asks a writer to put `format` and `version` first, and a document that
+does not is as conforming as one that does. Exit status is non-zero if any file fails, with
+one line per violation.
## Convert a logbook into DiveJSON
@@ -228,8 +230,13 @@ directory now yields 384 dives where it used to yield 342.
**What a recording carries beyond its samples.** Every reader fills §6.4a's `mode` and
§6.4c's `deco_model` where its files state them — UDDF from `` and the
`` a dive links, FIT from `dive_settings`, the Suunto app's JSON from
-`Header.Diving`, the DM5 XML from `` and `` — and the profile carries
-the readouts the computer *computed*, as distinct from what it measured. Which channels
+`Header.Diving`, the DM5 XML from `` and `` — and §6.4a's readouts, the
+surface pressure and the two ends of the oxygen clocks the computer used, on the recording
+beside them rather than on the dive. A format that states a readout once for the whole dive
+— UDDF's ``, Subsurface's `@cns` and `@otu` — gives it to the primary
+recording, and reports that reading where the dive has more than one. The profile carries
+the figures the computer *computed* sample by sample, as distinct from what it measured, on
+an axis in milliseconds that keeps every sub-second offset a source states. Which channels
those are, and in what units, is
[§6.4 of the specification](https://github.com/divejson/divejson/blob/main/spec/divejson.md);
which of them a given format states, and what it does with a device's absent-markers and
diff --git a/SPEC_REF b/SPEC_REF
index b3b35b4..3fc45e9 100644
--- a/SPEC_REF
+++ b/SPEC_REF
@@ -1 +1 @@
-929788950d59e3b8130ccdbd27811e44aeb074ff
+cdf4e03684b86f6d07e296d692d2f229d7476bee
diff --git a/divejson/converter.py b/divejson/converter.py
index 8dcf5b7..970073c 100644
--- a/divejson/converter.py
+++ b/divejson/converter.py
@@ -10,7 +10,8 @@
The sections below the policy classes carry the rest of what an adapter inherits rather
than rewrites: `decimal_of` and its representability bound, `rounded`'s half-away-from-zero
-convention and the two §6.5 channel scales; `capped`; `Identities`; and `position`. An
+convention, the millisecond axis and the two §6.5 channel scales; `capped`; the recording
+and where a readout stated on the dive goes; `Identities`; and `position`. An
adapter that reimplements one of these gets it subtly different, which is the failure this
module exists to prevent — the bound and the Null Island rule were each written once for
one format and are true of every format.
@@ -75,10 +76,10 @@
from datetime import datetime
from decimal import ROUND_HALF_UP, Decimal, InvalidOperation
from functools import cache
-from typing import Any, Literal
+from typing import Any, Literal, overload
from . import SPEC_VERSION, __version__
-from .validate import Issue, load_schema
+from .validate import READOUTS, Issue, load_schema
__all__ = [
"CENTIMETRES_PER_METRE",
@@ -87,8 +88,9 @@
"MAX_MAGNITUDE",
"MAX_MODEL_NAME",
"MAX_NAME",
- "MAX_NOTES",
+ "MILLISECONDS_PER_SECOND",
"PRODUCER_KEY",
+ "READOUTS",
"TENTHS_PER_UNIT",
"Claimed",
"Conversion",
@@ -112,13 +114,18 @@
"device",
"grouped",
"header",
+ "in_seconds",
"integer_of",
+ "milliseconds",
+ "onto_primary",
"position",
"profile_members",
"record_inferred",
"recorded",
"recording",
+ "recording_members",
"rounded",
+ "shared_readout",
"zero_is_an_answer",
]
@@ -318,7 +325,8 @@ def validates_alone(self) -> bool:
def header(exported_at: datetime) -> dict[str, Any]:
"""The four members every converted document opens with, in §4's order.
- `format` and `version` are the two §4 requires first and second. `exported_at` and
+ `format` and `version` are the two §4 asks a writer to put first and second, so that a
+ reader may dispatch before parsing further. `exported_at` and
`generator` are the two a converted document asserts about its own *run* rather than
about the source, which is why `divejson.compared` drops exactly those before a
fixture comparison.
@@ -394,9 +402,9 @@ def channel_floor(member: str) -> int | None:
A channel is a `$ref` to a series definition rather than a member carrying constraints
of its own, so `recorded` is not the question to ask about one — every call site it has
- passes `record="dive"` or `record="cylinder"`, and `_floor` finds nothing under a
- reference. This resolves the reference instead: the six decompression readouts share a
- definition whose `values` floor at zero, because no-decompression time, time to surface,
+ names a record's own member, and `_floor` finds nothing under a reference. This
+ resolves the reference instead: the six decompression readouts share a definition whose
+ `values` floor at zero, because no-decompression time, time to surface,
ppO₂, CNS and a gradient factor have no negative reading and a source that writes one is
spelling absence in the only space it had. Depth, ceiling and temperature share the
signed definition and floor at nothing.
@@ -470,10 +478,18 @@ def recorded(value: Decimal | int | None, *, record: str, member: str) -> bool:
# than zero.
#
# Divided by 1000, the largest factor any adapter applies to a number it has read — UDDF's
-# cubic metres to litres, and above any channel scale — so that checking the value as the
-# text is read also covers every value derived from it.
+# cubic metres to litres, every source's seconds to the profile axis's milliseconds, and
+# above any channel scale — so that checking the value as the text is read also covers
+# every value derived from it. A reader that multiplies before that factor, as `.ssrf`'s
+# `M:SS` does minutes into seconds, checks its own product against this bound too.
MAX_MAGNITUDE = Decimal(sys.float_info.max) / 1000
+# §5.1's grain for elapsed time on a profile axis — a Series' `times`, a profile's
+# `duration`, an event's `time` — where a dive's own `duration` and the `ndl` and `tts`
+# readings stay seconds. Every source this package reads states its sample times in
+# seconds, whole or fractional, so this is the one factor every reader's axis applies.
+MILLISECONDS_PER_SECOND = Decimal(1000)
+
def decimal_of(text: str | None) -> Decimal | None:
"""A number from source text, or `None` for anything that is not a usable one.
@@ -511,7 +527,7 @@ def rounded(value: Decimal) -> int:
"""The nearest integer, halves away from zero.
Python's own `round` is half-to-even, which is the right default for statistics and the
- wrong one for a reading: 2.5 seconds of elapsed time is 3, not 2.
+ wrong one for a reading: a sample stamped 2.5 ms in is at 3 ms, not 2.
"""
return int(value.to_integral_value(rounding=ROUND_HALF_UP))
@@ -520,13 +536,40 @@ def integer_of(value: Decimal | None) -> int | None:
return None if value is None else rounded(value)
+@overload
+def milliseconds(seconds: Decimal) -> int: ...
+@overload
+def milliseconds(seconds: None) -> None: ...
+@overload
+def milliseconds(seconds: Decimal | None) -> int | None: ...
+def milliseconds(seconds: Decimal | None) -> int | None:
+ """A source's elapsed seconds as a place on §6.5's axis, or nothing for no time at all.
+
+ Multiplied before it is rounded, which is the whole point: a fraction the source states
+ is kept to the millisecond rather than rounded away to the second, so a Suunto entry
+ 160 ms after the dive began is at 160 and not at 0.
+ """
+ return None if seconds is None else rounded(seconds * MILLISECONDS_PER_SECOND)
+
+
+def in_seconds(milliseconds: int) -> str:
+ """A place on the axis as a diver reads it in a report: `1200.02`, `30`.
+
+ The report speaks seconds because that is what every source format and every dive
+ computer's display does; the document's milliseconds are an encoding, and a note saying
+ a waypoint is "at 1200020" would send a diver looking for a number their file does not
+ contain. A whole second is written as the integer it is — `30`, never `30.0` — so a
+ report on a file that samples whole seconds shows no fraction the file does not.
+ """
+ return format((Decimal(milliseconds) / MILLISECONDS_PER_SECOND).normalize(), "f")
+
+
# -- text ----------------------------------------------------------------------------
-# The two length caps every adapter meets, whatever it is reading: §6's `notes` on any
-# record, and the 255 that every REQUIRED name in §6 shares — a site's, a trip's, a gear
-# item's, a diver's. A format whose own members reach further caps them here too, and the
-# caps only that format meets stay with it.
-MAX_NOTES = 10_000
+# The length cap every adapter meets, whatever it is reading: the 255 that every REQUIRED
+# name in §6 shares — a site's, a trip's, a gear item's, a diver's. A format whose own
+# members reach further caps them here too, and the caps only that format meets stay with
+# it. A `notes` member has none: it is prose, and a converter passes it through whole.
MAX_NAME = 255
@@ -623,7 +666,7 @@ def deco_model(
the DM5 XML's `` — so the rules that hold across all four are here rather
than four times over: §6.4c's member order, the name trimmed and capped with an empty
one read as absence (§5.4), the schema's range on a gradient factor, **both or neither**
- on the pair, **§3 rule 7's ordering**, and §6.4b's rule that an object with no members is
+ on the pair, **§3 rule 6's ordering**, and §6.4b's rule that an object with no members is
not written at all.
Every one of those is the same argument: a reading the format cannot hold resolves to an
@@ -673,7 +716,7 @@ def deco_model(
"dropped",
)
if len(pair) == 2 and pair["gf_low"] > pair["gf_high"]:
- # §3's rule 7, which the schema cannot express and this converter's own output is
+ # §3's rule 6, which the schema cannot express and this converter's own output is
# held to: a low above a high is a model nothing ran. **Both go**, because the file
# does not say which of the two is the wrong one and choosing would be §5.4's guess.
# It is dropped here rather than left to `validate_document`, which raises and takes
@@ -682,7 +725,7 @@ def deco_model(
note(
where,
f"{labels.get('gf_low', 'gf_low')} is {pair['gf_low']} and "
- f"{labels.get('gf_high', 'gf_high')} is {pair['gf_high']}, and §3 rule 7 records a low no higher "
+ f"{labels.get('gf_high', 'gf_high')} is {pair['gf_high']}, and §3 rule 6 records a low no higher "
"than its high; both are dropped, the source not saying which of the two is wrong",
"dropped",
)
@@ -705,43 +748,116 @@ def deco_model(
return built or None
+@cache
+def recording_members() -> tuple[str, ...]:
+ """§6.4a's Recording members in the section's own order, off the schema.
+
+ `profile_members`'s reason, and one of its own: a readout stated on the dive joins a
+ recording a reader has already built (`onto_primary`), and putting it back in its place
+ is a question of where the section lists it rather than of which order the members
+ happened to arrive in.
+ """
+ return tuple(load_schema()["$defs"]["recording"]["properties"])
+
+
def recording(
*,
device: dict[str, Any] | None = None,
mode: str | None = None,
deco_model: dict[str, Any] | None = None,
+ salinity: str | None = None,
started_at: str | None = None,
+ readouts: dict[str, float] | None = None,
source_files: list[dict[str, Any]] | None = None,
profile: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
"""§6.4a's Recording in the section's member order, or nothing at all.
Nothing at all is the point of the function. §3's rule 4 says a recording carries at
- least one of `device`, `profile` and `source_files`, so a source that describes a
- record of a dive without naming a device and without keeping a sample produces no
- recording — and every adapter that built one anyway would emit `recordings: [{}]` and
- fail its own output validation. `started_at` alone does not qualify: §6.4a reads an
- absent one as the dive's, so a recording carrying only a start describes nothing the
- dive does not already say. **`mode` and `deco_model` do not qualify either**, for the
- same reason and for one of their own: both describe how a computer was running rather
- than anything it recorded, and §3's rule 4 names the three members it names.
+ least one of `device`, `profile`, `source_files` and a readout, so a source that
+ describes a record of a dive without naming a device, keeping a sample or stating a
+ figure the device computed produces no recording — and every adapter that built one
+ anyway would emit `recordings: [{}]` and fail its own output validation. `started_at`
+ alone does not qualify: §6.4a reads an absent one as the dive's, so a recording carrying
+ only a start describes nothing the dive does not already say. **`mode`, `deco_model` and
+ `salinity` do not qualify either**, for the same reason and for one of their own: each
+ describes how a computer was set rather than anything it recorded, and a setting nothing
+ recorded a dive with is not a record of one.
+
+ `readouts` are the five `READOUTS` a reader found, keyed by member; a key outside them
+ raises `KeyError` rather than landing on the recording, the way `recorded` refuses a
+ member the schema does not have.
"""
- if not (device or profile or source_files):
+ for member in readouts or {}:
+ if member not in READOUTS:
+ raise KeyError(member)
+ # `is not None` for a readout and truthiness for the rest, because a zero is a reading
+ # here — the CNS a diver's first dive of the day starts on — where an empty device or
+ # an empty file list is nothing.
+ figures = {member: value for member, value in (readouts or {}).items() if value is not None}
+ if not (device or profile or source_files or figures):
return None
- built: dict[str, Any] = {}
- if device:
- built["device"] = device
- if mode:
- built["mode"] = mode
- if deco_model:
- built["deco_model"] = deco_model
- if started_at:
- built["started_at"] = started_at
- if source_files:
- built["source_files"] = source_files
- if profile:
- built["profile"] = profile
- return built
+ settings = {
+ "device": device,
+ "mode": mode,
+ "deco_model": deco_model,
+ "salinity": salinity,
+ "started_at": started_at,
+ "source_files": source_files,
+ "profile": profile,
+ }
+ return _in_order({**{member: value for member, value in settings.items() if value}, **figures})
+
+
+def _in_order(built: dict[str, Any]) -> dict[str, Any]:
+ return {member: built[member] for member in recording_members() if member in built}
+
+
+def onto_primary(
+ recordings: list[dict[str, Any]],
+ readouts: dict[str, float],
+ *,
+ note: Reporter,
+ where: str,
+ stated: str,
+) -> None:
+ """A readout the source states once for the whole **dive**, onto the recording it is.
+
+ `docs/converting.md`'s rule, for the formats that state one there — Subsurface's
+ `` and `@otu`. A readout is a computer's own arithmetic (§6.4a), so it goes on
+ the recording that computer produced: the **primary**, the first, the file not saying
+ which of its computers computed it. Where the dive has more than one recording that is a
+ decision about meaning rather than about scale, reported `resolved`; where it has none,
+ the readout is a recording of its own, which §6.4a allows — a CNS figure copied off a
+ wrist into a hand-kept log is a record of the dive nothing else produces.
+
+ `stated` names the source's own spelling for the report, the way `device`'s labels do.
+ The list is changed in place.
+ """
+ if not readouts:
+ return
+ if not recordings:
+ built = recording(readouts=readouts)
+ if built is not None:
+ recordings.append(built)
+ return
+ recordings[0] = _in_order({**recordings[0], **readouts})
+ if len(recordings) > 1:
+ note(where, shared_readout(stated, len(recordings)), "resolved")
+
+
+def shared_readout(stated: str, count: int) -> str:
+ """The `resolved` line for a readout stated on a dive that has `count` recordings.
+
+ Its own function because UDDF reaches the same finding by another road — its first link
+ is the primary whether or not it names a device, so the readout goes in as that
+ recording is built rather than onto it afterwards — and one finding is one sentence.
+ """
+ return (
+ f"the dive states {stated} once and has {count} recordings, and a readout is one computer's "
+ "figure; read as the primary recording's, the file not saying which computer computed it "
+ "(spec §6.4a)"
+ )
# -- identity ------------------------------------------------------------------------
diff --git a/divejson/fit.py b/divejson/fit.py
index 61ff2ee..7a9e0a3 100644
--- a/divejson/fit.py
+++ b/divejson/fit.py
@@ -78,6 +78,7 @@
device,
header,
integer_of,
+ milliseconds,
position,
record_inferred,
recorded,
@@ -166,11 +167,12 @@
UNCARRIED_GAS_STATUSES = frozenset({"disabled"})
# `dive_settings.water_type`, whose FIT enum is `{0: fresh, 1: salt, 2: en13319,
-# 3: custom}`. Three of the four are §6.2 members under the same name. `custom` is
-# deliberately absent: it says the diver dialled in a `water_density` number, which is not
-# a water type and which §6.2 has nowhere to put — so it is reported rather than rounded
-# to the nearest real water.
-WATER_TYPES = {"fresh": "fresh", "salt": "salt", "en13319": "en13319"}
+# 3: custom}`, is the density the computer was set to, so three of the four are §6.4a
+# `salinity` values under the same name and none of them is the dive's `water_type`: a
+# setting of the device is not a record of the water. `custom` is deliberately absent: it
+# says the diver dialled in a `water_density` number, which §6.4a has nowhere to put — so it
+# is reported rather than rounded to the nearest named setting.
+SALINITIES = {"fresh": "fresh", "salt": "salt", "en13319": "en13319"}
# `event.event` values that describe the dive, and the §6.5 event type each becomes. A
# table rather than a cast: this is Garmin's vocabulary, and the other 43 members of its
@@ -303,12 +305,12 @@ class _Point:
messages of which 431 carry a depth and 4,294 a temperature — and a transmitter's
`tank_update` is a message of its own at its own instant. They are collected onto one
instant here so that the §6.5 axis is built once, and so that a `record` and a
- `tank_update` at one second are one sample rather than two, the later of which the
+ `tank_update` at one timestamp are one sample rather than two, the later of which the
axis would drop.
`eq=False` so a point stays hashable and compares by identity. A dataclass that
generates `__eq__` sets `__hash__` to `None`, and these are used as dictionary keys —
- an event finds its second by looking its own instant's point up in the axis. Identity
+ an event finds its place by looking its own instant's point up in the axis. Identity
is also the comparison that means anything here: two instants that happened to record
the same depth are still two samples.
"""
@@ -694,8 +696,11 @@ def read_dive(self) -> dict[str, Any] | None:
# In §6.2's own member order, so a converted dive reads down the schema.
self.read_duration(session, summary, dive, where)
self.read_depths(session, summary, samples, dive, where)
- self.read_water_type(dive, where)
- self.read_oxygen(session, summary, dive, where)
+ # The salinity setting and the oxygen clocks are the computer's own (§6.4a), so they
+ # are read here and carried on its recording below rather than on the dive — and the
+ # clocks are enough on their own to make one, where the setting is not.
+ salinity = self.read_salinity(where)
+ readouts = self.read_oxygen(session, summary, where)
self.read_positions(samples, dive)
cylinders, sensors = self.read_cylinders(where)
@@ -706,11 +711,14 @@ def read_dive(self) -> dict[str, Any] | None:
# one to point at.
profile = self.read_profile(samples, cylinders, sensors, where)
# A FIT file is one dive written by one computer, so a converted document has
- # exactly one recording (§6.4a) — and none at all where the file names no computer
- # and kept no usable sample, §6.4a forbidding a recording that carries nothing.
+ # exactly one recording (§6.4a) — and none at all where the file names no computer,
+ # kept no usable sample and states no readout, §6.4a forbidding a recording that
+ # carries nothing.
built = recording(
device=self.read_device(session, where),
deco_model=self.read_deco_model(where),
+ salinity=salinity,
+ readouts=readouts,
profile=profile,
)
if built is not None:
@@ -833,8 +841,8 @@ def local_offset(self) -> timezone | None:
# -- the summary scalars -----------------------------------------------------
- def absent(self, member: str, source: str, where: str) -> None:
- self.note(where, f"the file records no {source}, so the dive's {member} is not carried", "absent")
+ def absent(self, member: str, source: str, where: str, *, record: str = "dive") -> None:
+ self.note(where, f"the file records no {source}, so the {record}'s {member} is not carried", "absent")
def read_duration(
self,
@@ -952,15 +960,16 @@ def read_oxygen(
self,
session: fitdecode.FitDataMessage,
summary: fitdecode.FitDataMessage | None,
- dive: dict[str, Any],
where: str,
- ) -> None:
+ ) -> dict[str, float]:
"""The dive's CNS and OTU totals — the summary first, then the session.
The mirror of `read_depths`, which prefers the session. The order is the other way
round because these are the *dive's* oxygen accounting: on a multi-dive Garmin
file the session totals cover the whole activity, while `summary` has already
- picked out the summary that describes the dive being read.
+ picked out the summary that describes the dive being read. They are the computer's
+ own arithmetic, so they are returned for its recording rather than written onto the
+ dive (§6.4a).
**`o2_toxicity` is the dive's ending OTU total rather than the OTUs it added**,
which the profile's bare "OTUs" unit does not settle. The corpus does: one dive
@@ -969,44 +978,51 @@ def read_oxygen(
delta it would have been 1. There is no `start_otu` anywhere in the FIT profile,
so `otu_start` has no source at all — see `docs/fit-mapping.md`.
"""
+ readouts: dict[str, float] = {}
for member, source in (("cns_start", "start_cns"), ("cns_end", "end_cns"), ("otu_end", "o2_toxicity")):
value = _first(_number(_native(summary, source)), _number(_native(session, source)))
if value is None:
- self.absent(member, f"{source} on its session or on a dive summary", where)
- elif recorded(value, record="dive", member=member):
- dive[member] = float(value)
+ self.absent(member, f"{source} on its session or on a dive summary", where, record="recording")
+ elif recorded(value, record="recording", member=member):
+ readouts[member] = float(value)
else:
self.note(
where,
f"the session's {source} is {value}, which the format records only from zero up; dropped",
"dropped",
)
+ return readouts
- def read_water_type(self, dive: dict[str, Any], where: str) -> None:
- """`dive_settings.water_type`, which is the only salinity evidence a FIT carries.
+ def read_salinity(self, where: str) -> str | None:
+ """`dive_settings.water_type`, which is the density the computer was set to.
- `en13319` stays `en13319` rather than being folded into `salt`: it is the
- calibration a computer ships set to, and rewriting it as the nearest real water
- would be inventing a reading. `custom` says the diver dialled in a density number,
- which §6.2 has no member for, and is reported rather than rounded off.
+ §6.4a's `salinity`, on the recording: a setting of this device rather than a record
+ of the water, so it never reaches the dive's `water_type`, and a reader deriving one
+ from the other is what §6.4a forbids. `en13319` stays `en13319` rather than being
+ folded into `salt`: it is the calibration a computer ships set to, and rewriting it
+ as the nearest real water would be inventing a reading. `custom` says the diver
+ dialled in a density number, which §6.4a has no member for, and is reported rather
+ than rounded off.
**A device that wrote no `dive_settings`, or wrote one with no `water_type`,
raises nothing.** The message is the computer's configuration rather than a record
of the dive, so a setting it did not write is not a reading the dive failed to
take — unlike the session summaries above, every one of which the device was
describing this dive when it left empty.
+
+ **Untested against a real file**: no FIT file in the corpus writes the field.
"""
value = _native(self.scan.settings, "water_type")
if not isinstance(value, str):
- return
- if value in WATER_TYPES:
- dive["water_type"] = WATER_TYPES[value]
- else:
- self.note(
- where,
- f"the device's water type is {value!r}, which this format has no value for; dropped",
- "dropped",
- )
+ return None
+ if value in SALINITIES:
+ return SALINITIES[value]
+ self.note(
+ where,
+ f"the device's water type setting is {value!r}, which this format has no value for; dropped",
+ "dropped",
+ )
+ return None
def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
"""The fix on the way in and the fix on the way out, split at the deepest sample.
@@ -1024,8 +1040,8 @@ def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
there is no pivot and so no answer, and nothing is written: a file that recorded
positions and never a depth cannot say which of them is the entry.
"""
- fixed = [(second, point) for second, point in samples.ordered() if point.latitude is not None]
- depths = [(second, point.depth) for second, point in samples.ordered() if point.depth is not None]
+ fixed = [(at, point) for at, point in samples.ordered() if point.latitude is not None]
+ depths = [(at, point.depth) for at, point in samples.ordered() if point.depth is not None]
if not fixed or not depths:
return
@@ -1036,8 +1052,9 @@ def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
before = [pair for pair in fixed if pair[0] <= pivot]
after = [pair for pair in fixed if pair[0] > pivot]
for member, chosen in (("entry_position", before[-1:]), ("exit_position", after[:1])):
- for second, point in chosen:
- where = f"dive/0/record/{second}"
+ for at, point in chosen:
+ # The elapsed second the record is at, which a FIT timestamp states whole.
+ where = f"dive/0/record/{at // 1000}"
found = position(point.latitude, point.longitude, note=self.note, where=where)
if found is not None:
dive[member] = found
@@ -1291,8 +1308,9 @@ def pressure(self, bar: Decimal | None, where: str) -> Decimal | None:
def axis(self, where: str) -> SampleAxis:
"""The dive's time axis, offered every instant the file recorded a reading at.
- The origin is the session's own start time, so the profile's seconds are elapsed
- time from the moment the dive began — the same instant `started_at` names. A
+ The origin is the session's own start time, so the profile's axis is the elapsed
+ milliseconds from the moment the dive began — the same instant `started_at` names —
+ whole seconds times a thousand, `timestamp` stating nothing finer. A
reading before it is dropped and reported by the axis, and a file whose session
recorded no start time falls back to its earliest reading, which is the only other
thing that can put a sample at zero.
@@ -1313,7 +1331,7 @@ def axis(self, where: str) -> SampleAxis:
start = _native(self.scan.session, "start_time")
origin = start if isinstance(start, datetime) else min(self.scan.points)
for at, point in self.scan.points.items():
- axis.offer(rounded(Decimal(str((at - origin).total_seconds()))), point)
+ axis.offer(milliseconds(Decimal(str((at - origin).total_seconds()))), point)
return axis
def read_profile(
@@ -1339,16 +1357,16 @@ def read_profile(
ceiling = Channel("ceiling")
temperature = Channel("temperature")
pressures = {sensor: Channel("pressures") for sensor in sensors}
- for second, point in samples.ordered():
+ for at, point in samples.ordered():
if point.depth is not None:
- depth.record(second, rounded(point.depth * CENTIMETRES_PER_METRE))
+ depth.record(at, rounded(point.depth * CENTIMETRES_PER_METRE))
if point.ceiling is not None and point.ceiling > 0:
- ceiling.record(second, rounded(point.ceiling * CENTIMETRES_PER_METRE))
+ ceiling.record(at, rounded(point.ceiling * CENTIMETRES_PER_METRE))
if point.temperature is not None:
- temperature.record(second, rounded(point.temperature * TENTHS_PER_UNIT))
+ temperature.record(at, rounded(point.temperature * TENTHS_PER_UNIT))
for sensor, bar in point.pressures.items():
if sensor in pressures and 0 <= bar <= MAX_CYLINDER_PRESSURE:
- pressures[sensor].record(second, rounded(bar * TENTHS_PER_UNIT))
+ pressures[sensor].record(at, rounded(bar * TENTHS_PER_UNIT))
events = self.read_events(samples, where)
numbered = [(number, pressures[sensor]) for number, sensor in enumerate(sensors) if sensor in pressures]
@@ -1389,11 +1407,11 @@ def read_events(self, samples: SampleAxis, where: str) -> list[dict[str, Any]]:
for number, index in enumerate(_native_raw(gas, "message_index") for gas in self.gases)
if isinstance(index, int) and not isinstance(index, bool)
}
- # The axis is what decided which instants have a place and what second each landed
- # on, so an event asks it rather than recomputing from the session's start time:
- # a sample the axis dropped for sharing a second with an earlier one is an instant
- # the profile does not reach, and an event there has nowhere to go either.
- seconds = {point: second for second, point in samples.ordered()}
+ # The axis is what decided which instants have a place and where each landed, so
+ # an event asks it rather than recomputing from the session's start time: a sample
+ # the axis dropped for sharing its millisecond with an earlier one is an instant the
+ # profile does not reach, and an event there has nowhere to go either.
+ places = {point: at for at, point in samples.ordered()}
events: list[dict[str, Any]] = []
for frame in self.scan.events:
@@ -1407,8 +1425,8 @@ def read_events(self, samples: SampleAxis, where: str) -> list[dict[str, Any]]:
if not isinstance(at, datetime):
continue
point = self.scan.points.get(at)
- second = seconds.get(point) if point is not None else None
- if second is None:
+ placed = places.get(point) if point is not None else None
+ if placed is None:
self.note(
where,
"an event is recorded at an instant the dive's samples do not reach, so it has no place "
@@ -1416,7 +1434,7 @@ def read_events(self, samples: SampleAxis, where: str) -> list[dict[str, Any]]:
"dropped",
)
continue
- event: dict[str, Any] = {"time": second, "type": kind} if kind else {"time": second}
+ event: dict[str, Any] = {"time": placed, "type": kind} if kind else {"time": placed}
data = _native(frame, "data")
if kind == "gas_switch" and isinstance(data, int) and not isinstance(data, bool):
number = positions.get(data & MESSAGE_INDEX_MASK)
diff --git a/divejson/series.py b/divejson/series.py
index 3c5bc7f..cdacc64 100644
--- a/divejson/series.py
+++ b/divejson/series.py
@@ -7,7 +7,9 @@
writes an array of objects with their own timestamps; Suunto's DM5 XML writes a
`` carrying every channel, `i:nil` where the sensor had nothing. What §6.5
wants out of all five is the same: channels sampled on their own axes, with strictly
-increasing integer times.
+increasing integer times in milliseconds (§5.1). Every source states its times in seconds,
+whole or fractional, and an adapter hands this module `converter.milliseconds` of them, so
+a fraction the source states keeps its place on the axis.
The rules that survive that translation are the ones in here, and no adapter re-derives
them:
@@ -17,8 +19,8 @@
order.
* **A sample with no time has no place on the axis**, and is dropped and reported. Nothing
else can put a reading anywhere.
-* **Two samples on one second keep the first and report the second**, because the times
- are integers and strictly increasing while a source's are usually neither.
+* **Two samples on one millisecond keep the first and report the second**, because the
+ times are integers and strictly increasing while a source's are usually neither.
* **The samples set the time axis and each channel takes only the samples that carried a
reading for it.** No channel is padded to another's length: a Subsurface dive keeps 431
depths beside 29 temperatures rather than inventing 402 readings.
@@ -44,13 +46,13 @@
from collections.abc import Mapping, Sequence
from typing import Any
-from .converter import Reporter, channel_floor, profile_members
+from .converter import Reporter, channel_floor, in_seconds, profile_members
__all__ = ["Channel", "SampleAxis"]
class Channel:
- """One sampled quantity: the seconds it has readings at, and the readings.
+ """One sampled quantity: the milliseconds it has readings at, and the readings.
Values are integers in the units §6.5 fixes for each channel — centimetres of depth,
tenths of a degree, tenths of a bar, hundredths of a bar of ppO₂ — so the scaling is the
@@ -73,15 +75,16 @@ def __init__(self, name: str) -> None:
self.refused = 0
self._floor = channel_floor(name)
- def record(self, second: int, value: int) -> bool:
- """Take one reading, unless the floor refuses it or the channel has one already.
+ def record(self, at: int, value: int) -> bool:
+ """Take one reading at a millisecond, unless the floor refuses it or one is there.
Two refusals and one return value, reported in two different places on purpose.
**A value below the channel's floor is refused here**, because that floor is §6.5's
and the sentence is the same in every format; the count is reported once, by
- `SampleAxis.profile`. A repeat on one second is two readings of one channel inside
- one sample — two tank pressures resolving to one cylinder — and only the caller
- knows which reading it was, so that one returns silently for the caller to report.
+ `SampleAxis.profile`. A repeat on one millisecond is two readings of one channel
+ inside one sample — two tank pressures resolving to one cylinder — and only the
+ caller knows which reading it was, so that one returns silently for the caller to
+ report.
A caller that reports its own `False` therefore has to be sure which refusal it
saw; `taken` is how it asks, and the one such caller uses it.
@@ -89,15 +92,15 @@ def record(self, second: int, value: int) -> bool:
if self._floor is not None and value < self._floor:
self.refused += 1
return False
- if self.taken(second):
+ if self.taken(at):
return False
- self.times.append(second)
+ self.times.append(at)
self.values.append(value)
return True
- def taken(self, second: int) -> bool:
- """Whether this channel already holds a reading at that second."""
- return bool(self.times) and self.times[-1] == second
+ def taken(self, at: int) -> bool:
+ """Whether this channel already holds a reading at that millisecond."""
+ return bool(self.times) and self.times[-1] == at
def __len__(self) -> int:
return len(self.times)
@@ -109,9 +112,10 @@ def member(self) -> dict[str, list[int]]:
class SampleAxis:
"""The time axis of one dive's profile, and the profile built on it.
- An adapter offers every sample it found with the second it was recorded at, then walks
- `ordered()` to fill its channels, then asks for `profile()`. What is dropped on the way
- is reported through the `note` this was built with.
+ An adapter offers every sample it found with the millisecond it was recorded at, then
+ walks `ordered()` to fill its channels, then asks for `profile()`. What is dropped on
+ the way is reported through the `note` this was built with, which speaks seconds
+ (`converter.in_seconds`) because the source and the diver both do.
"""
__slots__ = ("_note", "_where", "_noun", "_time_member", "_seen", "_offered", "_ordered")
@@ -136,40 +140,40 @@ def __init__(
self._offered: list[tuple[int, Any]] = []
self._ordered: list[tuple[int, Any]] | None = None
- def offer(self, second: int | None, payload: Any) -> None:
- """One of the source's samples, with the second it recorded — or `None` for none.
+ def offer(self, at: int | None, payload: Any) -> None:
+ """One of the source's samples, with its millisecond on the axis — or `None` for none.
`payload` is whatever the adapter needs to read the sample's channels off again on
the second pass; this class never looks inside it.
"""
- at = f"{self._where}/{self._noun}/{self._seen}"
+ where = f"{self._where}/{self._noun}/{self._seen}"
self._seen += 1
- if second is None:
+ if at is None:
self._note(
- at,
+ where,
f"the {self._noun} records no {self._time_member}, so it has no place on "
"the profile's time axis; dropped",
"dropped",
)
- elif second < 0:
- self._note(at, f"the {self._noun} is at {second} s, before the dive began; dropped", "dropped")
+ elif at < 0:
+ self._note(where, f"the {self._noun} is at {in_seconds(at)} s, before the dive began; dropped", "dropped")
else:
- self._offered.append((second, payload))
+ self._offered.append((at, payload))
def ordered(self) -> list[tuple[int, Any]]:
"""The samples that have a place on the axis, in recorded-time order."""
if self._ordered is None:
kept: list[tuple[int, Any]] = []
- for second, payload in sorted(self._offered, key=lambda pair: pair[0]):
- if kept and kept[-1][0] == second:
+ for at, payload in sorted(self._offered, key=lambda pair: pair[0]):
+ if kept and kept[-1][0] == at:
self._note(
self._where,
- f"two {self._noun}s share the second {second}; the later one is dropped, because "
- "the format's sample times are strictly increasing (spec §6.5)",
+ f"two {self._noun}s are both at {in_seconds(at)} s, to the millisecond; the later one is "
+ "dropped, because the format's sample times are strictly increasing (spec §6.5)",
"dropped",
)
continue
- kept.append((second, payload))
+ kept.append((at, payload))
self._ordered = kept
return self._ordered
diff --git a/divejson/ssrf.py b/divejson/ssrf.py
index 97c5dbc..a9284dd 100644
--- a/divejson/ssrf.py
+++ b/divejson/ssrf.py
@@ -21,8 +21,9 @@
reported** rather than converted by a factor no file has checked. That is the whole reason
this reader has no scale ambiguity of its own: UDDF's `` and `` are numbers
whose units the file never states, and there is no such number here. This reader therefore
-emits no `resolved` finding, and the only two kinds in its report are `absent` and
-`dropped`.
+emits one `resolved` finding and it is not about a scale — a `` or `@otu` on a
+dive with more than one recording, the file not saying whose figure it is — and past that
+the only two kinds in its report are `absent` and `dropped`.
**A dive has no id, so its identity is its position.** Subsurface keys a dive by its
computer's own dive id where there is one and by nothing at all otherwise; `@number` is the
@@ -58,8 +59,8 @@
from .converter import (
CENTIMETRES_PER_METRE,
+ MAX_MAGNITUDE,
MAX_NAME,
- MAX_NOTES,
PRODUCER_KEY,
TENTHS_PER_UNIT,
Conversion,
@@ -74,6 +75,8 @@
device,
header,
integer_of,
+ milliseconds,
+ onto_primary,
recorded,
recording,
rounded,
@@ -113,21 +116,38 @@
# A dive's `@date` and `@time`. Subsurface writes both to the digit, and the leniency here
# is for the seconds alone: a hand-edited file missing them costs a dive otherwise, and
-# §5.2's grammar requires them. There is no offset in either pattern because there is none
-# anywhere in the format — see `read_started_at`.
+# §5.2's grammar requires them. A `@date` with no `@time` at all is a date-only start, which
+# §5.2 has a value for. There is no offset in either pattern because there is none anywhere
+# in the format — see `read_started_at`.
_DATE = re.compile(r"\A\d{4}-\d{2}-\d{2}\Z")
_TIME = re.compile(r"\A(?P\d{2}):(?P\d{2})(?::(?P\d{2}))?(?P\.\d+)?\Z")
def _clock(raw: str) -> Decimal | None:
- """`'66:50'` as 4010 seconds, or nothing for text that is not a `M:SS` time."""
+ """`'66:50'` as 4010 seconds, or nothing for text that is not a `M:SS` time.
+
+ The product is held to `MAX_MAGNITUDE` as well as the minutes, because a sample time
+ takes two factors past its text — sixty here and a thousand on the axis — where the
+ bound covers one: minutes that clear it can still reach the axis as a number no double
+ holds, and a time that cannot be carried is text that is not a time.
+ """
match = _CLOCK.match(raw)
if match is None:
return None
minutes = decimal_of(match.group(1))
if minutes is None:
return None
- return minutes * SECONDS_PER_MINUTE + Decimal(match.group(2))
+ seconds = minutes * SECONDS_PER_MINUTE + Decimal(match.group(2))
+ return seconds if seconds <= MAX_MAGNITUDE else None
+
+
+def _calendar_date(text: str) -> bool:
+ """Whether a `YYYY-MM-DD` is a day the calendar has, which no pattern can check."""
+ try:
+ datetime.fromisoformat(f"{text}T00:00:00")
+ except ValueError:
+ return False
+ return True
# Every unit spelling this reader knows, against the way the number in front of it is read.
@@ -419,17 +439,19 @@ def read_dive(self, element: ET.Element, index: int) -> dict[str, Any] | None:
notes = text(child(element, "notes"))
if notes:
- dive["notes"] = self.capped(notes, MAX_NOTES, where, "the note")
+ dive["notes"] = notes
- # `cns='11%'` and `otu='31'`: a percentage and a bare count, both of them the dive's
- # *end* figure. Subsurface records no starting pair, which is why `cns_start` and
- # `otu_start` have no source here.
+ # `cns='11%'` and `otu='31'`: a percentage and a bare count, both of them the *end*
+ # figure of the computer that computed them. Subsurface records no starting pair,
+ # which is why `cns_start` and `otu_start` have no source here. They are §6.4a's
+ # readouts and not the dive's, so they wait for the recordings below.
+ readouts: dict[str, float] = {}
for member, source, unit in (("cns_end", "cns", "%"), ("otu_end", "otu", "")):
value = self.measure(attribute(element, source), unit, where, f"")
if value is None:
continue
- if recorded(value, record="dive", member=member):
- dive[member] = float(value)
+ if recorded(value, record="recording", member=member):
+ readouts[member] = float(value)
else:
self.note(where, f" is {value}, which the format records only from zero up; dropped", "dropped")
@@ -457,6 +479,11 @@ def read_dive(self, element: ET.Element, index: int) -> dict[str, Any] | None:
dive["cylinders"] = cylinders
recordings = self.read_recordings(element, dive, where)
+ # On the `` rather than on a ``, so they go where
+ # `docs/converting.md` sends a dive-level readout: the primary recording, or one of
+ # their own where no computer yielded one.
+ stated = " and ".join(f"" for member in readouts)
+ onto_primary(recordings, readouts, note=self.note, where=where, stated=stated)
if recordings:
dive["recordings"] = recordings
return dive
@@ -474,6 +501,11 @@ def read_started_at(self, element: ET.Element, where: str) -> str | None:
The two halves are composed rather than concatenated, because §5.2's grammar
requires the seconds and a hand-edited `time='11:49'` would otherwise reach the
document and fail its own validation.
+
+ **A `@date` with no `@time` is a date-only start**, the date alone (§5.2): the day was
+ recorded and the time of day was not, and midnight would be a time the file never
+ stated. It carries no offset note, there being no wall clock for an offset to be
+ missing from.
"""
date = attribute(element, "date")
clock = attribute(element, "time")
@@ -485,8 +517,16 @@ def read_started_at(self, element: ET.Element, where: str) -> str | None:
)
return None
if clock is None:
- self.note(where, "the dive records a date with no time of day; read as midnight", "absent")
- clock = "00:00:00"
+ if _DATE.match(date) is None or not _calendar_date(date):
+ self.note(where, "the dive's date is not a date; the dive is dropped (spec §6.2)", "dropped")
+ return None
+ self.note(
+ where,
+ "the dive records a date with no time of day; read as a date-only start rather than as "
+ "midnight (spec §5.2)",
+ "absent",
+ )
+ return date
parts = _TIME.match(clock)
if _DATE.match(date) is None or parts is None:
@@ -787,16 +827,16 @@ def read_profile(self, computer: ET.Element, where: str) -> dict[str, Any] | Non
axis = SampleAxis(self.note, where, noun="sample", time_member="time")
for sample in samples:
- axis.offer(integer_of(self.measure(attribute(sample, "time"), "min", where, "")), sample)
+ axis.offer(milliseconds(self.measure(attribute(sample, "time"), "min", where, "")), sample)
depth = Channel("depth")
temperature = Channel("temperature")
- for second, sample in axis.ordered():
+ for at, sample in axis.ordered():
metres = self.measure(attribute(sample, "depth"), "m", where, "")
if metres is not None:
- depth.record(second, rounded(metres * CENTIMETRES_PER_METRE))
+ depth.record(at, rounded(metres * CENTIMETRES_PER_METRE))
celsius = self.measure(attribute(sample, "temp"), "C", where, "")
if celsius is not None:
- temperature.record(second, rounded(celsius * TENTHS_PER_UNIT))
+ temperature.record(at, rounded(celsius * TENTHS_PER_UNIT))
return axis.profile({"depth": depth, "temperature": temperature})
diff --git a/divejson/suunto_json.py b/divejson/suunto_json.py
index 37d469d..3ebff7a 100644
--- a/divejson/suunto_json.py
+++ b/divejson/suunto_json.py
@@ -83,6 +83,8 @@
decimal_of,
device,
header,
+ in_seconds,
+ milliseconds,
position,
record_inferred,
recorded,
@@ -439,7 +441,11 @@ def _elapsed(moment: datetime, origin: datetime) -> Decimal:
"""
if (moment.tzinfo is None) != (origin.tzinfo is None):
moment, origin = moment.replace(tzinfo=None), origin.replace(tzinfo=None)
- return Decimal(str((moment - origin).total_seconds()))
+ # Exact to the microsecond a `datetime` holds, rather than through the float
+ # `total_seconds` returns: the axis keeps the millisecond, and a value that reached it by
+ # way of a binary fraction could round away from the one the file states.
+ delta = moment - origin
+ return Decimal(delta.days * 86_400 + delta.seconds) + Decimal(delta.microseconds) / Decimal(1_000_000)
@dataclass(slots=True)
@@ -564,8 +570,10 @@ def read_dive(self) -> dict[str, Any] | None:
# In §6.2's own member order, so a converted dive reads down the schema.
self.read_duration(dive, where)
self.read_depths(dive, where)
- self.read_oxygen(diving, dive, where)
- self.read_surface_pressure(diving, dive, where)
+ # The oxygen clocks and the surface pressure are the computer's own figures, so they
+ # are read here and carried on its recording below rather than on the dive.
+ readouts = self.read_oxygen(diving, where)
+ readouts.update(self.read_surface_pressure(diving, where))
self.read_positions(samples, dive)
cylinders, numbering = self.read_cylinders(diving, started.at, where)
@@ -581,6 +589,7 @@ def read_dive(self) -> dict[str, Any] | None:
device=self.read_device(diving, where),
mode=self.read_mode(diving, where),
deco_model=self.read_deco_model(diving, where),
+ readouts=readouts,
profile=profile,
)
if built is not None:
@@ -772,10 +781,10 @@ def read_depths(self, dive: dict[str, Any], where: str) -> None:
)
del dive["avg_depth"]
- def read_oxygen(self, diving: dict[str, Any], dive: dict[str, Any], where: str) -> None:
+ def read_oxygen(self, diving: dict[str, Any], where: str) -> dict[str, float]:
"""The oxygen clock, out of `Diving.StartTissue` and `Diving.EndTissue`.
- **CNS is a 0-1 fraction here and §6.2 holds whole percent**, which is invisible
+ **CNS is a 0-1 fraction here and §6.4a holds whole percent**, which is invisible
until the same dive is read out of two Suunto exports: `EndTissue.CNS: 0.069` is the
DM5 export's `7`. Carrying it unconverted would report a 69 %
oxygen clock as 0.069 %. OTU needs no conversion — it is the same absolute count in
@@ -783,9 +792,10 @@ def read_oxygen(self, diving: dict[str, Any], dive: dict[str, Any], where: str)
full float32 where the desktop one rounds.
Both members read a zero as an answer, which is the schema's decision rather than
- this reader's: §6.2 gives them `minimum: 0`, so the 0 a diver's first dive of the
+ this reader's: §6.4a gives them `minimum: 0`, so the 0 a diver's first dive of the
day starts on is a reading and not a placeholder.
"""
+ readouts: dict[str, float] = {}
for block, half in (("StartTissue", "start"), ("EndTissue", "end")):
tissue = diving.get(block)
tissue = tissue if isinstance(tissue, dict) else {}
@@ -795,7 +805,7 @@ def read_oxygen(self, diving: dict[str, Any], dive: dict[str, Any], where: str)
):
if value is None:
continue
- if not recorded(value, record="dive", member=member):
+ if not recorded(value, record="recording", member=member):
self.note(
where,
f"the export records {member.replace('_', ' ')} as {value}, which is below what the "
@@ -803,18 +813,19 @@ def read_oxygen(self, diving: dict[str, Any], dive: dict[str, Any], where: str)
"dropped",
)
continue
- dive[member] = float(value)
+ readouts[member] = float(value)
+ return readouts
- def read_surface_pressure(self, diving: dict[str, Any], dive: dict[str, Any], where: str) -> None:
- """`Diving.SurfacePressure`, in Pascal where §6.2 holds bar.
+ def read_surface_pressure(self, diving: dict[str, Any], where: str) -> dict[str, float]:
+ """`Diving.SurfacePressure`, in Pascal where §6.4a holds bar.
- §6.2 bounds the member at 0.4 to 1.2 bar, which is the range a barometer at a dive
+ §6.4a bounds the member at 0.4 to 1.2 bar, which is the range a barometer at a dive
site can read; a value outside it is a device that recorded something other than a
surface pressure, and is reported rather than clamped.
"""
pascal = _number(diving.get("SurfacePressure"))
if pascal is None:
- return
+ return {}
bar = pascal / PASCALS_PER_BAR
if not Decimal("0.4") <= bar <= Decimal("1.2"):
self.note(
@@ -823,8 +834,8 @@ def read_surface_pressure(self, diving: dict[str, Any], dive: dict[str, Any], wh
"allows; dropped",
"dropped",
)
- return
- dive["surface_pressure"] = float(bar)
+ return {}
+ return {"surface_pressure": float(bar)}
def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
"""The fix on the way in and the fix on the way out, split at the deepest sample.
@@ -842,8 +853,8 @@ def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
then — so without it these files yield an exit and no entry, while the app draws
both pins from the same export.
"""
- fixed = [(second, sample) for second, sample in samples.ordered() if sample.latitude is not None]
- depths = [(second, sample.depth) for second, sample in samples.ordered() if sample.depth is not None]
+ fixed = [(at, sample) for at, sample in samples.ordered() if sample.latitude is not None]
+ depths = [(at, sample.depth) for at, sample in samples.ordered() if sample.depth is not None]
if not fixed or not depths:
return
@@ -854,12 +865,12 @@ def read_positions(self, samples: SampleAxis, dive: dict[str, Any]) -> None:
before = [pair for pair in fixed if pair[0] <= pivot]
after = [pair for pair in fixed if pair[0] > pivot]
for member, chosen in (("entry_position", before[-1:]), ("exit_position", after[:1])):
- for second, sample in chosen:
+ for at, sample in chosen:
found = position(
sample.latitude,
sample.longitude,
note=self.note,
- where=f"dive/0/sample/{second}",
+ where=f"dive/0/sample/{in_seconds(at)}",
)
if found is not None:
dive[member] = found
@@ -960,7 +971,7 @@ def declared_cylinder(self, gas: dict[str, Any], where: str) -> dict[str, Any]:
limit = None if po2 is None else po2 / PASCALS_PER_BAR
if limit is not None:
if Decimal("0.4") <= limit <= Decimal(2):
- cylinder["po2_limit"] = float(limit)
+ cylinder["ppo2_limit"] = float(limit)
else:
self.note(
where,
@@ -1036,12 +1047,13 @@ def transmitted(self, origin: datetime) -> dict[int, tuple[Decimal | None, Decim
**Over the samples' own recorded instants, and not off the profile.** Two ways of
taking them off the profile lose the answer, and *neither is the merged axis* —
- which reproduces it, `axis` folding an entry into the second another channel's entry
+ which reproduces it, `axis` folding an entry into the instant another channel's entry
already holds rather than dropping it. What loses it is either of the two below,
measured on the file this reader is checked against, whose start pressure is
211.625 bar:
- * an **unmerged** axis, one entry per second with the first winning it whole, gives
+ * an **unmerged** whole-second axis, one entry per second with the first winning it
+ whole, gives
211.26562 — the earlier depth entry taking the second and carrying the cylinder
reading 0.1 s later away with it;
* the axis's pressure **channel**, which §6.5 stores in tenths of a bar, gives
@@ -1140,26 +1152,25 @@ def pressure(self, bar: Decimal | None, where: str) -> Decimal | None:
# -- the profile -------------------------------------------------------------
def axis(self, origin: datetime, where: str) -> SampleAxis:
- """The dive's time axis, offered one merged sample per second the file recorded at.
-
- The origin is `Header.DateTime`, so the profile's seconds are elapsed time from the
- instant `started_at` names — and a sample before it is dropped and reported by the
- axis, which is the right answer for a stream whose entries arrive out of order.
-
- **Entries that land on one second are merged rather than one of them being
- dropped**, and that is the whole difference between a faithful profile and a
- quarter of one. This exporter appends its sensor streams as separate entries: on the
- file this reader is measured against, 7 477 entries carry a depth, a temperature, a
- satellite fix or a battery reading, almost never two of those at once, and they
- collide on the whole seconds §6.5 requires. Offering them one at a time leaves the
- axis choosing between a depth and a temperature recorded at the same instant, and it
- keeps 345 of the dive's 431 depths. Merging keeps all 431 — the count the same
- dive's FIT reading gives — because each of §6.5's channels carries its own times and
- two different channels at one second were never in competition.
-
- What is still a collision is one **channel** twice on a second, and that is reported
- per channel rather than per sample: a file whose streams overlap throughout would
- otherwise write thousands of identical lines saying one thing about the file.
+ """The dive's time axis, offered one merged sample per millisecond the file recorded at.
+
+ The origin is `Header.DateTime`, so the profile's axis is the elapsed milliseconds
+ from the instant `started_at` names — the fraction each `TimeISO8601` states kept, so
+ `suunto-ocean.json`'s first depth sits at 160 rather than at 0 — and a sample before
+ it is dropped and reported by the axis, which is the right answer for a stream whose
+ entries arrive out of order.
+
+ **Entries that land on one millisecond are merged rather than one of them being
+ dropped.** This exporter appends its sensor streams as separate entries: on the file
+ this reader is measured against, 7 477 entries carry a depth, a temperature, a
+ satellite fix or a battery reading, almost never two of those at once. Each of §6.5's
+ channels carries its own times, so two entries of different channels stamped the same
+ millisecond were never in competition, and merging them keeps every depth the dive
+ recorded — 431, the count the same dive's FIT reading gives.
+
+ What is still a collision is one **channel** twice on a millisecond, and that is
+ reported per channel rather than per sample: a file whose streams overlap throughout
+ would otherwise write thousands of identical lines saying one thing about the file.
"""
axis = SampleAxis(self.note, where, noun="sample", time_member="TimeISO8601")
merged: dict[int, _Sample] = {}
@@ -1170,10 +1181,10 @@ def axis(self, origin: datetime, where: str) -> SampleAxis:
if moment is None:
undated += 1
continue
- second = rounded(_elapsed(moment.at, origin))
- standing = merged.get(second)
+ at = milliseconds(_elapsed(moment.at, origin))
+ standing = merged.get(at)
if standing is None:
- merged[second] = self.sample(raw)
+ merged[at] = self.sample(raw)
else:
_merge(standing, self.sample(raw), collisions)
@@ -1182,12 +1193,12 @@ def axis(self, origin: datetime, where: str) -> SampleAxis:
for channel, count in sorted(collisions.items()):
self.note(
where,
- f"{count} {channel} readings land on a second the dive already has one at; the later reading "
+ f"{count} {channel} readings land on a millisecond the dive already has one at; the later reading "
"is dropped, because the format's sample times are strictly increasing (spec §6.5)",
"dropped",
)
- for second, sample in merged.items():
- axis.offer(second, sample)
+ for at, sample in merged.items():
+ axis.offer(at, sample)
return axis
def sample(self, raw: dict[str, Any]) -> _Sample:
@@ -1286,15 +1297,15 @@ def read_profile(
# Counted rather than reported one at a time, for `axis`'s reason: this device writes
# its absent-marker for hundreds of consecutive samples.
empty_tts = 0
- for second, sample in samples.ordered():
+ for at, sample in samples.ordered():
if sample.depth is not None:
- depth.record(second, rounded(sample.depth * CENTIMETRES_PER_METRE))
+ depth.record(at, rounded(sample.depth * CENTIMETRES_PER_METRE))
if sample.ceiling is not None and sample.ceiling > 0:
- ceiling.record(second, rounded(sample.ceiling * CENTIMETRES_PER_METRE))
+ ceiling.record(at, rounded(sample.ceiling * CENTIMETRES_PER_METRE))
if sample.kelvin is not None:
- temperature.record(second, rounded((sample.kelvin - KELVIN_OFFSET) * TENTHS_PER_UNIT))
+ temperature.record(at, rounded((sample.kelvin - KELVIN_OFFSET) * TENTHS_PER_UNIT))
if sample.ndl is not None:
- ndl.record(second, rounded(sample.ndl))
+ ndl.record(at, rounded(sample.ndl))
if sample.tts is not None:
if sample.tts == 0:
empty_tts += 1
@@ -1304,16 +1315,16 @@ def read_profile(
# that floor is applied and reported, once for every format. Refusing a
# negative here instead would drop it silently, which is the one thing
# the shared floor exists to stop.
- tts.record(second, rounded(sample.tts))
+ tts.record(at, rounded(sample.tts))
if sample.gradient_factor is not None:
- gradient_factor.record(second, rounded(sample.gradient_factor))
+ gradient_factor.record(at, rounded(sample.gradient_factor))
if sample.surface_gradient_factor is not None:
- surface_gradient_factor.record(second, rounded(sample.surface_gradient_factor))
+ surface_gradient_factor.record(at, rounded(sample.surface_gradient_factor))
for source_number, pascal in sample.pressures.items():
bar = pascal / PASCALS_PER_BAR
number = numbering.get(source_number)
if number in pressures and 0 <= bar <= MAX_CYLINDER_PRESSURE:
- pressures[number].record(second, rounded(bar * TENTHS_PER_UNIT))
+ pressures[number].record(at, rounded(bar * TENTHS_PER_UNIT))
if empty_tts:
self.note(
where,
@@ -1370,19 +1381,19 @@ def read_events(
would only double it. A `GasSwitch` has no `Active` and is not a pair.
"""
events: list[dict[str, Any]] = []
- for second, sample in samples.ordered():
+ for at, sample in samples.ordered():
for name, payload in sample.events:
- event = self.event(name, payload, second, numbering, where)
+ event = self.event(name, payload, at, numbering, where)
if event is not None:
events.append(event)
return events
def event(
- self, name: str, payload: Any, second: int, numbering: dict[int, int], where: str
+ self, name: str, payload: Any, at: int, numbering: dict[int, int], where: str
) -> dict[str, Any] | None:
"""One `{name: payload}` pair as a §6.5 event, or nothing worth a marker."""
if name == "GasSwitch" and isinstance(payload, dict):
- event: dict[str, Any] = {"time": second, "type": "gas_switch"}
+ event: dict[str, Any] = {"time": at, "type": "gas_switch"}
number = payload.get("GasNumber")
if not isinstance(number, bool) and isinstance(number, int):
# The source's own number resolved to the cylinder's position, so a marker
@@ -1404,14 +1415,14 @@ def event(
# than wording the diver was shown, so carrying it would put "Deco" on a marker
# nobody read.
stop = STOP_TYPES.get(reported.lower())
- return None if stop is None else {"time": second, "type": stop}
+ return None if stop is None else {"time": at, "type": stop}
if name in ALERT_NAMES:
# The type its wording earns, **and** that wording as the label: §6.6's
# vocabulary was seeded from this list, one value per distinct meaning, and
# "Ceiling Broken" is still what the diver was shown. An alert the table does
# not name arrives with no type at all rather than forced into the nearest one.
kind = ALERT_TYPES.get(reported.lower())
- event = {"time": second, "type": kind} if kind is not None else {"time": second}
+ event = {"time": at, "type": kind} if kind is not None else {"time": at}
event["label"] = reported
return event
return None
@@ -1423,13 +1434,13 @@ def scaled(self, value: Decimal | None, factor: Decimal) -> Decimal | None:
def _merge(standing: _Sample, arriving: _Sample, collisions: dict[str, int]) -> None:
- """Fold a later entry's readings into the one already holding that second.
+ """Fold a later entry's readings into the one already holding that millisecond.
The first reading of a channel wins and the later one is counted, which is
`series.py`'s rule applied one level down: the channels are what §6.5 makes strictly
- increasing, and two entries at one second are only in competition where they carry the
+ increasing, and two entries at one millisecond are only in competition where they carry the
same channel. Events are not a channel and are all kept — a gas switch and an alarm on
- one second are two things that happened.
+ one millisecond are two things that happened.
"""
for member, channel in (
("depth", "depth"),
diff --git a/divejson/suunto_xml.py b/divejson/suunto_xml.py
index 9935db1..e926cd3 100644
--- a/divejson/suunto_xml.py
+++ b/divejson/suunto_xml.py
@@ -25,7 +25,7 @@
decompression algorithm; an archive of the owner's whole export directory now converts to
**384** dives where it used to produce 342 and a report saying why the other 42 were
missing. The 42 are exactly the 42 that carry no `` at all — and since they had
-never been read past their mode, carrying them is also what first made the same-second
+never been read past their mode, carrying them is also what first made the sample-time
collision rule fire on a real file. Nothing here is skipped for its mode, which is where
this reader and the Suunto app JSON one part company: a run and a dive are the same shape
there, and `ActivityType` is the only thing that separates them.
@@ -63,7 +63,6 @@
from .converter import (
CENTIMETRES_PER_METRE,
- MAX_NOTES,
PRODUCER_KEY,
TENTHS_PER_UNIT,
Conversion,
@@ -73,12 +72,13 @@
Note,
NoteKind,
Scope,
- capped,
deco_model,
decimal_of,
device,
header,
+ in_seconds,
integer_of,
+ milliseconds,
recorded,
recording,
rounded,
@@ -154,7 +154,7 @@
# pressure it puts on a cylinder in one unit.
MAX_CYLINDER_PRESSURE = Decimal(350)
-# The bounds §6.2 puts on a surface pressure and §6.3 on a ppO₂ limit, in the units those
+# The bounds §6.4a puts on a surface pressure and §6.3 on a ppO₂ limit, in the units those
# members hold. Both are wide enough that a reading outside one is a device that recorded
# something other than what the element claims, which is reported rather than clamped.
MIN_SURFACE_PRESSURE, MAX_SURFACE_PRESSURE = Decimal("0.4"), Decimal("1.2")
@@ -320,24 +320,26 @@ def number(self, element: ET.Element | None, name: str, where: str) -> Decimal |
self.note(where, f"<{name}> is not a number this reader can carry; dropped", "dropped")
return value
- def scalar(self, dive: dict[str, Any], member: str, name: str, where: str) -> None:
- """One recorded number onto one dive member, by the member's own constraint.
+ def scalar(
+ self, target: dict[str, Any], member: str, name: str, where: str, *, record: str = "dive"
+ ) -> None:
+ """One recorded number onto one member of a record, by the member's own constraint.
Which way a zero reads is the schema's decision rather than this module's, so
`recorded` is asked with the member the value is headed for: a `0`
is a dive whose depth the computer never had, while a zero `` is the oxygen
clock a diver's first dive of the day starts on.
- Every member written through here is one §6.2 types as a `number`, so the reading
- is carried as a float. `duration`, the one whole-number member this reader maps, has
+ Every member written through here is one §6 types as a `number`, so the reading is
+ carried as a float. `duration`, the one whole-number member this reader maps, has
`read_duration` of its own — an integer member has to be rounded *before* its
constraint is asked about, and that ordering is the point of separating them.
"""
value = self.number(self.root, name, where)
if value is None:
return
- if recorded(value, record="dive", member=member):
- dive[member] = float(value)
+ if recorded(value, record=record, member=member):
+ target[member] = float(value)
else:
self.note(
where,
@@ -410,18 +412,20 @@ def read_dive(self) -> dict[str, Any] | None:
self.read_duration(dive, where)
notes = _recorded_text(self.root, "Note")
if notes is not None:
- dive["notes"] = capped(notes, MAX_NOTES, note=self.note, where=where, member="")
+ dive["notes"] = notes
self.read_depths(dive, where)
self.scalar(dive, "bottom_temperature", "BottomTemperature", where)
self.read_conditions(where)
+ # The computer's own figures (§6.4a), so they wait for its recording below.
+ readouts: dict[str, float] = {}
for member, name in (
("cns_start", "CnsStart"),
("cns_end", "CnsEnd"),
("otu_start", "OtuStart"),
("otu_end", "OtuEnd"),
):
- self.scalar(dive, member, name, where)
- self.read_surface_pressure(dive, where)
+ self.scalar(readouts, member, name, where, record="recording")
+ self.read_surface_pressure(readouts, where)
cylinders = self.read_cylinders(where)
profile = self.read_profile(cylinders, where)
@@ -438,6 +442,7 @@ def read_dive(self) -> dict[str, Any] | None:
device=self.read_device(where),
mode=mode,
deco_model=self.read_deco_model(mode, where),
+ readouts=readouts,
profile=profile,
)
if built is not None:
@@ -648,10 +653,10 @@ def read_conditions(self, where: str) -> None:
"dropped",
)
- def read_surface_pressure(self, dive: dict[str, Any], where: str) -> None:
- """``, in Pascal where §6.2 holds bar.
+ def read_surface_pressure(self, readouts: dict[str, float], where: str) -> None:
+ """``, in Pascal where §6.4a holds bar.
- §6.2 bounds the member at 0.4 to 1.2 bar, which is the range a barometer at a dive
+ §6.4a bounds the member at 0.4 to 1.2 bar, which is the range a barometer at a dive
site can read. That bound is also the backstop for the unit: every one of the 384
exports in hand lands in 103 100 to 106 700, which is barometric on the Pascal
reading and a hundred metres of seawater on the millibar one.
@@ -668,7 +673,7 @@ def read_surface_pressure(self, dive: dict[str, Any], where: str) -> None:
"dropped",
)
return
- dive["surface_pressure"] = float(bar)
+ readouts["surface_pressure"] = float(bar)
# -- cylinders ---------------------------------------------------------------
@@ -712,7 +717,7 @@ def read_cylinder(self, mixture: ET.Element, number: int, where: str) -> _Cylind
if MIN_PO2_LIMIT <= po2 <= MAX_PO2_LIMIT:
# Already bar here (`1.4`), where the app's JSON export of the
# same dive writes the same limit in Pascal.
- cylinder["po2_limit"] = float(po2)
+ cylinder["ppo2_limit"] = float(po2)
else:
self.note(
where,
@@ -842,14 +847,13 @@ def read_switch_times(self, mixture: ET.Element, where: str) -> list[int]:
times: list[int] = []
for index, change in enumerate(_children(_child(mixture, "DiveGasChanges"), "DiveGasChange")):
at = f"{where}/gas change/{index}"
- seconds = self.number(change, "GasChangeTime", at)
- if seconds is None:
+ placed = milliseconds(self.number(change, "GasChangeTime", at))
+ if placed is None:
continue
- second = rounded(seconds)
- if second < 0:
- self.note(at, f"the gas change is at {second} s, before the dive began; dropped", "dropped")
+ if placed < 0:
+ self.note(at, f"the gas change is at {in_seconds(placed)} s, before the dive began; dropped", "dropped")
continue
- times.append(second)
+ times.append(placed)
return times
# -- the profile -------------------------------------------------------------
@@ -860,12 +864,11 @@ def read_profile(self, cylinders: list[_Cylinder], where: str) -> dict[str, Any]
Every sample element carries every channel, nil where the sensor had nothing — a
mid-dive transmitter dropout is a nil `` on a sample whose `` is
unaffected — so the channels sit on their own axes and none is padded to another's
- length. Two samples on one second are therefore a real collision here rather than
+ length. Two samples on one instant are therefore a real collision here rather than
two sensor streams that were never in competition, and the axis settles them per
- channel; it fires on 37 of the corpus's exports, every one of them a freedive, where
- a 1 s sampling interval meets a `