M3/E5/Issue #18 — HEL1OS Parsers - #23
Merged
Merged
Conversation
Implements `SPEC-parsers` §2.5–§2.9 and §4 for HEL1OS: band light curves, PHA spectra, housekeeping, per-detector GTI, an event reader, and orbit identity with version precedence. No merge API, no coverage map, no aggregation and no dual write — those belong to the write path (#19). GOVERNANCE FIRST. Verifying against all 391 orbits proved three contract checks undecidable at float64 resolution: R-1 H3 rejected 647 of 1,564 spectra products and §2.8's header-span check rejected 94 of 391 housekeeping products, every one of them by one or two representable steps. `SPEC-parsers` r7 therefore defines `col_span`, adds the §5.1 time-representation allowance ε_t = 1 ms — the value CONTRADICTION-006 Defect A approved in code and never recorded in the contract — and gives the §2.5 and §2.8 time checks the F-06 id they lacked. Recorded as CONTRA-007, CLOSED by r7. §2.5's `mjd` non-decreasing rule is falsified by every event HDU in the archive (1,564 of 1,564; 37.9 M backward steps; largest 1,156 s). It is NOT amended: the parser enforces it and fails closed, and the falsification is recorded OPEN as CONTRA-008 for the maintainer to rule on. D6: the missing §2.5 row checks (span, V-EVT-2, `ener>0`), the declared `ener` and `CTR` units, CZT `pix`/`offsetchn`, and every §2.8 decisive column with its declared unit and archive dtype. Only `czt1temp` and `czt2temp` are required finite, as §2.8 states. D3: `_fits.py` moves from `parsers/solexs/` to `parsers/` so both instruments share the no-default FITS accessors without either importing the other, and gains `declared_unit()`. SoLEXS behaviour is unchanged and regression-tested. Verified with the real 391-orbit corpus: links, contracts, imports, architecture 563, unit 583, integration 146. Clean export from the staged tree: unit 581 (2 skipped), integration 100 (22 skipped), every real-data test skipping rather than failing. Archive-wide, 389 of 391 orbits now parse end to end; the 2 that do not are the known duplicate-HK archive defects, still terminating at F-16. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canonical issue
M3 / E5 / Issue #18 — HEL1OS parsers. Milestone M3, Epic E5 (Ingest), TIS §15 row 18:
| 18 | 18 | E5 | 900 | L | 17 | per-field vs spec | no imputation | parse fixtures |.Dependencies and eligibility evidence
6a96084(PR #20)1d65c63(PR #18)Observation, invariants used here)530a412(PR #15)dfe068b,6c26b33,6beeeaa,807bdf4Eligibility recomputed from
origin/main:docs/tis/TIS-v1.0.md§15 — complete{1…17},eligible
{#18, #22, #23, #25, #26, #27, #39}; lowest Order selects #18.Base:
origin/main=a2893dc.Scope
The five HEL1OS product parsers plus orbit identity and version precedence:
Supporting, approved changes:
_fits.pymoves fromparsers/solexs/toparsers/so bothinstrument packages share the no-default FITS accessors without either importing the other, and
gains
declared_unit(); the three SoLEXS import lines follow it;parsers/__init__.py's staledocstring is corrected.
Scientific contract — ε_t = 1 ms (
SPEC-parsersr7 §5.1)Verifying against all 391 orbits proved three contract checks undecidable at float64 resolution.
Implemented from the r6 text alone, R-1 H3 rejected 647 of 1,564 spectra products and §2.8's
header-span check rejected 94 of 391 housekeeping products — every rejection within about two
representable float64 steps of its bound.
r7 therefore defines
col_spanandheader_span, adds the time-representation allowanceε_t = 1 mswith an explicit scope (§2.5 span and V-EVT-2, §2.7 R-1 H3, §2.8 header span — andnowhere else), and assigns F-06 to the §2.5 and §2.8 time checks that carried no rule id.
V-EVT-2 receives a comparison rule derived from the data itself:
≤ ½·r_isot + ε_t.ε_tis not invented here. It is the value the owner approved in CONTRADICTION-006 Defect Aas an implementation-only fix and the value the Milestone V–VII parsers already applied to R-1
(
_FLOAT_EPS_S = 1e-3). r7 moves it from code into the contract, because an implementation-onlyfix to a contract defect does not survive a re-implementation — which is exactly what happened.
ε_tnever touches a physical time difference: §2.8 inversion statistics stay unthresholded,§2.6's
MJDstays strictly increasing, GTI durations are unchanged, and F-09's exact SoLEXSidentity is untouched.
Every replaced clause is quoted beside its amendment; nothing was overwritten silently.
Defect records
V-EVT-2), plus Observations D (bands need not share a time axis), E (§2.8's detector-health
names are abbreviations), F (three different overlap-pair counts: 46 / 49 / 50), G (archive-wide
HK inversions reaching 1,153 s).
mjdnon-decreasing rule, falsified archive-wide.CONTRA-008 — the event parser remains fail-closed
§2.5's non-decreasing rule is falsified by 1,564 of 1,564 event HDUs (391 of 391 orbits;
37,912,843 backward steps; largest 1,156.36 s). The rule has NOT been amended, reinterpreted or
weakened. The parser enforces it as written and terminates at F-16 on the first decrease,
after yielding only rows that passed every other §2.5 rule. Consequently no real event stream
completes today, and that is the intended behaviour until the maintainer rules.
The specification issue is unresolved. CONTRA-008 records the measurement and a proposed
amendment that is deliberately not applied; the archive-wide findings are evidence for the
record, not permission to bypass the rule. An integration test pins the falsification, so the day
the rule changes, the test changes with it.
Archive-wide verification (all 391 orbits, real corpus)
The two exceptions are
HLS_20260201_120005_43198sec_lev1_V111andHLS_20260202_000005_43183sec_lev1_V111, the duplicate-HK orbits the V&V plan lists as knownarchive defects. They still terminate at F-16, by design. H3 residuals across all 1,564 spectra
lie within ±1.36 µs of the bin bound — about 735× inside
ε_t.Event verification
An independent read-only probe examined 1,352,158,522 event rows across 1,564 detector HDUs:
ener ≤ 0: 0 rowsutc-isotvsmjd: within ±0.5000 ms on every row, 0 parse failures — exactly half theresolution the string carries, which is what r7's V-EVT-2 rule encodes
mjdsteps: 37,912,843 across every HDU → the ordering rule remains fail-closedSoLEXS regression (after the
_fits.pyrelocation)With the real archive: 98 unit + 31 integration tests pass
(
test_solexs_parsers,test_issdc_pradan,test_solexs_parse_with_platform,test_issdc_pradan_acquire). The imports gate passes unchanged, and the test asserting neitherinstrument imports the other still holds. Only the docstring, the new accessor and three import
lines changed; SoLEXS behaviour is identical.
Verification
tests/v2)Clean export
Built with
git archivefrom the staged tree — 3,774 files, no.git, no corpus:links pass, architecture 563, unit 581 passed + 2 skipped, integration 100 passed + 22 skipped,
research 131 passed + 57 skipped. Every real-data test skips rather than fails (12 of the
skips are HEL1OS), satisfying E5 §17 and STD-12.
The export also reports 24 errors, which are the pre-existing IMPL-012 condition and unrelated
to this PR:
tests/integration/test_v1_tag_restores.pyandtests/integration/test_salvage_provenance.pycollect exactly 24 tests, and they assert onrepository refs — the
v1-surya-finaltag. Agit archiveexport carries no.gitand thereforeno tags, so they error there and pass in any real checkout (integration is 146/146 with the
corpus). The same 24 were recorded at #13, #15, #16 and #17.
Limitations
is a specification decision, not an implementation one.
origin/mainin 8 files and in all 11 tags. This PR does not touch them, does not use them, andmust not be read as addressing them. History rewriting is out of scope here.
work supplies evidence toward them.
coverage map in M3/E5/#16 — ISSDC-PRADAN adapter #19.
.hk(§2.4) remains unassigned to any issue — the Engineering Plan defect reported at M2/E4/#14 — Tier 2 manifest schema #17.Explicitly out of scope
#19'sgridandwrite; 1-minute aggregation; dual write; the §4 minute-level coverage map andversion-resolution log; T1–T7 canonical tables; curation; dataset/label/protocol/environment/method
registries; evaluation; evidence; portal; ML; and SoLEXS
.hk.Do not merge on my account: this is a contributor PR from
slazyverse. Rexy-5097 reviews andmerges.
🤖 Generated with Claude Code