Skip to content

Cover all SIM_MET RUNs against MET model (#99 Bucket B)#626

Merged
simnaut merged 2 commits into
mainfrom
sim-met-breadth
May 25, 2026
Merged

Cover all SIM_MET RUNs against MET model (#99 Bucket B)#626
simnaut merged 2 commits into
mainfrom
sim-met-breadth

Conversation

@simnaut
Copy link
Copy Markdown
Owner

@simnaut simnaut commented May 25, 2026

Summary

Closes the SIM_MET item of #99 Bucket B (was "1 of 9 RUNs").

SIM_MET ships eight model-exercising RUNs whose names suggest three atmosphere models (MET_VER / GRAM_MET / JAC_COMP), but its S_define instantiates exactly one model: jeod::METAtmosphere. JEOD v5.4 has no GRAM or Jacchia model anywheremodels/environment/atmosphere/ contains only MET and base_atmos, and a tree-wide grep for gram|jacchia returns nothing. The GRAM_MET / JAC_COMP suffixes name the altitude/latitude sample schedule used by external comparison studies, not a distinct model under test. Every RUN logs the same MET density/temperature via vehicle.atmos_state.

All RUNs are therefore portable Tier-2 cross-checks against our MetAtmosphere. This adds the four previously-uncovered sweeps:

RUN Sweep Notes
RUN_T01_GRAM_MET 100–1500 km altitude extends ceiling 500 km past T02_MET_VER, into deep-hydrogen branch
RUN_T02_GRAM_MET latitude -90°..+90° @ ~650 km includes the poles
RUN_T01_JAC_COMP 100–1000 km altitude Jacchia-comparison grid
RUN_T02_JAC_COMP 100–500 km altitude denser low-altitude grid

Reference CSVs regenerated incrementally via the Docker xtask regenerate-tier3 flow (config from JEOD source only) and committed. CSV headers verified to carry all six expected columns (no DRAscii drop).

Tolerances

Observed max errors sit at the f64 noise floor (density rel ~3e-15, temperature ~1.1e-12 K). The latitude sweep reaches ~1440 K exospheric temps where one f64 ULP is ~2.3e-13 K, so its temperature tolerance is 1.2e-12 K (1.05× observed) vs the 1e-12 K floor elsewhere; density stays at 1e-12.

Not ported

RUN_data_compare re-runs RUN_T01_MET_VER's inputs only to confirm JEOD's several internal MET implementations (preferred vs deprecated AtmosphereState/METAtmosphereState) agree with each other — it adds no new sample point for us to validate. Documented in the module header.

Disposition

7 of the 8 model-exercising RUNs are now genuinely cross-validated against our MET model; the 8th (RUN_data_compare) is a redundant intra-JEOD self-consistency check. Nothing is out-of-scope-non-portable — there is no GRAM/Jacchia physics to port.

Verification

  • cargo nextest run -p astrodyn_atmosphere — 39 passed
  • cargo fmt --check — clean
  • cargo clippy -p astrodyn_atmosphere --tests -- -D warnings — clean

🤖 Generated with Claude Code

…t B)

SIM_MET ships eight RUNs whose names suggest three atmosphere models
(MET_VER / GRAM_MET / JAC_COMP), but its S_define instantiates exactly
one model: jeod::METAtmosphere. JEOD v5.4 has no GRAM or Jacchia model
anywhere (models/environment/atmosphere/ holds only MET + base_atmos).
The GRAM/JAC suffixes name the altitude/latitude sample SCHEDULE used by
external comparison studies, not a distinct model under test — every RUN
logs the same MET density/temperature via vehicle.atmos_state.

All RUNs are therefore portable Tier-2 cross-checks against our
MetAtmosphere. This adds the four previously-uncovered sweep RUNs:

  - RUN_T01_GRAM_MET: 100-1500 km altitude (extends the ceiling 500 km
    past RUN_T02_MET_VER, into the deep-hydrogen branch)
  - RUN_T02_GRAM_MET: latitude -90..+90 deg at ~650 km (incl. poles)
  - RUN_T01_JAC_COMP: 100-1000 km altitude (Jacchia grid)
  - RUN_T02_JAC_COMP: 100-500 km altitude (denser low-altitude grid)

Reference CSVs regenerated incrementally via the Docker xtask flow and
committed. Observed max errors stay at the f64 noise floor (density
rel ~3e-15, temperature ~1.1e-12 K); the latitude sweep reaches ~1440 K
exospheric temps where one ULP is ~2.3e-13 K, so its temperature
tolerance is 1.2e-12 K (1.05x observed) vs the 1e-12 K floor elsewhere.

RUN_data_compare is not ported: it re-runs RUN_T01_MET_VER's inputs only
to confirm JEOD's several internal MET implementations agree, adding no
new sample point. This closes the SIM_MET item: 7 of 8 model-exercising
RUNs covered, the 8th being a redundant self-consistency check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 25, 2026 07:41
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands Tier-2 JEOD cross-validation coverage for the MetAtmosphere implementation by adding the previously unvalidated SIM_MET RUN sweeps (GRAM/JAC schedules) and committing regenerated JEOD reference CSVs for those RUNs.

Changes:

  • Add four new Tier-2 tests to validate additional SIM_MET altitude/latitude sweeps against JEOD reference outputs.
  • Commit new JEOD reference CSVs for the added RUNs (GRAM altitude/latitude and JAC comparison grids).
  • Update the Trick reference-generation script to include the additional SIM_MET RUNs (but see comment re: JAC required output filenames).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
trick/generate_references.sh Adds new SIM_MET RUNs to the reference-generation group (skip-check filenames for JAC runs need correction).
crates/astrodyn_atmosphere/tests/tier2_atmosphere.rs Adds Tier-2 test cases for the additional SIM_MET RUN sweeps and documents the GRAM/JAC naming.
crates/astrodyn_verif_jeod/test_data/met_t01_gram_met.csv New JEOD reference output for RUN_T01_GRAM_MET (100–1500 km sweep).
crates/astrodyn_verif_jeod/test_data/met_t02_gram_met.csv New JEOD reference output for RUN_T02_GRAM_MET (latitude sweep).
crates/astrodyn_verif_jeod/test_data/met_t01_jac_met.csv New JEOD reference output for RUN_T01_JAC_COMP (100–1000 km sweep).
crates/astrodyn_verif_jeod/test_data/met_t02_jac_met.csv New JEOD reference output for RUN_T02_JAC_COMP (100–500 km sweep).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread trick/generate_references.sh Outdated
The DRAscii output base (met_ASCII) canonicalizes to ${label}_met.csv,
so the JAC runs produce met_t0{1,2}_jac_met.csv. The required skip-check
names said _jac_comp.csv, which has_output would never match — forcing the
whole SIM_MET group to re-run on every incremental regen. Align with the
actual (and committed) output filenames.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@simnaut simnaut enabled auto-merge (squash) May 25, 2026 17:52
@simnaut simnaut merged commit ea95597 into main May 25, 2026
18 checks passed
@simnaut simnaut deleted the sim-met-breadth branch May 25, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants