Cover all SIM_MET RUNs against MET model (#99 Bucket B)#626
Merged
Conversation
…t B)
SIM_MET ships eight RUNs whose names suggest three atmosphere models
(MET_VER / GRAM_MET / JAC_COMP), but its S_define instantiates exactly
one model: jeod::METAtmosphere. JEOD v5.4 has no GRAM or Jacchia model
anywhere (models/environment/atmosphere/ holds only MET + base_atmos).
The GRAM/JAC suffixes name the altitude/latitude sample SCHEDULE used by
external comparison studies, not a distinct model under test — every RUN
logs the same MET density/temperature via vehicle.atmos_state.
All RUNs are therefore portable Tier-2 cross-checks against our
MetAtmosphere. This adds the four previously-uncovered sweep RUNs:
- RUN_T01_GRAM_MET: 100-1500 km altitude (extends the ceiling 500 km
past RUN_T02_MET_VER, into the deep-hydrogen branch)
- RUN_T02_GRAM_MET: latitude -90..+90 deg at ~650 km (incl. poles)
- RUN_T01_JAC_COMP: 100-1000 km altitude (Jacchia grid)
- RUN_T02_JAC_COMP: 100-500 km altitude (denser low-altitude grid)
Reference CSVs regenerated incrementally via the Docker xtask flow and
committed. Observed max errors stay at the f64 noise floor (density
rel ~3e-15, temperature ~1.1e-12 K); the latitude sweep reaches ~1440 K
exospheric temps where one ULP is ~2.3e-13 K, so its temperature
tolerance is 1.2e-12 K (1.05x observed) vs the 1e-12 K floor elsewhere.
RUN_data_compare is not ported: it re-runs RUN_T01_MET_VER's inputs only
to confirm JEOD's several internal MET implementations agree, adding no
new sample point. This closes the SIM_MET item: 7 of 8 model-exercising
RUNs covered, the 8th being a redundant self-consistency check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Expands Tier-2 JEOD cross-validation coverage for the MetAtmosphere implementation by adding the previously unvalidated SIM_MET RUN sweeps (GRAM/JAC schedules) and committing regenerated JEOD reference CSVs for those RUNs.
Changes:
- Add four new Tier-2 tests to validate additional
SIM_METaltitude/latitude sweeps against JEOD reference outputs. - Commit new JEOD reference CSVs for the added RUNs (GRAM altitude/latitude and JAC comparison grids).
- Update the Trick reference-generation script to include the additional
SIM_METRUNs (but see comment re: JAC required output filenames).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
trick/generate_references.sh |
Adds new SIM_MET RUNs to the reference-generation group (skip-check filenames for JAC runs need correction). |
crates/astrodyn_atmosphere/tests/tier2_atmosphere.rs |
Adds Tier-2 test cases for the additional SIM_MET RUN sweeps and documents the GRAM/JAC naming. |
crates/astrodyn_verif_jeod/test_data/met_t01_gram_met.csv |
New JEOD reference output for RUN_T01_GRAM_MET (100–1500 km sweep). |
crates/astrodyn_verif_jeod/test_data/met_t02_gram_met.csv |
New JEOD reference output for RUN_T02_GRAM_MET (latitude sweep). |
crates/astrodyn_verif_jeod/test_data/met_t01_jac_met.csv |
New JEOD reference output for RUN_T01_JAC_COMP (100–1000 km sweep). |
crates/astrodyn_verif_jeod/test_data/met_t02_jac_met.csv |
New JEOD reference output for RUN_T02_JAC_COMP (100–500 km sweep). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The DRAscii output base (met_ASCII) canonicalizes to ${label}_met.csv,
so the JAC runs produce met_t0{1,2}_jac_met.csv. The required skip-check
names said _jac_comp.csv, which has_output would never match — forcing the
whole SIM_MET group to re-run on every incremental regen. Align with the
actual (and committed) output filenames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the SIM_MET item of #99 Bucket B (was "1 of 9 RUNs").
SIM_METships eight model-exercising RUNs whose names suggest three atmosphere models (MET_VER/GRAM_MET/JAC_COMP), but itsS_defineinstantiates exactly one model:jeod::METAtmosphere. JEOD v5.4 has no GRAM or Jacchia model anywhere —models/environment/atmosphere/contains onlyMETandbase_atmos, and a tree-wide grep forgram|jacchiareturns nothing. TheGRAM_MET/JAC_COMPsuffixes name the altitude/latitude sample schedule used by external comparison studies, not a distinct model under test. Every RUN logs the same MET density/temperature viavehicle.atmos_state.All RUNs are therefore portable Tier-2 cross-checks against our
MetAtmosphere. This adds the four previously-uncovered sweeps:RUN_T01_GRAM_METT02_MET_VER, into deep-hydrogen branchRUN_T02_GRAM_METRUN_T01_JAC_COMPRUN_T02_JAC_COMPReference CSVs regenerated incrementally via the Docker
xtask regenerate-tier3flow (config from JEOD source only) and committed. CSV headers verified to carry all six expected columns (no DRAscii drop).Tolerances
Observed max errors sit at the f64 noise floor (density rel ~3e-15, temperature ~1.1e-12 K). The latitude sweep reaches ~1440 K exospheric temps where one f64 ULP is ~2.3e-13 K, so its temperature tolerance is 1.2e-12 K (1.05× observed) vs the 1e-12 K floor elsewhere; density stays at 1e-12.
Not ported
RUN_data_comparere-runsRUN_T01_MET_VER's inputs only to confirm JEOD's several internal MET implementations (preferred vs deprecatedAtmosphereState/METAtmosphereState) agree with each other — it adds no new sample point for us to validate. Documented in the module header.Disposition
7 of the 8 model-exercising RUNs are now genuinely cross-validated against our MET model; the 8th (
RUN_data_compare) is a redundant intra-JEOD self-consistency check. Nothing is out-of-scope-non-portable — there is no GRAM/Jacchia physics to port.Verification
cargo nextest run -p astrodyn_atmosphere— 39 passedcargo fmt --check— cleancargo clippy -p astrodyn_atmosphere --tests -- -D warnings— clean🤖 Generated with Claude Code