Background
For each Tier 3 scenario the workspace has up to three comparisons:
- runner ↔ JEOD CSV (physics correctness against an external oracle)
- runner ↔ bevy (the two consumers of
astrodyn produce bit-identical state)
- bevy ↔ JEOD (the Bevy adapter is correct against the oracle)
(3) isn't tested directly today. It's implied by transitivity from (1) + (2): if runner matches JEOD within tolerance, and bevy matches runner bit-for-bit, then bevy matches JEOD within the same tolerance. For that argument to hold, every (1) needs a corresponding (2).
Architectural intent
Parity tests should be a superset of runner↔JEOD tests:
- For every
crates/astrodyn_verif_jeod/tests/tier3_*.rs test there is a sibling bevy_parity_* test that propagates the same scenario through Bevy and asserts bit-identical state with the runner.
- Plus additional parity tests for Bevy-adapter mechanisms that don't have a JEOD analog (chained attach events, body-action lifecycle, frame switch, source mutation, …).
The first set carries transitivity to JEOD. The second set policies Bevy-adapter-specific mechanisms.
Current state
Mapping by topic name (best-effort match between tier3_* and bevy_parity_* filenames after stripping the family prefixes):
| Set |
Count |
| Topics with both runner↔JEOD and runner↔bevy |
6 |
| Tier 3 topics missing a parity counterpart (the gap) |
64 |
| Parity topics with no Tier 3 counterpart (Bevy-mechanism stress, intentional) |
23 |
The 6 already-covered topics are: attach_detach_trajectory, gj, kinematic_propagation, mass_attach_detach, relative, srp.
Gap inventory (64 Tier 3 scenarios with no parity counterpart)
apollo8_frame_switch
apollo_mass_tree
apollo_trajectory
attach_mass
battin
complex_attach_detach
contact
drag_6dof
drag_analytical
drag_flatplate_ver
drag_rot_verif
drag_ver
drag_verif
dyncomp_combinations
dyncomp_run10
dyncomp_run2
dyncomp_run3
dyncomp_run4
dyncomp_run5
dyncomp_run6
dyncomp_run7
dyncomp_run9
dyncomp_run_attach_to_ref_frame
earth_moon
euler
euler_edge
force_torque_response
integ_analytical
integ_comparison
integ_gj_orders
lsode
lvlh
lvlh_edge
lvlh_extended
lvlh_relative
lvlh_rot_init_propagation
mars_orbit
mercury
met
ned
ned_edge
orbelem
orbelem_comprehensive
orbinit_docker
orbinit_edge
orbinit_families
orbinit_roundtrip
planetary
polar_motion
ref_attach
relative_extended
shadow_2a
solar_beta
solar_beta_edge
solar_beta_extended
srp_1st_order
srp_basic
srp_rk4_thermal
tide_verif
time_docker
time_reversal
time_scales_comprehensive
timescale
torque_simple
Most are physics scenarios — Apollo lunar transfer, Mars/Mercury/Earth-Moon trajectories, the SIM_dyncomp run2–run10 family, every LVLH/NED/orbelem/orbinit derived-state edge case, the drag and integrator-comparison families, and so on.
Proposed implementation shape
Rather than hand-writing 64 parallel parity test files, the cost-effective approach is a single extension on VerificationCase that turns any Tier 3 scenario definition into a parity test:
// crates/astrodyn_verif_parity/src/lib.rs (new) — or in verif_jeod
pub trait VerificationCaseExt: Sized {
/// Materialize the scenario into both an `astrodyn_runner::Simulation`
/// and an `astrodyn_bevy::App`, step both for the case's duration,
/// and assert every component is bit-identical (`f64::to_bits()`)
/// at every reference-CSV time step.
fn run_and_assert_parity(self);
}
impl VerificationCaseExt for VerificationCase {
fn run_and_assert_parity(self) { /* ... */ }
}
Then each gap test becomes a one-liner:
// crates/astrodyn_verif_parity/tests/bevy_parity_dyncomp_run2.rs
#[test]
fn bevy_parity_dyncomp_run2_3dof() {
astrodyn_verif_jeod::run_verification::sim_dyncomp::run2_3dof()
.run_and_assert_parity();
}
This way the scenario definition stays in one place (verif_jeod::run_verification::sim_*), the parity assertion is one extension implementation, and the per-scenario test files are mechanical wrappers that are cheap to add and maintain.
Open design questions
- Where does
run_and_assert_parity live? Options: (a) on VerificationCase itself in astrodyn_verif_jeod; (b) on a parity-side trait in astrodyn_verif_parity. (a) consolidates "everything verification" but pulls bevy/astrodyn_bevy deps into verif_jeod; (b) keeps Bevy out of the JEOD-verification crate but introduces a parity-side extension. Probably (b) — verif_jeod should stay Bevy-free.
- Does the parity assertion need to run at the same cadence as the JEOD comparison? Tier 3 tests assert at the reference-CSV cadence (typically 1–60 s); parity asserts every integration tick (ms-scale). Two cadences = different assertion shapes.
- 6-DOF vs 3-DOF: the Tier 3
VerificationCase has 3-DOF and 6-DOF flavors of dyncomp_run2 etc.; both should get a parity sibling, or the parity test should iterate over both.
- Exclusion list for scenarios where parity can't hold structurally (e.g., scenarios using runner-only APIs that have no Bevy equivalent yet): mark with
#[ignore] plus a tracking comment, or carve out a known-gap list checked by CI.
Done criteria
Context
Background
For each Tier 3 scenario the workspace has up to three comparisons:
astrodynproduce bit-identical state)(3) isn't tested directly today. It's implied by transitivity from (1) + (2): if runner matches JEOD within tolerance, and bevy matches runner bit-for-bit, then bevy matches JEOD within the same tolerance. For that argument to hold, every (1) needs a corresponding (2).
Architectural intent
Parity tests should be a superset of runner↔JEOD tests:
crates/astrodyn_verif_jeod/tests/tier3_*.rstest there is a siblingbevy_parity_*test that propagates the same scenario through Bevy and asserts bit-identical state with the runner.The first set carries transitivity to JEOD. The second set policies Bevy-adapter-specific mechanisms.
Current state
Mapping by topic name (best-effort match between
tier3_*andbevy_parity_*filenames after stripping the family prefixes):The 6 already-covered topics are:
attach_detach_trajectory,gj,kinematic_propagation,mass_attach_detach,relative,srp.Gap inventory (64 Tier 3 scenarios with no parity counterpart)
Most are physics scenarios — Apollo lunar transfer, Mars/Mercury/Earth-Moon trajectories, the SIM_dyncomp run2–run10 family, every LVLH/NED/orbelem/orbinit derived-state edge case, the drag and integrator-comparison families, and so on.
Proposed implementation shape
Rather than hand-writing 64 parallel parity test files, the cost-effective approach is a single extension on
VerificationCasethat turns any Tier 3 scenario definition into a parity test:Then each gap test becomes a one-liner:
This way the scenario definition stays in one place (
verif_jeod::run_verification::sim_*), the parity assertion is one extension implementation, and the per-scenario test files are mechanical wrappers that are cheap to add and maintain.Open design questions
run_and_assert_paritylive? Options: (a) onVerificationCaseitself inastrodyn_verif_jeod; (b) on a parity-side trait inastrodyn_verif_parity. (a) consolidates "everything verification" but pullsbevy/astrodyn_bevydeps intoverif_jeod; (b) keeps Bevy out of the JEOD-verification crate but introduces a parity-side extension. Probably (b) —verif_jeodshould stay Bevy-free.VerificationCasehas 3-DOF and 6-DOF flavors ofdyncomp_run2etc.; both should get a parity sibling, or the parity test should iterate over both.#[ignore]plus a tracking comment, or carve out a known-gap list checked by CI.Done criteria
VerificationCaseExt::run_and_assert_parity(or equivalent) implemented and tested on a pilot scenario (suggestsim_dyncomp::run2_3dof).#[test]per family).tier3_topics ⊂ bevy_parity_topicsis true after the change.#[ignore = "..."]) and counted in CI to prevent silent regression.Context
crates/astrodyn_verif_jeod/src/run_verification/for the Tier 3 scenario definitions;crates/astrodyn_verif_parity/tests/bevy_parity*.rsfor the existing parity test shape.