This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.
Relevant code
|
num_samples = 0 |
|
num_fails = 0 |
|
|
|
for pressure in tqdm(["025", "050", "075", "100", "125", "150"]): |
|
init_traj = read(test_data / f"P{pressure}.traj", ":") |
|
final_traj = read(test_data / f"P{pressure}.traj", ":") |
|
for i in tqdm(range(len(init_traj))): |
|
init = init_traj[i] |
|
final = final_traj[i] |
|
assert init.get_chemical_formula() == final.get_chemical_formula() |
|
try: |
|
dft, lam = test_one(init, final, int(pressure), calc, fmax, max_steps) |
|
except Exception as e: |
|
logging.error( |
|
f"Error during test_one at pressure {pressure}, index {i}: {e}" |
|
) |
|
dft, lam = None, None |
|
if dft is None or lam is None: |
|
num_fails += 1 |
|
continue |
|
num_samples += 1 |
|
all_labels.append(dft) |
|
all_preds.append(lam) |
|
|
|
return { |
|
"MAE": mean_absolute_error(all_labels, all_preds), # A3/atom |
|
"RMSE": root_mean_squared_error(all_labels, all_preds), # A3/atom |
|
"success_rate": (num_samples - num_fails) / num_samples, |
Impact
num_samples is incremented only after a successful calculation, while num_fails is incremented for failed attempts. The final formula is (num_samples - num_fails) / num_samples, which can underreport success and can crash when every sample fails.
For example, with one successful sample and one failed sample, the current formula reports (1 - 1) / 1 = 0.0 instead of the expected 1 / 2 = 0.5.
Suggested fix
Track total attempts separately from successful samples. Compute success_rate = num_successes / total_attempts, and return controlled NaN or None metrics plus 0.0 success when there are no successful samples.
This issue was found by a Codex global repository scan of tracked non-test files at commit
8c93925cb10b401b2b83c738bd9263fd74474468.Relevant code
LAMBench/lambench/tasks/calculator/pressure/pressure.py
Lines 57 to 84 in 8c93925
Impact
num_samplesis incremented only after a successful calculation, whilenum_failsis incremented for failed attempts. The final formula is(num_samples - num_fails) / num_samples, which can underreport success and can crash when every sample fails.For example, with one successful sample and one failed sample, the current formula reports
(1 - 1) / 1 = 0.0instead of the expected1 / 2 = 0.5.Suggested fix
Track total attempts separately from successful samples. Compute
success_rate = num_successes / total_attempts, and return controlledNaNorNonemetrics plus0.0success when there are no successful samples.