Skip to content

[Code scan] Pressure task reports success_rate with the wrong denominator #434

Description

@njzjz

This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.

Relevant code

num_samples = 0
num_fails = 0
for pressure in tqdm(["025", "050", "075", "100", "125", "150"]):
init_traj = read(test_data / f"P{pressure}.traj", ":")
final_traj = read(test_data / f"P{pressure}.traj", ":")
for i in tqdm(range(len(init_traj))):
init = init_traj[i]
final = final_traj[i]
assert init.get_chemical_formula() == final.get_chemical_formula()
try:
dft, lam = test_one(init, final, int(pressure), calc, fmax, max_steps)
except Exception as e:
logging.error(
f"Error during test_one at pressure {pressure}, index {i}: {e}"
)
dft, lam = None, None
if dft is None or lam is None:
num_fails += 1
continue
num_samples += 1
all_labels.append(dft)
all_preds.append(lam)
return {
"MAE": mean_absolute_error(all_labels, all_preds), # A3/atom
"RMSE": root_mean_squared_error(all_labels, all_preds), # A3/atom
"success_rate": (num_samples - num_fails) / num_samples,

Impact

num_samples is incremented only after a successful calculation, while num_fails is incremented for failed attempts. The final formula is (num_samples - num_fails) / num_samples, which can underreport success and can crash when every sample fails.

For example, with one successful sample and one failed sample, the current formula reports (1 - 1) / 1 = 0.0 instead of the expected 1 / 2 = 0.5.

Suggested fix

Track total attempts separately from successful samples. Compute success_rate = num_successes / total_attempts, and return controlled NaN or None metrics plus 0.0 success when there are no successful samples.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions