[Code scan] Downstream aggregation drops domains instead of incomplete models

[njzjz]

This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.

Relevant code

LAMBench/lambench/metrics/vishelper/metrics_calculations.py

Lines 152 to 164 in 8c93925

	domain_level_metrics = {}
	for domain, columns in domain_columns.items():
	domain_df = raw_results[columns]
	domain_level_metrics[domain] = domain_df.mean(axis=1)
	domain_results = pd.DataFrame(domain_level_metrics)
	domain_results.dropna(
	axis=1, inplace=True
	) # drop models with missing domain results
	# # Now aggregate all domains to get the final generalizability metrics for each model
	return domain_results.to_dict(orient="index"), domain_results.mean(
	axis=1
	).to_dict()

Impact

domain_results has models as rows and domains as columns, but dropna(axis=1) removes columns. If one model is missing a downstream domain, that domain is dropped for every model, and the incomplete model can still be ranked using the remaining domains.

This changes leaderboard scores by removing valid domains globally instead of excluding or penalizing the incomplete model.

Suggested fix

Make missing-domain policy explicit. For example, compute domain means with skipna=False and then dropna(axis=0) for incomplete models, or apply a documented penalty to missing domains. Add a regression test with one model missing a single domain.