Skip to content

Clarification on leaderboard aggregation formulas (AS Score, Correctness, fast_1, G-Mean Speedup) #99

@Lolo1222

Description

@Lolo1222

Hello, thanks for releasing PassNet.

I have a reproducibility question about the leaderboard / Table 1 aggregation formulas.

From the paper and code, I understand the within-sample computation of ES(t) and AS. However, I could not find the exact cross-sample aggregation used to produce the final leaderboard metrics.

Could you please clarify:

  1. How is model-level AS Score aggregated from the 200 evaluation samples?
  2. Are Correctness and fast_1 computed globally over all 2,060 subgraphs, or averaged from per-sample values?

Thanks a lot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions