Skip to content

Details about the alpha precision evaluation #27

@QDKStorm

Description

@QDKStorm

I noticed these two lines in the code eval/eval_quality.py

    Alpha_Precision_all = qual_res['delta_precision_alpha_naive']
    Beta_Recall_all = qual_res['delta_coverage_beta_naive']

But when I checked the paper that proposed alpha accuracy (Alaa, Ahmed, et al. "How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models." International Conference on Machine Learning. pmlr, 2022.) The paper appears to use the one-class classifier, this means that you should usequal_res['delta_precision_alpha_OC']

Why is delta_precision_alpha_naive used as the value for alpha precision here?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions