Skip to content

Remove PubMedQA built-in benchmark code #64

Description

@arschles

It already exists in the registry server, and should be executed from there rather than the code.

We should make sure that the following are fixed first:

And, after #59 is fixed, the remote benchmark definition should use the new label style rather than the existing exact_match style. Doing this will improve the percentage of parsed responses and thus improve benchmark performance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions