CoQEx

1. Introduction
2. Data
3. Code
4. Citation
5. License

Introduction

We introduce CoQEx as a method to answering queries on entity counts by consolidating evidences across multiple text snippets. The CoQEx methodology uses a span-based QA model to separately extract candidate count contexts and instances from the top-50 search-engine snippets. The user is shown the following components.

An answer inference is predicted by a distribution-aware inference over count contexts.

The count contexts are further classified into semantic groups with respect to the inferred answer to form the explanation by contexts. They are grouped based on whether the contexts are quite similar to the inferred answer or if they represent a subset of the inferred answer or if they are incomparable.

The instances are ranked by their compatibility with the answer type. They form the explanation by instances since they likely ground the counts into their constituting entities. CoQEx extracts the answer type from the query.

The snippets are annotated with the count context and instance candidates to form the explanation by provenance.

Unlike previous systems, our method infers final answers from multiple observations, supports semantic qualifiers for the counts, and provides evidence by enumerating representative instances.

Here we provide the data for our paper:

• Answering Count Queries with Explanatory Evidence Ghosh et al. SIGIR 2022 (https://dl.acm.org/doi/pdf/10.1145/3477495.3531870).
• Answering Count Queries with Structured Answers from Text Ghosh et al. submitted to JoWS (https://arxiv.org/pdf/2209.07250.pdf)

And the code for setting up an interactive user demonstration. Try out our user demonstration of CoQEx which allows free 100 queries per day.

Data

The data for the SIGIR'22 and JoWS experiments can be found in the respective folders in data. The dataset comprises:

• CoQuAD dataset - training data for inference, groundtruth annoatation for infernce, contextualization and explanations, annotated dataset characteristics. Version 2 contains more evaluation data and query characteristics.

• stresstest - a set of hand-curated challenging count queries with groundtruth inferences.

• lcquad_v2 - subset of count queries from LC-QuAD v2 dataset with groundtruth inferences.

• natural_questions - a subset of count queries from Natural Questions dataset with groundtruth inferences.

Run ./download.sh to download all data in the dataset folder or download them individually.

Code

The code for setting up a Flask app of CoQEx is included in the folder coqex/.

A pipeline for executing CoQEx coming up soon.

Citation

If you use our work please cite us:

@inproceedings{ghosh2022answering,
    title = "Answering Count Queries with Explanatory Evidence",
    author = "Shrestha Ghosh and Simon Razniewski and Gerhard Weikum",
    booktitle = "To appear at SIGIR 2022",
    month = jul,
    year = "2022",
}

License

Shield:

This work is licensed under a Creative Commons Attribution 4.0 International License.