search-engine-comparator

Status: archived. Built in March 2025 during search-relevance evaluation work on NASA ADS/SciX. Complete for what it was; not maintained. The follow-on tool is search-comparisons.

What it is

A web app for side-by-side comparison of scholarly search results. One query fans out to NASA ADS/SciX, Google Scholar, and Semantic Scholar; the app measures how much the ranked result lists agree and lets you re-rank the SciX list interactively to see how scoring changes would shift it. A Web of Science client is stubbed (API URL and key wiring in place) but was never configured.

What it does

• Overlap measurement. Jaccard similarity over result sets and rank-biased overlap (the rbo library, p = 0.98) over ranked lists, with a configurable choice of which metadata fields (title, abstract, authors, DOI, year) count as a match.
• Visualization. Venn diagrams of source overlap plus per-pair metric tables and charts, rendered by React components (VennDiagram, MetricsTable, ComparisonResults).
• Ranking experiments. The /api/boost-experiment endpoint re-scores ADS/SciX results with configurable boosts: citation count via log1p scaling, recency with a choice of exponential, inverse, linear, or sigmoid decay, per-document-type weights, and refereed and open-access bonuses. A separate SciX Modifier tab experiments with title-keyword and field-weight changes against the original SciX ordering.

How it is built

• backend/: a FastAPI service (main.py, about 2,400 lines) exposing /api/compare, /api/boost-experiment, and debug routes. ADS and Semantic Scholar are queried over their HTTP APIs with httpx; Google Scholar goes through scholarly with free-proxy rotation and falls back to direct HTML parsing when blocked.
• frontend/: React 18 with Material UI and Recharts, proxying API calls to the backend on port 8000.
• startup.sh ran both halves locally; API keys (ADS/SciX, Semantic Scholar, Web of Science) were read from a .env file in backend/.

Related work

search-comparisons, a fork of adsabs/search-comparisons, continued this line of work with the same engine set plus query-intent analysis. It is archived as well.

MIT License.