Support for bioxrv

[tomkimpson]

Hi Tim 👋

Saw this advertised on the SPAR-AI slack and thought it was super cool / helpful.

I wanted to explore how hard it would be to extend this to include bioxrv (https://www.biorxiv.org)? Lots of biology and math-bio papers can end up here rather than on arXiv.

Bioxrv allows authors to just submit PDFs, unlike in arXiv where you have to submit .tex. However, it DOES have HTML/XML representations, see https://www.biorxiv.org/tdm and https://www.biorxiv.org/about/FAQ.

What do you think? Of course, one could spin-out a version of arxiv2md specifically for bioxrv, but I think it makes more sense for both to be under the same framework. What do you think?

[timf34]

Hey Tom, thanks for the suggestion - this is a great idea!

I've done some investigation into what this would involve. Here's what I found:

The good news: the internal data models (SectionNode, IngestionResult) are source-agnostic, so bioRxiv papers would flow through the same markdown conversion and output formatting once parsed. The URL parsing and HTML fetching changes are relatively straightforward too.

The main challenge: bioRxiv uses a completely different HTML structure (HighWire Press/Drupal) compared to arXiv's LaTeXML-generated HTML. There's essentially zero overlap in CSS classes and DOM layout:

• Different title, author, abstract, section, figure, table, and citation markup
• This means a new HTML parser would be needed alongside the existing arXiv one

One limitation to be aware of: bioRxiv renders math equations as GIF images in the HTML with no LaTeX source available. For most biology papers this is fine (minimal math), but math-heavy papers would have degraded output.

Suggested approach:

• Add a strategy/adapter pattern with ArxivParser and BiorxivParser implementations
• Share the generic markdown serialization (paragraphs, lists, inline text) but add bioRxiv-specific handlers for figures, tables, citations etc.
• Add bioRxiv URL/DOI detection to the query parser

PRs are very welcome if you'd like to take a crack at it! Happy to help review and guide along the way.