Skip to content

Wikipedia reading-lists import: first collection implementation #4

Description

@oojBuffalo

Problem

src/scrolls/sources/wikipedia.py has exactly one entry point — fetch_item(item), one page at a time via the REST API. There is no way to bring in the articles a user has actually saved. Getting 200 saved Wikipedia articles into the library today means running scrolls add 200 times, and add accepts exactly one URL (cli.py:185).

Wikipedia is the right first implementation of the #2 contract: it is the project's oldest adapter (ADR 0002), it has no auth in place today so the credential path gets exercised honestly, and the resulting items flow through an already-proven fetch/classify/render pipeline — so the new surface is the only thing under test.

Decision already taken (2026-07-29, Elijah)

Reading lists — what you tap "Save" on in the Wikipedia mobile app or while logged in, synced to the account and organised into named lists. Not the watchlist (that is "notify me on edits," a different intent) and not contributions.

Reached via MediaWiki's ReadingLists API, which requires authenticating as the user. Verify the current endpoint, auth flow, and pagination against live MediaWiki documentation before building — do not implement from assumption. If the API turns out to be unavailable or app-only in practice, stop and report that rather than silently substituting the watchlist.

Scope

First implementation of the collection on-ramp contract from #2.

Acceptance criteria

  • Wikipedia declares reading lists as an enumerable collection, per the Collection on-ramp: a fourth ingest path for user-saved collections #2 contract.
  • A CLI path pulls every article across the user's reading lists into the library in one command, paginating the full collection rather than a first page.
  • List membership is preserved — a named reading list should survive as tags or an equivalent user-curation field, matching how the Pocket importer carries tags and the bookmarks importer carries folders (ADR 0030).
  • saved_at comes from Wikipedia's own "when you saved it" timestamp, never published_at — the invariant every existing importer holds.
  • Items dedupe by id against anything already added by URL, so scrolls add of an article and a later collection pull converge on one item (the ADR 0009 identity-alignment property).
  • Missing or rejected credentials fail the whole run with a typed envelope naming exactly what to set — no silent partial import.
  • Fixture-driven tests covering: multi-list pull, pagination past one page, dedupe against an existing item, an empty collection, and the no-credentials failure.
  • docs/adapters.md Collections field for wikipedia updated from gap to the shipped path (Audit every source adapter against the collection-import principle #3).
  • docs/cli.md updated with the real captured output, per the repo's contract-docs convention.

Verification

uv run pytest -q
SCROLLS_HOME=$(mktemp -d) uv run scrolls init
# then the new command against a real account, with captured output in the PR
uv run scrolls list --source wikipedia
uv run scrolls doctor

Depends on #2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified and ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions