Skip to content

Model hub: accept source: "community" and consume the hosted index #46

Description

@w4ffl35

Context

spikeforge-hub-api (https://github.com/capsize-games/spikeforge-hub-api) now
exists and serves GET /v1/index.json in this project's own catalog schema —
deliberately, so the client needs no second validator. But this repo's own
HubEntry schema doesn't accept the value that index uses:

SOURCES: Tuple[str, ...] = ("bundled", "reference", "url", "hf_repo")

"community" is not in that tuple (confirmed by reading
spikeforge_hub/entry.py:35 directly). A hosted community entry currently
fails HubEntry.from_dict() validation and would be reported through
issues() as malformed rather than rendered.

What's needed

  1. Add "community" to SOURCES in spikeforge_hub/entry.py.
  2. spikeforge_hub/CURATION.md needs a short note distinguishing this from
    every other source: a community entry is never hand-added to
    models.json and never ships in the wheel — see
    plans/hub_accounts_plan.md §6 in this repo for the trust-vocabulary
    reasoning (machine-checked / unchecked, no verified label).
  3. A hosted-index client: fetch GET /v1/index.json from the configured hub
    API base URL with ETag caching into HUB_CACHE_DIR (matching
    spikeforge_hub/cache.py's existing layout conventions), then call the
    existing spikeforge_hub.catalog.load_catalog() on the cached file —
    no second schema, no second validator, per the plan's explicit design
    goal.
  4. spikeforge_hub/downloads.py needs a community source case that
    resolves the entry's url (a cdn.spikeforge.net path in the index) and
    downloads it the same way a hf_repo entry does today.
  5. Reserved-handle awareness: spikeforge_hub/entry.py's existing curated ids
    (reference/..., nir/..., etc.) and a community id's @handle/slug
    shape must never collide — this is already enforced hub-api-side
    (hub_api/catalog/names.py::RESERVED_HANDLES), so this repo doesn't need
    to re-validate it, just needs to render both id shapes correctly.

Server-side WS protocol (separate but related)

server/hub_messages.py and server/hub_handlers.py need two new message
pairs per plans/hub_accounts_plan.md §8.2:

  • hub_pull / hub_pull_state — download a community version into the
    cache, mirroring the existing hub_download_state progress/cancel shape.
  • bundle_build / bundle_built — build a saved model into a .spkf via
    the existing spikeforge/serving/bundle.py, reporting path/size/sha256 so
    the dashboard can then PUT it to the hub API.

These are additive messages — a MINOR protocol bump per
plans/arch-0001-protocol-contract.md, not a PROTOCOL_MAJOR change. Note
hub_list/hub_search need no change: they already just render whatever
catalog.load_catalog() returns, so a community entry with source: "community" appears in the existing payloads once (1) above ships.

Suggested split

This is plausibly two PRs: (a) catalog/entry schema + hosted-index client +
downloads, (b) the WS protocol additions + dashboard-facing plumbing. Land
(a) first — (b) depends on it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitectural design or cross-cutting structural changeenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions