Skip to content

Model hub: verification sandbox runner (P4) #48

Description

@w4ffl35

Context

spikeforge-hub-api has the storage and quota half of community publishing
built and deployed-ready, but every published artifact currently carries the
unchecked trust label because nothing runs this project's own
inspect/compat/NIR/drift/energy tools against an uploaded artifact. That
half — the thing that would make machine-checked mean something — is
tracked in spikeforge-hub-api as "Verification callback endpoint (P4,
hub-api half)"; this issue is the runner that calls that endpoint.

Read plans/hub_accounts_plan.md §7 in this repo before starting — it's the
design of record, including the security reasoning below.

What's needed

A GitHub Actions workflow, triggered by repository_dispatch from
spikeforge-hub-api, that:

  1. Fetches the one artifact via a signed, read-only, single-object URL (the
    hub-api side of this contract is the other issue).
  2. Runs the existing pipeline, in order: spikeforge.serving.bundle.verify
    (checksums, manifest shape), torch.load(..., weights_only=True) (already
    how bundle.py loads weights — non-negotiable for untrusted content),
    spikeforge_hub.inspect, spikeforge_hub.compat, NIR export + the
    reference-interpreter drift check (the same check
    spikeforge/interpreter/ already runs for the curated catalog — reuse it,
    don't reimplement it), then spikeforge_targets.energy for a report.
  3. Posts a signed report back to POST /internal/v1/verifications/{id} on
    the hub API, HMAC'd with a shared secret (GitHub Actions secret on this
    repo, matching SPIKEFORGE_HUB_VERIFICATION_SECRET in the hub-api's
    config).
  4. A rejection names which step failed and why, in the uploader's words —
    "weights are not loadable" is actionable, "verification failed" is not
    (this project's own honesty bar, stated in docs/model-hub.md).

Security — read this before writing the workflow

Do not run this on the existing self-hosted spikeforge-ci runner. That
runner holds the Hetzner deploy key (deploy-hetzner.yml uses it). Running
torch.load on user-supplied bytes on a persistent self-hosted runner is
named explicitly in the plan (§7.2) as the single most predictable way this
whole design could be compromised. Use GitHub-hosted ephemeral runners —
fresh VMs, thrown away per job, with no access to that key.

Additional guards the plan calls for: a decompressed-size cap before reading
zip members (bundle.py's archive.read() currently has none — a zip bomb
is otherwise a runner OOM, not a clean rejection), a wall-clock timeout, and
no network egress beyond the two endpoints this job actually needs.

Scope note

This issue is the runner only. The report schema it posts needs to match
what hub_api/catalog/trust.py::label_for expects
(version.verification["passed"]) — coordinate with whoever picks up the
hub-api-side issue, or do both together.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitectural design or cross-cutting structural changeenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions