Conversation
A GitHub Actions workflow, triggered by repository_dispatch from
spikeforge-hub-api, that fetches one community-uploaded artifact from a
signed URL and runs it through the existing verification pipeline on a
GitHub-hosted (never self-hosted) runner: bundle checksum/manifest
verification and the weights_only=True load already used by bundle.py,
spikeforge_hub.inspect and .compat, the NIR export and reference-
interpreter drift check spikeforge_hub.import_model already runs for the
curated catalog, and a spikeforge_targets.energy report. It posts a
signed pass/fail report to the hub API's (not yet built)
POST /internal/v1/verifications/{version_id}, naming which step failed
and why on rejection rather than a generic failure message.
Also adds a decompressed-size cap to bundle.py's archive reader, which
had none: a zip bomb was previously an unbounded read into memory rather
than a clean, named rejection.
The self-hosted spikeforge-ci runner (deploy-hetzner.yml) is deliberately
never used here, since it holds the Hetzner deploy key and running
torch.load on a stranger's bytes there is the plan's own worst-case
scenario (plans/hub_accounts_plan.md §7.2).
Every job here sets up Python with `cache: pip`, and on this fleet that step fails outright: "No file ... matched to [**/requirements.txt or **/pyproject.toml], make sure you have checked out the target repository". All seven pyproject.toml files are present in `packages/` at that exact path -- checked on the runner's own workspace while a job sat failed. The runners keep `_work` on another drive behind a symlink, and the action's dependency-file glob does not see through it. It has taken out `blocked-deps`, `client`, `docs` and all five `extras` jobs, and it took out the dashboard's release workflow earlier today for the same reason. It is not reliably reproducible -- an earlier run on main globbed fine -- which makes it worse, not better: a pipeline that merges on green will see spurious red it cannot distinguish from a real failure. Removing it costs nothing. A self-hosted runner keeps `~/.cache/pip` between jobs by itself, so the action was caching a cache.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #48.
What this adds
.github/workflows/hub-verify.yml: a new workflow, triggered byrepository_dispatch(hub-artifact-verify), that runs onubuntu-latestonly — never the self-hostedspikeforge-cirunnerdeploy-hetzner.ymluses, since that runner holds the Hetzner deploykey and running
torch.loadon a stranger's bytes there is thescenario
plans/hub_accounts_plan.md§7.2 names as the mostpredictable way this design could be compromised. It's hardened with
step-security/harden-runnerinblockmode, allow-listing GitHub'sown checkout/runner endpoints, the PyPI/PyTorch install endpoints, and
the two hub-api endpoints the job talks to (
cdn.spikeforge.net,hub.spikeforge.net), and has a 20-minute wall-clock timeout.scripts/hub_verify/: the runner itself, reusing the existing pipelinerather than reimplementing it —
spikeforge.serving.bundleforchecksum/manifest verification and the
weights_only=Trueload,spikeforge_hub.inspect/.compat,spikeforge.nir_bridge.validatefor the same NIR export + reference-interpreter drift check
spikeforge_hub.import_modelruns for the curated catalog, andspikeforge_targets.energyfor the SOP/MAC/AC report. A rejectionnames the failing step and the actual reason (e.g. "bundle: weights
are not loadable: ...", "drift: the exported NIR graph drifts from
the trained module at ...") rather than a generic failure.
spikeforge/serving/bundle.py's archivereader (
bundle_manifest.MAX_DECOMPRESSED_BYTES, 256 MiB — matchingthe plan's own per-artifact quota, §5.3), checked against each zip
entry's declared size before any entry is read. Before this change
_read_archivehad no cap at all, so a zip bomb was an unbounded readinto memory rather than a clean, named rejection.
The contract this implements
Neither this endpoint nor its hub-api counterpart existed in code as of
this session (
spikeforge-hub-api#4is open, no branch/PR). The onebinding requirement I found in committed code is
hub_api/catalog/trust.py::label_for, which only showsmachine-checkedwhenversion.verification["passed"]is the JSONboolean
true. Everything else below is this repo's proposal for therest of the contract; issue #48 and hub-api#4 both describe it only in
prose ("pass/fail, reasons, NIR/compat/energy summary"), so it needs a
look from whoever picks up hub-api#4 before both sides are wired
together for real:
POST {SPIKEFORGE_HUB_API_BASE_URL}/internal/v1/verifications/{version_id}with a JSON body:
version_id,passed(bool),reasons(list ofstrings, empty iff
passed),checks(bundle/inspect/compat/drift/energydetail),summary,generated_at,runner.X-Spikeforge-Hub-Signature: sha256=<hex hmac>,HMAC-SHA256 over the exact raw request body bytes (the same
convention GitHub itself uses for webhook signatures), under the
shared secret
SPIKEFORGE_HUB_VERIFICATION_SECRET— matchingconfig.verification_secret/SPIKEFORGE_HUB_VERIFICATION_SECRETalready declared in
hub_api/config.py.repository_dispatchwithevent_type: hub-artifact-verifyandclient_payload: {version_id, artifact_url}.The callback base URL is deliberately not read from the dispatch
payload — it's a fixed repository variable
(
vars.SPIKEFORGE_HUB_API_BASE_URL) this job trusts, notattacker-influenced data.
loadability and the drift check are hard gates;
compat's presetclassification and the energy report are informational only (a
.spkfbundle already carries its own explicitTopologySpec, sounlike a bare downloaded artifact it never needs to match a known
preset to be verifiable).
This repo needs two things configured before the workflow can run for
real: the
SPIKEFORGE_HUB_VERIFICATION_SECRETsecret and theSPIKEFORGE_HUB_API_BASE_URLrepository variable.One thing worth flagging
My review copy of this repo has
plans/hub_accounts_plan.md(the designof record this issue points at) only on a local, unpushed commit — it
is not present on
origin/mainas of this branch. This PR's codecomments cite it by section (§5.3, §7, §7.2); that's fine once the plan
lands on
main, but a reviewer diffing againstmaintoday won't findthe file. Worth checking before merge.
Testing
pytest(full suite,-m "not network"): 1372 passed, 8 skipped, 3deselected — no regressions in the existing bundle/hub suites.
pytest tests/test_hub_verify_*.py tests/test_serving_bundle.py: 34passed, including the new decompressed-size-cap test and an
end-to-end pipeline test that builds a real
.spkfbundle and runs itthrough the full funnel.
ruff check .: clean.mypy spikeforge/serving/bundle.py spikeforge/serving/bundle_manifest.py:clean (note:
spikeforge.serving.bundleis already inmypy.ini'signore_errorslist, pre-existing).scripts/hub_verifyisn't partof the
mypy/ci.ymlgate (onlyspikeforge/,spikeforge_targets/,spikeforge_hub/,spikeforge_io/,spikeforge_serve/,spikeforge_clients/,server/are), but I ranit manually against the new package and it's clean too.
Actions, and there is no live hub-api deployment or real
SPIKEFORGE_HUB_VERIFICATION_SECRETto exercise the callback againstend to end. The
harden-runneregress allowlist is my best-effortread of what GitHub Actions' own infrastructure needs; I'd recommend
a first run with
egress-policy: auditto confirm it before relyingon
block.