feat(round): check WHICH solver produced a PoC, instead of trusting the backend that ran it - #238
Merged
Merged
Conversation
…he backend that ran it The backend runs every miner's agent itself, so "only the approved solver was credited" is a claim it makes about itself and nothing downstream could check. That matters here more than usual: the corpus is public OSS-Fuzz bugs with published reference PoCs, so an enclave run of a lookup table is exactly as genuinely attested as an enclave run of an agent that derived the crash. The measurement is the only thing separating them. Submissions now carry the receipt (backend #42), so `solver_refusal` holds each one to the pin the backend PUBLISHES at /v2/solver: the right workload_sha256, a hardware quote that verified, a report that binds this dispatch, and — when the backend reports on it — an enclave that read the payload it was sent. Every missing piece fails closed. A refused submission scores zero WITHOUT being benchmarked: its PoCs may well crash the target, but a crash nobody can attribute to the approved solver is what this lane exists not to pay for, and rebuilding a corpus for it wastes round time. Refusal is a VERDICT, not an abstention. `evaluated=False` means "I could not judge this" and the backend excludes it from the average; a refusal was judged and must count. The two are one field apart and a payout hangs on the difference. CYBERGYM_REQUIRE_APPROVED_SOLVER=1 turns it on, off by default with a warning that says what is not being checked. Demanding the pin when the backend publishes none RAISES rather than returning None: silently not checking would leave a validator believing it enforces something it does not. It equally must not zero the whole field for that — the existing benchmark-failure path records it and retries, so the validator abstains. A backend that publishes nothing is not the miners' fault.
…s formatter CI caught F821 on the fetch_solver annotation; ruff format then rewrote four files. Verified the refusal logic survived the reformat rather than assuming it did.
Member
Author
|
Self-review before merge (owner-directed — jared asked for the pin to be publishable and checkable; posting the pass here for post-merge audit). Risk-direction first, because this is the first change that can zero a miner who solved tasks:
Residual, stated: this checks which program ran, not that the program could not look the answer up. That needs the egress-restricted enclave (cathedral-sandbox#142 item 3), still blocked on the mint. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pairs with cybergym-backend #42, which publishes the pin and puts the receipt on the submission wire. Together they finish our side of cathedral-sandbox#142 item 4.
Why
The backend runs every miner's agent itself, so "only the approved solver was credited" is a claim the backend makes about itself, and nothing downstream could check it. That matters more here than usual: our corpus is public OSS-Fuzz bugs with published reference PoCs, so an enclave run of a lookup table is exactly as genuinely attested as an enclave run of an agent that derived the crash. The measurement is the only thing separating them — and a validator that doesn't check it is trusting the party it exists to check.
What
solver_refusalholds each submission to the pin the backend publishes at/v2/solver:workload_sha256intel_verifiedreport_data_matchpayload_boundFails closed on every missing piece, with one deliberate exception:
payload_boundabsent is not refused. An older backend omits the key, and refusing every miner over a field that does not exist would be this validator's fault, not theirs. Present-and-false is a real failure and is refused.A refused submission scores zero without being benchmarked. Its PoCs may well crash the target; a crash nobody can attribute to the approved solver is what this lane exists not to pay for, and rebuilding a corpus for it also spends round time we don't have.
Refusal is a verdict, not an abstention.
evaluated=Falsemeans "I could not judge this" and the backend excludes it from the average. A refusal was judged and must count. The two are one field apart and a payout hangs on the difference — so it is tested explicitly.Turning it on
CYBERGYM_REQUIRE_APPROVED_SOLVER=1. Off by default, with a startup warning naming what is not being checked. Demanding the pin when the backend publishes none raises rather than returningNone— silently not checking would leave a validator believing it enforces something it does not. It equally must not zero the field for that: the existing benchmark-failure path records the error, does not advancereported_round, and retries, so the validator abstains. A backend that publishes nothing is not the miners' fault.Verification
151 passedacross the v2 round suite (adds 22): every refusal reason, the case-insensitive compare, the older-backend exemption, verdict-vs-abstention, the never-benchmarked property, off-by-default, pin resolution including the malformed and unpublished cases, and the daemon path end to end — including that no verdicts are posted when the pin is demanded but missing.