feat(publisher): admit cathedral_voice_hybrid with receipt gates

[bateesatobi]

Summary

• Allow cathedral_voice_hybrid on POST /v1/external-scores/violet (was rejected as invalid_source_for_violet_endpoint).
• Fail-closed admission: complete=true, dedicated bearer + mandatory HMAC, metadata.receipt_verified, per-row cathedral_voice_receipt_v1 on positive scores, honest GPU flags, reject simulated TDX unless CATHEDRAL_VOICE_HYBRID_ALLOW_SIMULATION=1.
• Composition: fraction-required + block external_primary (same posture as confidential completeness/auth); does not join confidential 10% global cap or confidential_primary.

Test plan

• python3.12 -m pytest scaffold/publisher/tests/test_voice_hybrid.py (13 passed)
• Publisher deploy: set CATHEDRAL_EXTERNAL_SCORES_TOKEN_CATHEDRAL_VOICE_HYBRID + CATHEDRAL_EXTERNAL_SCORES_HMAC_SECRET_CATHEDRAL_VOICE_HYBRID
• Post one hybrid report from Cathedral Voice validator; confirm 202 and blend metadata

Made with Cursor

[wallscaler]

Sorry for the slow response on this one.

Read it through. The shape is right: adding cathedral_voice_hybrid to MANDATORY_HMAC_SOURCES, COMPLETE_REQUIRED_SOURCES, and the allowed-source set mirrors the cathedral_confidential_tdx posture, and pushing row/metadata validation into a separate voice_hybrid module keeps the shared ingest path readable rather than growing another branch inside _normalize_report.

Two things before this lands.

1. AUDIENCE_REQUIRED_SOURCES was not extended, and I suspect that is an oversight rather than a decision.

You added the source to the other three fail-closed sets, but external_scores.py:78 still reads:

AUDIENCE_REQUIRED_SOURCES = {"cathedral_confidential_tdx"}

The consequence is at :116:

expected = configured_score_audience(source)
if expected is not None:
    ...enforce (network, netuid) == expected, else score_audience_mismatch...
    return network, netuid
# falls through to the permissive branch, where network/netuid are optional

configured_score_audience() returns None for any source outside that set, so a voice_hybrid report skips the strict binding entirely and reaches the branch where network may be absent. The report is therefore never bound to (finney, 39), and the same body is accepted by a publisher configured for a different network or netuid. For the source you modelled this on, that is blocked by design.

Counter-argument, so this is weighted honestly: the HMAC secret is per source (CATHEDRAL_EXTERNAL_SCORES_HMAC_SECRET_CATHEDRAL_VOICE_HYBRID). If that secret differs per deployment, a cross-post fails on HMAC and this is defense in depth rather than a live hole. I would still rather it match the posture of the source it copies, and it is a one-line change. If you left it out deliberately, say why and I will drop it.

2. No CI has run on this branch. gh pr checks 118 reports nothing, so the 13 passing tests are your local run only. I want the publisher suite and both Python lanes green here, and specifically the integration lane, since this touches the shared admission path rather than only new code.

Separately, the two deploy items in your test plan are still unticked. Is the intent to land this ahead of the token and HMAC secret provisioning, or should it wait until one real hybrid report has gone through end to end and returned 202? I am fine either way, I just do not want it merged and then sitting half-provisioned, because an admitted-but-unprovisioned source fails at 503 rather than anywhere obvious.

[bateesatobi]

Not an omission I intended to keep — you are right, and it is now a one-line join of cathedral_voice_hybrid into AUDIENCE_REQUIRED_SOURCES.

HMAC-per-source is defense in depth, not a substitute for binding. Without the audience set, configured_score_audience() returned None and a hybrid report could omit (network, netuid) and be stored on a publisher configured for a different subnet. That is the opposite of the confidential posture this source copies.

Follow-up on this branch (393d23d):

• AUDIENCE_REQUIRED_SOURCES = {"cathedral_confidential_tdx", "cathedral_voice_hybrid"}
• tests: missing audience → invalid_score_audience; mismatch → score_audience_mismatch; unset CATHEDRAL_WEIGHT_POLICY_* → score_audience_not_configured
• local: python3.12 -m pytest scaffold/publisher/tests/test_voice_hybrid.py → 16 passed
• push should kick CI on this PR (publisher suite + Python 3.11/3.12 + integration lane). I will not treat local green as sufficient.

Deploy / merge intent: land the code after CI is green, but do not enable ingest for this source until dedicated token + HMAC are provisioned and one real hybrid report has returned 202. An admitted-but-unprovisioned source failing 503 is worse than waiting. Unticked deploy items stay unticked until that E2E 202.

[wallscaler]

Artifact / Question / Out of scope

• Artifact: PR #118 "feat(publisher): admit cathedral_voice_hybrid with receipt gates" at head 70fed14 (fork bateesatobi/cathedral-validator-voice). Three commits: 09e2739 (feature), 393d23d (audience bind), 70fed14 (merge of main at 4234d5b). Five files, +504/-7. All file:line cites are to the head unless marked (main).
• Question: Did 393d23d land as described. Were the earlier review points addressed. Fresh pass on: (a) server-side vs client-asserted gates, (b) composition and cap interaction with the confidential source, (c) simulation flag defaults across deploy paths, (d) rebase risk against current main.
• Out of scope: the Cathedral Voice producer itself, the receipt format's cryptographic design, the cybergym lane, pre-existing macOS-only test failures, the retired relay installer.

Status of the earlier review points

1. AUDIENCE_REQUIRED_SOURCES not extended. Addressed, but not by the line the author described. 393d23d added the source to the set. The merge 70fed14 then took main at 4234d5b, where #136 (47c3525) had already deleted the set in favor of a publisher-wide bind. At head the set does not exist. Every source goes through _report_audience (scaffold/publisher/external_scores.py:136-146, called at :432). The three tests the author named exist at scaffold/publisher/tests/test_voice_hybrid.py:92-121 and pass (16/16 on head and on main+PR). Net effect matches the confidential posture. The author's comment describes 393d23d, not the head.
2. No CI on the branch. Not addressed, and the blocker is on the maintainer side. The PR is cross-repository. Both workflow runs (09e2739, 70fed14) sit in action_required, waiting for a maintainer to approve a fork run. gh pr checks shows nothing because no check-run was ever created. Approve the run or push the branch into the org repo.
3. Deploy items and merge intent. Answered: land code after green CI, do not enable ingest until token and HMAC are provisioned and one real 202 is seen. See Finding 2: under the production profile, "enable later" is not an env change.

Verdict

Changes requested. Merging the head onto current main turns three gating tests red (tests/thin/test_operator_docs_safety.py x2, tests/boundary/test_canonical_validator_authority.py x1) because the PR restores docs/VIOLET_EXTERNAL_SCORES.md, which main retired to an eight-line # Retired stub in 293e554. GitHub already reports the PR as CONFLICTING (mergeStateStatus: DIRTY) on that one file.

Findings

F1. The docs hunk breaks the gating lanes after rebase (merge-blocker).

• Code path: PR diff to docs/VIOLET_EXTERNAL_SCORES.md. On main the file is an eight-line retired pointer. tests/thin/test_operator_docs_safety.py:333-343 asserts it starts with # Retired; :290 asserts cathedral-validator serve is absent from active docs; tests/boundary/test_canonical_validator_authority.py asserts the same for retired console commands.
• Behavior: the Python 3.11 and 3.12 lanes (tests/thin, tests/boundary) go red the moment this branch is rebased or merged. Verified: merged tree 24 failed vs main 21 failed in the same venv; the three extra are exactly these tests.
• Fix: drop the docs file from the PR. If the voice-hybrid section is needed, it belongs in docs/PUBLISHER_CONFIGURATION.md, the live page.

F2. Composition is unreachable under the production profile, and the PR does not say so.

• Code path: scaffold/publisher/launch_profile.py:191-192 (main) pins CATHEDRAL_EXTERNAL_SCORES_MODE=confidential_primary and CATHEDRAL_EXTERNAL_SCORES_SOURCE=cathedral_confidential_tdx under v2-converged. The shipped unit selects production (deploy/publisher/cathedral-scorer-sn39.service:21) and the env example selects that profile (deploy/publisher/cathedral-scorer-sn39.env.example:9). scaffold/publisher/weights.py:1297-1333 degrades confidential_primary to burn when the source is not in CONFIDENTIAL_PRIMARY_SOURCES; voice hybrid is excluded by design (:150).
• Behavior: in production a voice hybrid report is admitted and stored (ingest is source-agnostic) but never composed. The blend path the PR configures (weights.py:1387 onward) is dead under the only profile production boots. The docs recipe (MODE=blend, FRACTION=0.1) describes the compatibility profile, which production refuses to start.
• Relevance: the author's plan ("enable ingest later after E2E 202") assumes a config flip. Enabling this lane needs a new named launch profile and a decision on what confidential_primary means once a second source exists. Neither is in this PR.

F3. The receipt gates are shape checks on client-asserted fields; nothing is verified server-side.

• Code path: scaffold/publisher/voice_hybrid.py:116 accepts metadata.receipt_verified as a client boolean. :76-81 check request_hash, audio_content_hash, signature for non-empty only. :100-103 check mrtd and quote for non-empty only. :96-99 read debug and simulated as client booleans. :84-87 read the GPU flags likewise. No signature is verified against any key, there is no mrtd allowlist, no quote is parsed.
• Behavior: the only control between a POST and stored positive scores is the dedicated bearer plus HMAC (scaffold/publisher/app.py:3522-3533 (main)). A producer holding the HMAC secret that sends simulated: false and gpu_attested: false is admitted regardless of what ran. The "reject simulated TDX" gate stops an honest producer that labels itself, not a dishonest or compromised one. Economic exposure is the full configured external fraction, bounded by MAX_FRACTION (default 0.5, weights.py:1121), not by the 10% hard cap.
• Relevance: this is the PR's headline control ("fail-closed admission", "receipt gates"). It matches the cathedral_confidential_tdx posture (no publisher-side attestation check for that source either), so it is not a regression. The PR body and docs describe it as more than it is.
• Ask, either: (i) pin an expected mrtd via env and verify receipt.signature against a configured Cathedral Voice public key, or (ii) reword PR body and docs to "schema-gated; the per-source HMAC secret is the trust anchor".

F4. Two new bypass flags are not registered with the production denylist.

• Code path: voice_hybrid.py:34-40 adds CATHEDRAL_VOICE_HYBRID_ALLOW_SIMULATION (default off) and CATHEDRAL_VOICE_HYBRID_REQUIRE_TDX (default on). launch_profile.py:145-172 (main) is _PRODUCTION_FORBIDDEN_TRUTHY, enforced at :1077-1088, pinned by scaffold/publisher/tests/test_strict_production_config.py:1000. Neither new flag is listed.
• Behavior: CATHEDRAL_VOICE_HYBRID_ALLOW_SIMULATION=1 or CATHEDRAL_VOICE_HYBRID_REQUIRE_TDX=0 boots cleanly in production. A copied dev env file silently disables the one gate the PR calls production-critical.
• Answer to (c): default is off in code, and no deploy path sets it (grep over deploy/, both service units, the env example: zero references). The profile does not defend it.
• Fix: add ALLOW_SIMULATION to _PRODUCTION_FORBIDDEN_TRUTHY and to RETIRED_PRODUCTION_FLAGS in the test; pin REQUIRE_TDX to true in _PRODUCTION_PINNED_VALUES.

F5. The audience fence suite fails for the new source.

• Code path: scaffold/publisher/tests/test_external_scores_audience_fence.py:56-178 parametrizes over ALLOWED_ENDPOINT_SOURCES with a minimal report (:31-40: complete, one score, no metadata, no receipt). Voice hybrid rejects it with receipt_verified_required (external_scores.py:476).
• Behavior: three failures on the merged tree that do not exist on main (test_foreign_audience_row_cannot_move_the_local_fence, test_unaudienced_row_cannot_move_the_local_fence, test_epoch_fence_lock_and_gate_serialise_the_same_audience, each [cathedral_voice_hybrid]). The publisher directory is an advisory lane in CI (.github/workflows/tests.yml:224, continue-on-error), so it will not block, but the new source is then never exercised by the fence tests.
• Fix: have the test's _report builder attach hybrid metadata and a receipt for that source, or expose a minimal-valid-report helper from voice_hybrid.py.

Refuted

• (b) Two sources exceed policy together. Refuted. external_scores_source() (weights.py:961) selects one source; _apply_external_scores (:1427) and _apply_confidential_primary (:1297) both read it; cybergym has its own fraction. latest_snapshot_scores filters by source and audience. Voice hybrid and confidential TDX cannot compose in the same vector. What is true: voice hybrid's blend share is capped only by MAX_FRACTION, and MAX_FRACTION=1 allows 100% in blend mode even with external_primary blocked. That is the PR's stated design and unreachable in production (F2), so not a finding.
• The audience fix did not land. Refuted as a defect. The named set is gone; the behavior is enforced and tested.
• Head does not apply on main. Refuted for code. external_scores.py on main is byte-identical to the merge-base (empty diff 4234d5b..main). weights.py auto-merges. normalize_report, bearer_authorized_for_source, verify_hmac_for_source, external_scores_mode, _external_blend_weights all exist under the same names. Only the docs file conflicts.
• Simulation flag enabled somewhere in deploy. Refuted. No reference outside voice_hybrid.py, its tests, and the PR docs.
• Zero-score rows without receipt are a credit hole. Refuted. Score 0 rows are stored as 0 and latest_snapshot_scores treats 0 as revoke. No credit path.
• Local failures are PR regressions. Refuted for 22 of 25 publisher-suite failures and 21 of 24 gating-lane failures. Identical on main-only in the same venv. The remainder are F1 and F5.

Observations

• voice_hybrid.py:109-110 is dead: status != "ok" already raised at :72. The docstring at :108 ("zero scores may still carry receipts (revoke)") is false; a receipt with status revoked is rejected as receipt_not_ok.
• validate_report_metadata runs once per report (external_scores.py:476) and again per row (voice_hybrid.py:133). Harmless.
• measurement.format is optional (voice_hybrid.py:94, if fmt and ...); a missing format passes. measurement.hotkey is optional (:105). Both moot given F3.
• receipt is persisted for every source. external_scores.py:501 reads raw["receipt"] for all sources; :526-527 stores it into the normalized entry, which feeds report_json and the report_sha256 digest. Any existing producer that sends a receipt key gets unvalidated data persisted and a different digest than before, which is the epoch fence and idempotent-retry key. No producer is known to send that key. Gate :526-527 on the hybrid source.
• Every commit carries Co-authored-by: Cursor and the PR body a "Made with Cursor" line. Repo convention is no AI attribution. Squash-merge with a clean message.
• 22 publisher-suite and 21 gating-lane failures are pre-existing on main in this venv (macOS-only set plus score_audience_not_configured even with the env exported). Not investigated; CI is Linux.

Questions for the author

1. What is the intended production path: a new launch profile (for example v2-converged-voice) or a change to confidential_primary semantics? Neither is in this PR; without one the composition is unreachable.
2. Who holds the Cathedral Voice receipt-signing key, and is there a public key the publisher can verify against? If yes, F3 is small. If no, say so in the docs.
3. Is any existing producer sending a receipt key on rows today?
4. Please: rebase onto main, drop the docs file change (or move the section to docs/PUBLISHER_CONFIGURATION.md), add both flags to the production denylist, fix the fence-test parametrization, then I will approve the workflow run.

Verification record

• Head 70fed14 from refs/pull/118/head. Merge-base with main 4234d5b. Main is 185 commits ahead; 6 touch scaffold/publisher, none touch external_scores.py.
• Local, one-off, not CI: Python 3.11 venv, fastapi 0.136.3, bittensor 10.5.0, macOS.

Run	main only	main + PR
test_voice_hybrid.py	n/a	16 passed
publisher external-score suites, CI env	22 failed / 84 passed (without strict config)	25 failed / 441 passed (with strict config)
tests/thin + tests/boundary minus integration extras	21 failed / 2521 passed	24 failed / 2518 passed

• The three extra publisher failures are F5. The three extra gating failures are F1. Main-only has one failure absent from the merged run (test_actionable_chain_failures, errno test, flaky).
• GitHub: mergeable: CONFLICTING, mergeStateStatus: DIRTY; workflow runs for both PR shas in action_required.