feat: Replace ANNOUNCE -> (PUBLISH_NAMESPACE + PUBLISH) and SUBSCRIBE_NAMESPACE#134
Open
itzmanish wants to merge 25 commits intocloudflare:mainfrom
Open
feat: Replace ANNOUNCE -> (PUBLISH_NAMESPACE + PUBLISH) and SUBSCRIBE_NAMESPACE#134itzmanish wants to merge 25 commits intocloudflare:mainfrom
itzmanish wants to merge 25 commits intocloudflare:mainfrom
Conversation
* handles publish_namespace correctly and register to the coordinator * adds subscribe_namespace request handling * updates moq-pub which correct publish_namespace and handles incoming subscribe request at application level
When a TrackReader in the cache has been closed (due to publisher disconnect, error, etc.), new subscribers would receive the stale cached reader instead of triggering a fresh upstream subscription. This change: - Adds TrackReader::is_closed() to detect closed/dropped tracks - Updates TracksReader::subscribe() to check liveness before returning cached tracks, evicting stale entries when found Includes unit tests for both the bug scenario and normal deduplication.
Replace is_some() + unwrap() patterns with if-let to satisfy clippy::unnecessary_unwrap lint in Rust 1.93+. Affected files: - moq-pub/src/media.rs - moq-relay-ietf/src/relay.rs
A standardized test client for MoQT interoperability testing. Implements test scenarios based on the moq-interop-runner framework. Test cases: - setup-only: Connect, complete SETUP exchange, close gracefully - announce-only: Announce namespace, receive OK - subscribe-error: Subscribe to non-existent track, expect error - announce-subscribe: Publisher announces, subscriber subscribes - subscribe-before-announce: Out-of-order setup handling - publish-namespace-done: Announce then send PUBLISH_NAMESPACE_DONE Also enables TLS_DISABLE_VERIFY environment variable support in moq-native-ietf for containerized testing workflows. Usage: moq-test-client --relay https://localhost:4443 moq-test-client --relay https://localhost:4443 --test setup-only moq-test-client --list
cb7c85d to
233bb54
Compare
fix: clippy warnings fix: clippy warning fix: cargo fmt
233bb54 to
0ce8f0d
Compare
…rrors Bug: When multiple tracks (audio + video) are played concurrently in moq-sub, each track's recv_group() writes objects individually to a shared stdout via Arc<Mutex<O>>. Since fMP4 objects alternate between moof (metadata) and mdat (media data) boxes, a concurrent task can acquire the lock between a moof and its corresponding mdat, causing interleaving: [video moof] → [audio moof] → [audio mdat] → [video mdat] ffmpeg then reads video moof + audio mdat as a single segment, parsing MP4 traf box bytes (0x74726166) as H.264 NAL unit sizes, producing: Invalid NAL unit size (1953653094 > 413) Error splitting the input into NAL units This was not observed on the main branch because the relay's announce- based subscribe path (TracksReader::subscribe) creates per-subscriber Track pairs forwarded through the consumer.rs event loop, delivering objects at the natural publish rate with low interleave probability. On feat/publish, the relay's TrackInfo/subscribe_upstream architecture uses shared tracks (OnceLock) with inline forward_upstream(), creating a tighter setup window that increases concurrent delivery probability between audio and video tracks. Fix: Buffer moof objects in recv_group() and write moof+mdat as a single atomic operation under one lock acquisition, preventing any interleaving regardless of relay delivery timing.
…lookup The subscriber maintained two separate alias maps and Notify instances to resolve track_alias for subscribe (pull) vs publish (push) flows. Since the publisher assigns track_alias in both cases, they share a single namespace — the dual-map design required a complex tokio::select! race with fallback in both recv_stream_inner and recv_datagram. Replace with a single TrackOrigin enum, one map, and one Notify.
englishm-cloudflare
pushed a commit
to englishm-cloudflare/moq-rs-1
that referenced
this pull request
Mar 12, 2026
…gering subscriber support Add new types and trait methods to prepare the Coordinator API surface for upcoming protocol features. All new methods have default implementations that return no-op/empty results, so existing implementors compile without changes. New types: - ScopeConfig: Per-scope configuration (origin_fallback, lingering_subscribe) - NamespaceSubscription: RAII handle for SUBSCRIBE_NAMESPACE with existing matches - NamespaceInfo: Identity of a registered namespace - RelayInfo: Relay endpoint for forwarding notifications - TrackRegistration: RAII handle for track-level PUBLISH - TrackEntry: Identity of a registered track - TrackSubscription: RAII handle for lingering subscriber interest New Coordinator methods: - get_scope_config(scope): Get per-scope configuration - subscribe_namespace(scope, prefix): Register interest in namespace prefix - unsubscribe_namespace(scope, prefix): Remove namespace prefix interest - lookup_namespace_subscribers(scope, namespace): Find interested relays - register_track(scope, namespace, track): Register track availability - unregister_track(scope, namespace, track): Remove track registration - list_tracks(scope, namespace): List tracks under a namespace - subscribe_track(scope, namespace, track): Pre-register track interest - unsubscribe_track(scope, namespace, track): Remove track interest - lookup_track_subscribers(scope, namespace, track): Find waiting subscribers These stubs align with moq-worker RPCs (GetScopeConfig, RegisterSubscribeNamespace, ListNamespaces, LookupSubscribers, RegisterTrack, ListTracks, etc.) and enable the relay layer to be wired up once PR cloudflare#134 (PUBLISH_NAMESPACE/SUBSCRIBE_NAMESPACE protocol support) lands.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the legacy
ANNOUNCE-based namespace advertisement and subscription routing with the draft-16 alignedPUBLISH_NAMESPACE,PUBLISH, andSUBSCRIBE_NAMESPACEmessage flows. This is a major protocol evolution that decouples namespace advertisement from subscription routing, enabling both push-based (PUBLISH) and pull-based (SUBSCRIBE) media delivery through the relay.Motivation
The
ANNOUNCEmechanism in earlier drafts bundled namespace advertisement with subscription routing — when a publisher announced a namespace, incoming SUBSCRIBEs were routed through the Announce object directly. Draft-16 separates these concerns:This separation enables more flexible relay architectures where publishers and subscribers can use different delivery strategies for the same content.
Changes
Protocol Layer (
moq-transport/src/session/)announce.rs(Announce/AnnounceRecv) — replaced by decoupled message handlerspublish_namespace.rs— outbound PUBLISH_NAMESPACE (publisher → relay)publish_namespace_received.rs— inbound PUBLISH_NAMESPACE handler (renamed fromannounced.rs)published.rs— outbound PUBLISH with full subgroup/datagram serving (publisher → relay)publish_received.rs— inbound PUBLISH handler (relay side)subscribe_namespace.rs— outbound SUBSCRIBE_NAMESPACEsubscribe_namespace_received.rs— inbound SUBSCRIBE_NAMESPACE handlerpublisher.rs— removedannounce(), addedpublish_namespace(),publish(),subscribe_namespace_received()APIs; all SUBSCRIBEs now route throughunknown_subscribedqueuesubscriber.rs— added PUBLISH message handling, publish alias trackingsubscribed.rs— addedis_closed()cancellation checks in serve pathsRelay (
moq-relay-ietf/)local.rs: Complete rewrite — introducedTrackInfostate machine (Pending → Subscribing → Subscribed → Publishing → Closed) andLocalsregistry withLocalsEntrycontainingTracksReader,TracksWriter, and per-trackHashMap<String, Arc<TrackInfo>>for concurrent push/pull managementproducer.rs:serve_subscribe()now usesTrackInfosystem — callsget_or_create_track_info(), checksshould_subscribe_upstream(), forwards viasubscribe_upstream(). Addedserve_subscribe_namespace()handlerconsumer.rs:serve()renamed toserve_publish_namespace()— registers both reader AND writer with Locals. Addedserve_publish()for handling inbound PUBLISH messages (push-based delivery)Serve Layer (
moq-transport/src/serve/)tracks.rs: Addedforward_upstream()on TracksReader,insert()andremove()on TracksWriter for relay track managementtrack.rs: Addedis_closed(),largest_location(), comprehensive test suite for stale track cache evictionsubgroup.rs/stream.rs/datagram.rs: Addedis_closed()checksClients
moq-pub: Switched frompublisher.announce(reader)topublisher.publish_namespace(namespace)+ explicitserve_subscriptions()loopmoq-clock-ietf: Updated to usepublish_namespace()APImoq-test-client: Updated scenarios for PUBLISH message flowBug Fixes Included
TrackInfowithOnceLock+ inlineforward_upstream) delivers track data with tighter timing than the old announce-based path, which increases the probability of concurrent audio/video stdout writes interleaving between moof and mdat boxes. This caused H.264 decode errors (Invalid NAL unit sizewhere0x74726166= "traf" box type was parsed as NAL data). Fixed by buffering moof and writing moof+mdat as a single atomic operation inmoq-sub'srecv_group().TracksReader::subscribe()now checksis_closed()before returning cached tracks, evicting stale entries and requesting fresh ones from the publisher. Prevents subscribers from being stuck on dead tracks after publisher reconnection.TrackInfostate machine prevents conflicts when both PUBLISH and SUBSCRIBE arrive for the same track.Testing
test_stale_track_cache_bug)test_track_deduplication_while_alive)test_track_not_stale_after_subgroups_transition,test_track_stale_after_subgroups_writer_dropped)moq-test-clientscenarios