From 20837c98c6ae2ac1456dd65568036784bbd2c6e6 Mon Sep 17 00:00:00 2001 From: Bronek Kozicki Date: Thu, 10 Sep 2026 19:38:02 +0100 Subject: [PATCH 1/5] Carry column metadata on publication requests Replace SendGossip.block with optional GossipMetadata, wrapping the existing block record and adding GossipDataColumn. PublishDataColumn carries its column record directly. Attach slot, block_root, and column_index at both existing publication sites after KZG verification. Preserve the RPC sync gate and the absence of publication for buffered, held, and reconstructed columns. Control carries the column record into its converted SendGossip request, which stays local to peer-manager. Controller tests observe encoding, sender exclusion, IWANT service, and no second spine publication. The converted record's fields have no independent observation because peer-manager ignores metadata when routing. The boundary still serves block metadata only. Its existing socket test also injects both column request variants to protect block subscribers. Column SSE serving follows in the next commit. Leave Persist and Available behavior unchanged. A table checks gossip and following RPC metadata for custody and non-custody columns. Syncing RPC processes its column without requesting publication. Focused cases cover Fulu proposer resolution, held copies, buffered Gloas columns, and EL reconstruction. A mixed valid/invalid KZG case requires publication of the valid column and silence for the invalid one. The signed EF staged-parent case checks both arrival orders. Buffered copies remain silent; fresh copies request relay after their parent is staged. The fixture loader discovers the sidecar without embedding its hash and derives metadata from its contents. The staged-parent case requires installed Fulu EF fixtures. An absent fixture directory reports unavailable coverage; an incomplete installed case fails. The final run exercised this case with fixtures present. One counting blob provides real KZG cells with distinguishable proofs. The focused Fulu proposer test pre-seeds the signature cache because its empty validator registry cannot verify signatures. Column gossip fixtures reuse SSZ as an unread protobuf handle. Sidecars enter production handlers, and loop_body completes validation. Publication assertions check metadata and topic without fixing batch contents or requiring deferred verification. Persistence observations establish successful processing on otherwise silent paths. Controller fixtures use synthetic transport payloads. The pre-existing EL reconstruction reservation-failure bug remains outside this change. Validation: just fmt-check, just clippy, just nextest, and git diff --check exited zero. Nextest passed 1,368 tests and skipped five. Assisted-by: Codex:gpt-6-astra --- crates/application_boundary/src/lib.rs | 10 +- crates/application_boundary/tests/tile.rs | 24 +- crates/beacon_state/tile/src/tile/gossip.rs | 6 +- crates/beacon_state/tile/src/tile/tests.rs | 16 +- .../tile/src/tile/tests/block_relay.rs | 7 +- crates/columns/src/tile.rs | 263 +++++++++++---- crates/columns/src/tile/tests/publication.rs | 308 ++++++++++++++++++ crates/common/src/lib.rs | 22 +- crates/common/src/spine.rs | 14 +- crates/common/src/spine/messages.rs | 24 +- crates/control/src/tile.rs | 13 +- crates/control/src/tile/tests.rs | 85 ++++- crates/peer/src/manager/mod.rs | 2 +- crates/peer/src/manager/promises.rs | 9 +- 14 files changed, 677 insertions(+), 126 deletions(-) create mode 100644 crates/columns/src/tile/tests/publication.rs diff --git a/crates/application_boundary/src/lib.rs b/crates/application_boundary/src/lib.rs index 502883c5..3bac9be1 100644 --- a/crates/application_boundary/src/lib.rs +++ b/crates/application_boundary/src/lib.rs @@ -4,8 +4,8 @@ use flux::{spine::SpineAdapter, tile::Tile}; use silver_beacon_api::{BeaconApi, SlotStatus}; use silver_beacon_state_data::{BeaconStateReader, SpecConfig}; use silver_common::{ - BeaconStateEvent, BlockStage, Enr, GossipBlock, Identify, Keypair, PeerEvent, SilverSpine, - SyncUpdate, TProducer, TRandomAccess, + BeaconStateEvent, BlockStage, Enr, GossipBlock, GossipMetadata, Identify, Keypair, PeerEvent, + SilverSpine, SyncUpdate, TProducer, TRandomAccess, }; use silver_config::EngineConfig; use silver_engine_api::EngineApi; @@ -100,8 +100,10 @@ impl ApplicationBoundaryTile { _ => {} }); adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { block: Some(GossipBlock { slot, block_root }), .. } = - event + if let PeerEvent::SendGossip { + metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root })), + .. + } = event { beacon.publish_block_gossip(slot, &block_root); } diff --git a/crates/application_boundary/tests/tile.rs b/crates/application_boundary/tests/tile.rs index 1a1f03fd..40ac8519 100644 --- a/crates/application_boundary/tests/tile.rs +++ b/crates/application_boundary/tests/tile.rs @@ -14,9 +14,9 @@ use silver_beacon_api::SlotStatus; use silver_beacon_state_data::{BeaconStateOwner, SpecConfig}; use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ELSyncStatus, EngineFcuReq, EngineReq, EngineResp, - Enr, GossipBlock, GossipTopic, Identify, Keypair, MessageId, P2pStreamId, - PayloadValidationStatus, PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, TCache, - TCacheProducer, ssz_view::STATUS_V2_SIZE, + Enr, GossipBlock, GossipDataColumn, GossipMetadata, GossipTopic, Identify, Keypair, MessageId, + P2pStreamId, PayloadValidationStatus, PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, + TCache, TCacheProducer, ssz_view::STATUS_V2_SIZE, }; use silver_config::EngineConfig; use silver_engine_api::test_el::{FCU_VALID_RESULT, FakeEl, write_jwt}; @@ -161,7 +161,7 @@ fn block_relay(slot: u64, byte: u8) -> PeerEvent { msg_hash: MessageId { id: [byte; 20] }, recv_ts: Nanos::now(), protobuf, - block: Some(GossipBlock { slot, block_root: [byte; 32] }), + metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root: [byte; 32] })), } } @@ -843,12 +843,26 @@ fn block_subscriptions_select_imports_and_preserve_repeated_relay_requests() { let mixed = EventsSubscriber::new(addr, "block,block_gossip", 5, &mut crank); let mut unrelated = block_relay(9, 0xaf); - let PeerEvent::SendGossip { topic, block: metadata, .. } = &mut unrelated else { + let PeerEvent::SendGossip { topic, metadata, protobuf, .. } = &mut unrelated else { unreachable!() }; *topic = GossipTopic::BeaconAttestation(0); *metadata = None; + let ssz = *protobuf; inj.produce(unrelated); + + let column = GossipDataColumn { slot: 13, block_root: [0xaf; 32], column_index: 5 }; + let mut column_relay = unrelated; + let PeerEvent::SendGossip { topic, metadata, .. } = &mut column_relay else { unreachable!() }; + *topic = GossipTopic::DataColumnSidecar(5); + *metadata = Some(GossipMetadata::DataColumn(column)); + inj.produce(column_relay); + inj.produce(PeerEvent::PublishDataColumn { + originator: P2pStreamId::new(1, 0, StreamProtocol::DataColumnSidecarsByRange, true), + topic: GossipTopic::DataColumnSidecar(5), + ssz, + column, + }); inj.produce(block_relay(10, 0xac)); inj.produce(block_relay(10, 0xac)); inj.produce(block_received(11, 0xab, BlockStage::Applied)); diff --git a/crates/beacon_state/tile/src/tile/gossip.rs b/crates/beacon_state/tile/src/tile/gossip.rs index 3589250e..4ed5353a 100644 --- a/crates/beacon_state/tile/src/tile/gossip.rs +++ b/crates/beacon_state/tile/src/tile/gossip.rs @@ -6,8 +6,8 @@ use silver_beacon_state_data::{ }; use silver_common::{ ATTESTATION_SUBNETS, BeaconStateEvent, BlockSource, EngineNewPayloadEnvelopeReq, EngineReq, - GossipBlock, GossipTopic, LOCAL_GOSSIP_STREAM_ID, MAX_BLOBS_PER_BLOCK, NewGossipMsg, PeerEvent, - SyncNeed, TCacheRead, TRead, hex32, + GossipBlock, GossipMetadata, GossipTopic, LOCAL_GOSSIP_STREAM_ID, MAX_BLOBS_PER_BLOCK, + NewGossipMsg, PeerEvent, SyncNeed, TCacheRead, TRead, hex32, metrics::timed, ssz_view::{ AttestationDataView, AttesterSlashingView, ExecutionPayloadEnvelopeView as Envelope, @@ -1306,7 +1306,7 @@ impl BeaconStateTile { msg_hash: m.msg_hash, recv_ts: m.recv_ts, protobuf: m.protobuf, - block, + metadata: block.map(GossipMetadata::Block), }); } diff --git a/crates/beacon_state/tile/src/tile/tests.rs b/crates/beacon_state/tile/src/tile/tests.rs index e1a4735b..a92e4f62 100644 --- a/crates/beacon_state/tile/src/tile/tests.rs +++ b/crates/beacon_state/tile/src/tile/tests.rs @@ -10,9 +10,9 @@ use silver_beacon_state_data::{ StateReadView, ValSeed, Withdrawals, }; use silver_common::{ - BlockStage, EngineNewPayloadResp, GossipBlock, GossipTopic, LOCAL_GOSSIP_STREAM_ID, MessageId, - P2pStreamId, PeerEvent, StreamProtocol, SyncNeed, TCache, TCacheProducer, TCacheRead, - TProducer, + BlockStage, EngineNewPayloadResp, GossipBlock, GossipMetadata, GossipTopic, + LOCAL_GOSSIP_STREAM_ID, MessageId, P2pStreamId, PeerEvent, StreamProtocol, SyncNeed, TCache, + TCacheProducer, TCacheRead, TProducer, column_util::block_root_fulu, ssz_view::{ ATTESTATION_DATA_SIZE, AttestationView, BEACON_BLOCK_BODY_FIXED, BYTES_PER_KZG_COMMITMENT, @@ -821,15 +821,15 @@ fn block_relay_requires_a_resolved_proposer() { let mut relays = Vec::new(); adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, block, .. } = event { + if let PeerEvent::SendGossip { topic, metadata, .. } = event { assert_eq!(topic, GossipTopic::BeaconBlock); - relays.push(block); + relays.push(metadata); } }); let expected = GossipBlock { slot, block_root: block_root_fulu(&bytes) }; assert_eq!( relays, - if want_relay { vec![Some(expected)] } else { vec![] }, + if want_relay { vec![Some(GossipMetadata::Block(expected))] } else { vec![] }, "{target:?}, slot {slot}" ); } @@ -3268,8 +3268,8 @@ mod block_relay; fn non_block_relays(adapter: &mut SpineAdapter) -> Vec { let mut topics = Vec::new(); adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, block, .. } = event { - assert_eq!(block, None, "{topic:?} carries no block metadata"); + if let PeerEvent::SendGossip { topic, metadata, .. } = event { + assert_eq!(metadata, None, "{topic:?} carries no SSE metadata"); topics.push(topic); } }); diff --git a/crates/beacon_state/tile/src/tile/tests/block_relay.rs b/crates/beacon_state/tile/src/tile/tests/block_relay.rs index 137fe0a2..1122b5d8 100644 --- a/crates/beacon_state/tile/src/tile/tests/block_relay.rs +++ b/crates/beacon_state/tile/src/tile/tests/block_relay.rs @@ -19,9 +19,12 @@ impl Published { sink.consume(|event: BeaconStateEvent, _| events.push(event)); let mut relays = Vec::new(); sink.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, block, .. } = event { + if let PeerEvent::SendGossip { topic, metadata, .. } = event { assert_eq!(topic, GossipTopic::BeaconBlock); - relays.push(block.expect("every block relay carries its metadata")); + let Some(GossipMetadata::Block(block)) = metadata else { + panic!("every block relay carries block metadata") + }; + relays.push(block); } }); Self { events, relays } diff --git a/crates/columns/src/tile.rs b/crates/columns/src/tile.rs index 2bbde276..1f846c71 100644 --- a/crates/columns/src/tile.rs +++ b/crates/columns/src/tile.rs @@ -12,9 +12,9 @@ use flux_profiler::timed; use silver_beacon_state_data::{B256, BeaconStateReader, SLOTS_PER_EPOCH, SpecConfig}; use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ColumnSource, DataColumnsEvent, DataKind, - EngineResp, GossipTopic, IngestionTime, NewGossipMsg, Origin, P2pStreamId, PeerEvent, - RequestId, RpcInbound, RpcSeverity, SilverSpine, SilverSpineProducers, StreamProtocol, - SyncNeed, SyncUpdate, TCacheRead, TProducer, TRandomAccess, TRead, Wheel, + EngineResp, GossipDataColumn, GossipMetadata, GossipTopic, IngestionTime, NewGossipMsg, Origin, + P2pStreamId, PeerEvent, RequestId, RpcInbound, RpcSeverity, SilverSpine, SilverSpineProducers, + StreamProtocol, SyncNeed, SyncUpdate, TCacheRead, TProducer, TRandomAccess, TRead, Wheel, column_util::{self as util, KzgScratch}, ssz_view::{NUMBER_OF_COLUMNS, SignedBeaconBlockView, StatusView}, ticker::SlotTicker, @@ -521,6 +521,11 @@ impl DataColumnsTile { } fn resolve_validated(&mut self, mut p: PendingKzg, producers: &mut SilverSpineProducers) { + let column = GossipDataColumn { + slot: p.slot, + block_root: p.block_root, + column_index: p.column_index, + }; match mem::replace(&mut p.relay, RelayMeta::None) { RelayMeta::Gossip { topic, msg_hash, recv_ts, protobuf } => { producers.produce(PeerEvent::SendGossip { @@ -529,7 +534,7 @@ impl DataColumnsTile { msg_hash, recv_ts, protobuf, - block: None, + metadata: Some(GossipMetadata::DataColumn(column)), }); } RelayMeta::Rpc { ssz } if self.sync_state.is_synced() => { @@ -537,6 +542,7 @@ impl DataColumnsTile { originator: p.stream_id, topic: GossipTopic::DataColumnSidecar(p.column_index), ssz, + column, }); } _ => {} @@ -774,8 +780,8 @@ mod tests { use silver_beacon_state_data::{BeaconState, BeaconStateOwner}; use silver_common::{ - BlockSource, BlockStage, EngineReq, P2pStreamId, StreamProtocol, TCache, TCacheProducer, - TCacheRead, + BlockSource, BlockStage, EngineGetBlobsResp, EngineReq, MessageId, Nanos, P2pStreamId, + StreamProtocol, TCache, TCacheProducer, TCacheRead, ssz_view::{ DATA_COLUMN_SIDECAR_MIN, DataColumnSidecarFuluView, NUMBER_OF_COLUMNS, SIGNED_BEACON_BLOCK_MIN, @@ -785,6 +791,8 @@ mod tests { use super::*; + mod publication; + const CUSTODY_COLUMNS: u128 = (1u128 << 3) | (1u128 << 7); /// A tile on its own spine, with an injector adapter to read what it @@ -795,6 +803,9 @@ mod tests { inj: SpineAdapter, conn: SpineAdapter, tile: DataColumnsTile, + gossip_p: TProducer, + rpc_p: TProducer, + engine_p: TProducer, _spine: Box, _dir: TempDir, } @@ -807,31 +818,38 @@ mod tests { impl Rig { fn new(custody: u128) -> Self { - Self::with_state( - custody, - BeaconStateOwner::empty_test(0).reader(), - SpecConfig::mainnet(), - ) + Self::with_spec(custody, SpecConfig::mainnet()) + } + + fn gloas(custody: u128) -> Self { + Self::with_spec(custody, SpecConfig { gloas_fork_epoch: 0, ..SpecConfig::mainnet() }) + } + + fn with_spec(custody: u128, spec: SpecConfig) -> Self { + let mut state = BeaconStateOwner::empty_test(0); + let anchor = state.roll_fresh(); + state.publish_state_id(anchor); + Self::with_state(custody, state.reader(), spec) } fn with_state(custody: u128, beacon_state: BeaconStateReader, spec: SpecConfig) -> Self { - let gossip_tc = TCache::producer("gossip_blocks", 1024 * 1024); - let gossip_consumer = gossip_tc.cache_ref().random_access("gossip_cons", true).unwrap(); + let gossip_p = TCache::producer("gossip_blocks", 1024 * 1024); + let gossip_consumer = gossip_p.cache_ref().random_access("gossip_cons", true).unwrap(); let persist_gossip_tc = TCache::producer("persist_gossip_blocks", 1024 * 1024); let persist_gossip_consumer = persist_gossip_tc.cache_ref().random_access("persist_gossip_cons", true).unwrap(); - let rpc_tc = TCache::producer("rpc_blocks", 1024 * 1024); - let rpc_consumer = rpc_tc.cache_ref().random_access("rpc_cons", true).unwrap(); + let rpc_p = TCache::producer("rpc_blocks", 1024 * 1024); + let rpc_consumer = rpc_p.cache_ref().random_access("rpc_cons", true).unwrap(); let persist_rpc_tc = TCache::producer("persist_rpc_blocks", 1024 * 1024); let persist_rpc_consumer = persist_rpc_tc.cache_ref().random_access("persist_rpc_cons", true).unwrap(); - let engine_resp_tc = TCache::producer("engine_resp", 1024 * 1024); + let engine_p = TCache::producer("engine_resp", 1024 * 1024); let engine_resp_consumer = - engine_resp_tc.cache_ref().random_access("engine_resp_cons", true).unwrap(); + engine_p.cache_ref().random_access("engine_resp_cons", true).unwrap(); let tile = DataColumnsTile::new( ColumnConsumers { @@ -856,14 +874,84 @@ mod tests { inj.consume(|_: DataColumnsEvent, _| {}); inj.consume(|_: SyncNeed, _| {}); inj.consume(|_: EngineReq, _| {}); - Self { inj, conn, tile, _spine: spine, _dir: dir } + inj.consume(|_: PeerEvent, _| {}); + Self { inj, conn, tile, gossip_p, rpc_p, engine_p, _spine: spine, _dir: dir } + } + + fn turn(&mut self) { + self.tile.loop_body(&mut self.conn); + } + + fn engine_blobs(&mut self, block_root: BlockRoot, slot: u64, frame: &[u8]) { + let data = tcache_write(&mut self.engine_p, frame); + self.inj.produce(EngineResp::GetBlobs(EngineGetBlobsResp { + block_root, + slot, + ok: true, + blobs_present: 1, + data, + })); + } + + fn follow(&mut self, head_root: BlockRoot) { + self.tile.sync_state.set_sync_target(SyncUpdate::Following); + let mut ssz = status_ssz(0); + ssz[44..76].copy_from_slice(&head_root); + self.tile.sync_state.update(ssz); + } + + fn gossip_sidecar(&mut self, index: u64, bytes: &[u8]) { + let ssz = tcache_write(&mut self.gossip_p, bytes); + let recv_ts = Nanos::now(); + let mut id = [0u8; 20]; + id.copy_from_slice(&bytes[..20]); + let gossip = NewGossipMsg { + stream_id: P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), + topic: GossipTopic::DataColumnSidecar(index), + msg_hash: MessageId { id }, + recv_ts, + ssz, + // Fixture bypass: networking is absent, so protobuf reuses the SSZ handle. + protobuf: ssz, + }; + self.tile.gossip_sidecar(index, gossip, &mut self.conn.producers); + } + + fn rpc_sidecar(&mut self, bytes: &[u8]) { + let ssz = tcache_write(&mut self.rpc_p, bytes); + let sidecar = self.tile.consumers.rpc.acquire(ssz); + self.tile.handle_data_column_sidecar( + PendingColumn { + stream_id: P2pStreamId::new( + 1, + 0, + StreamProtocol::DataColumnSidecarsByRange, + true, + ), + sidecar, + gossip_subnet: None, + recv_ts: IngestionTime::now(), + }, + RelayMeta::Rpc { ssz }, + &mut self.conn.producers, + ); + } + + fn block(&mut self, bytes: &[u8]) { + let ssz = tcache_write(&mut self.gossip_p, bytes); + let read = self.tile.consumers.gossip.acquire(ssz); + self.tile.handle_beacon_block( + read, + P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), + &mut self.conn.producers, + ); } fn drain(&mut self) -> Produced { let mut out = Produced::default(); - self.inj.consume(|ev: DataColumnsEvent, _| match ev { + self.inj.consume(|event: DataColumnsEvent, _| match event { DataColumnsEvent::Available { .. } => out.available += 1, - DataColumnsEvent::Persist { .. } => out.persisted += 1, + DataColumnsEvent::Persist { .. } => out.receipts.push(event), }); self.inj.consume(|need: SyncNeed, _| match need { SyncNeed::Missing { .. } => out.missing.push(need), @@ -873,19 +961,61 @@ mod tests { SyncNeed::BackfillPrefill(_) => {} }); self.inj.consume(|_: EngineReq, _| out.engine += 1); + self.inj.consume(|event: PeerEvent, _| match event { + PeerEvent::SendGossip { .. } | PeerEvent::PublishDataColumn { .. } => { + out.publications.push(event) + } + _ => {} + }); out } } + fn tcache_write(producer: &mut TProducer, bytes: &[u8]) -> TCacheRead { + let mut reservation = producer.reserve(bytes.len(), true).expect("tcache reserve"); + reservation.write_all(bytes).expect("tcache write"); + reservation.flush().expect("tcache flush"); + reservation.read() + } + #[derive(Default)] struct Produced { available: usize, custody_complete: usize, - persisted: usize, + receipts: Vec, + publications: Vec, engine: usize, missing: Vec, } + impl Produced { + fn column_publications(&self) -> Vec<(ColumnSource, GossipTopic, GossipDataColumn)> { + self.publications + .iter() + .map(|event| match *event { + PeerEvent::SendGossip { + metadata: Some(GossipMetadata::DataColumn(column)), + topic, + .. + } => (ColumnSource::Gossip, topic, column), + PeerEvent::PublishDataColumn { column, topic, .. } => { + (ColumnSource::Rpc, topic, column) + } + _ => panic!("expected a column publication, got {event:?}"), + }) + .collect() + } + + fn persisted(&self, root: BlockRoot, index: u64) -> bool { + self.receipts.iter().any(|event| { + matches!(event, + DataColumnsEvent::Persist { block_root, column_index, .. } + if *block_root == root && *column_index == index + ) + }) + } + } + /// Minimal fulu `SignedBeaconBlock` carrying blob commitments: message at /// offset 100, body at 184, commitments spanning body[400..500). fn blob_block_bytes(slot: u64) -> Vec { @@ -934,15 +1064,30 @@ mod tests { SpecConfig { fulu_fork_epoch: 0, ..SpecConfig::mainnet() } } - /// Decoded EF vector, `None` when the beacon state tile crate has not - /// fetched them (`make` in that crate's directory). - fn ef_vector(case: &str, file: &str) -> Option> { - let path = Path::new(env!("CARGO_MANIFEST_DIR")) - .join("../beacon_state/tile/consensus-spec-tests/tests/mainnet/fulu") - .join(case) - .join(file); - let compressed = std::fs::read(path).ok()?; - Some(snap::raw::Decoder::new().decompress_vec(&compressed).unwrap()) + fn ef_sidecar(case: &str) -> Option<(Vec, Vec)> { + let fixtures = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../beacon_state/tile/consensus-spec-tests/tests/mainnet/fulu"); + if !fixtures.try_exists().expect("check EF fixture directory") { + eprintln!("EF sidecar coverage unavailable: run just ef-tests-download"); + return None; + } + let directory = fixtures.join(case); + let sidecars: Vec<_> = std::fs::read_dir(&directory) + .expect("installed EF fixtures must contain the sidecar case") + .map(|entry| entry.expect("read EF case entry").path()) + .filter(|path| { + let name = path.file_name().unwrap().to_string_lossy(); + name.starts_with("data_column_sidecar_") && name.ends_with(".ssz_snappy") + }) + .collect(); + let [sidecar] = sidecars.as_slice() else { + panic!("expected one sidecar in {}", directory.display()); + }; + let decode = |path: &Path| { + let compressed = std::fs::read(path).expect("read EF fixture"); + snap::raw::Decoder::new().decompress_vec(&compressed).expect("decode EF fixture") + }; + Some((decode(sidecar), decode(&directory.join("state.ssz_snappy")))) } fn reader_over(state_ssz: &[u8]) -> BeaconStateReader { @@ -1003,50 +1148,41 @@ mod tests { // was built on. const CASE: &str = "networking/gossip_data_column_sidecar/pyspec_tests/\ gossip_data_column_sidecar__ignore_parent_not_seen"; - const SIDECAR: &str = "data_column_sidecar_\ - 0x30d93f0be7cac9f7481a5799ac6ec7c8b726e8ced6bf6aa1e5d01e794dfb741e.ssz_snappy"; - let Some(sidecar) = ef_vector(CASE, SIDECAR) else { return }; - let reader = reader_over(&ef_vector(CASE, "state.ssz_snappy").unwrap()); + let Some((sidecar, state)) = ef_sidecar(CASE) else { return }; + let reader = reader_over(&state); + let slot = DataColumnSidecarFuluView::slot(&sidecar); + let index = DataColumnSidecarFuluView::index(&sidecar); + let block_root = util::block_root_from_sidecar(&sidecar); let parent_root = *DataColumnSidecarFuluView::parent_root(&sidecar); - let parent_slot = DataColumnSidecarFuluView::slot(&sidecar) - 1; - let staged_parent = || block_received(BlockStage::AwaitData, parent_root, parent_slot); + let staged_parent = || block_received(BlockStage::AwaitData, parent_root, slot - 1); for parent_first in [false, true] { - let (mut consumer, ssz) = produce_block(&sidecar, "staged_parent_sidecar"); - let mut rig = Rig::with_state(CUSTODY_COLUMNS | 1, reader.clone(), fulu_from_genesis()); + let mut rig = Rig::with_state(1 << index, reader.clone(), fulu_from_genesis()); rig.tile.sync_state.set_sync_target(SyncUpdate::Following); rig.tile.sync_state.update(status_ssz(0)); - let read = consumer.acquire(ssz); if parent_first { rig.tile.handle_beacon_state_event(staged_parent(), &mut rig.conn.producers); } - rig.tile.data_columns( - PendingColumn { - stream_id: P2pStreamId::new( - 2, - 2, - StreamProtocol::DataColumnSidecarsByRoot, - false, - ), - sidecar: read, - gossip_subnet: None, - recv_ts: IngestionTime::now(), - }, - RelayMeta::None, - &mut rig.conn.producers, - ); + rig.gossip_sidecar(index, &sidecar); if !parent_first { rig.tile.handle_beacon_state_event(staged_parent(), &mut rig.conn.producers); } - if !rig.tile.kzg_batch.is_empty() { - rig.tile.flush_kzg_batch(&mut rig.conn.producers); - } + rig.turn(); let out = rig.drain(); - assert_eq!( - out.persisted, 1, - "parent_first={parent_first}: the sidecar is ours to keep" + if parent_first { + assert_eq!(out.column_publications(), [( + ColumnSource::Gossip, + GossipTopic::DataColumnSidecar(index), + GossipDataColumn { slot, block_root, column_index: index } + )]); + } else { + assert!(out.publications.is_empty(), "a buffered copy is not relayed"); + } + assert!( + out.persisted(block_root, index), + "parent_first={parent_first}: the sidecar was processed" ); } } @@ -1210,7 +1346,7 @@ mod tests { let out = rig.drain(); assert_eq!( - out.available + out.persisted + out.engine + out.missing.len(), + out.available + out.receipts.len() + out.engine + out.missing.len(), 0, "len {len}: a malformed block says nothing" ); @@ -1239,7 +1375,7 @@ mod tests { let out = rig.drain(); assert!(ret.is_none(), "no column tracking below the floor"); - assert_eq!(out.available + out.persisted + out.engine + out.missing.len(), 0); + assert_eq!(out.available + out.receipts.len() + out.engine + out.missing.len(), 0); } /// A sidecar this tile already validated is still offered to storage: @@ -1279,7 +1415,8 @@ mod tests { "{protocol:?}: a duplicate is never relayed" ); assert_eq!( - out.persisted, 1, + out.receipts.len(), + 1, "{protocol:?}: storage is the one that knows whether it landed" ); assert_eq!( diff --git a/crates/columns/src/tile/tests/publication.rs b/crates/columns/src/tile/tests/publication.rs new file mode 100644 index 00000000..c378b3ce --- /dev/null +++ b/crates/columns/src/tile/tests/publication.rs @@ -0,0 +1,308 @@ +use silver_common::{ + ssz_hash::kzg_commitments_inclusion_proof, + ssz_view::{BEACON_BLOCK_BODY_FIXED, DATA_COLUMN_SIDECAR_GLOAS_MIN, EXECUTION_PAYLOAD_BID_MIN}, +}; + +use super::*; + +struct BlockBlob { + commitment: [u8; 48], + blob: c_kzg::Blob, + cells: Box<[c_kzg::Cell; c_kzg::CELLS_PER_EXT_BLOB]>, + proofs: Box<[c_kzg::KzgProof; c_kzg::CELLS_PER_EXT_BLOB]>, +} + +impl BlockBlob { + /// A zero blob gives every column identical cells and proofs, hiding + /// swapped proofs. Counting field elements keeps columns + /// distinguishable. + fn counting() -> Self { + let settings = c_kzg::ethereum_kzg_settings(0); + let mut bytes = [0u8; c_kzg::BYTES_PER_BLOB]; + for (i, element) in bytes.chunks_exact_mut(32).enumerate() { + element[30..32].copy_from_slice(&(i as u16).to_be_bytes()); + } + let blob = c_kzg::Blob::new(bytes); + let commitment = settings.blob_to_kzg_commitment(&blob).unwrap().to_bytes().into_inner(); + let (cells, proofs) = settings.compute_cells_and_kzg_proofs(&blob).unwrap(); + Self { commitment, blob, cells, proofs } + } + + /// Matches the engine tile's `engine_getBlobsV2` tcache frame format. + fn el_frame(&self) -> Vec { + let mut out = 1u32.to_le_bytes().to_vec(); + out.push(1); + out.push(NUMBER_OF_COLUMNS as u8); + for proof in self.proofs.iter() { + out.extend_from_slice(&proof.to_bytes().into_inner()); + } + out.extend_from_slice(&(c_kzg::BYTES_PER_BLOB as u32).to_le_bytes()); + out.extend_from_slice(self.blob.as_ref()); + out + } + + fn fulu_sidecar(&self, index: u64, block: &[u8]) -> Vec { + let body = SignedBeaconBlockView::body(block); + let mut header = [0u8; 208]; + header[0..8].copy_from_slice(&SignedBeaconBlockView::slot(block).to_le_bytes()); + header[8..16].copy_from_slice(&SignedBeaconBlockView::proposer_index(block).to_le_bytes()); + header[16..48].copy_from_slice(SignedBeaconBlockView::parent_root(block)); + header[48..80].copy_from_slice(SignedBeaconBlockView::state_root(block)); + header[80..112].copy_from_slice(&util::body_root(body)); + + let mut out = Vec::with_capacity(util::data_column_sidecar_len(1)); + util::push_data_column_sidecar_prefix( + &mut out, + index, + 1, + &header, + &kzg_commitments_inclusion_proof(body), + ); + out.extend_from_slice(&self.cells[index as usize].to_bytes()); + out.extend_from_slice(&self.commitment); + out.extend_from_slice(&self.proofs[index as usize].to_bytes().into_inner()); + out + } + + fn gloas_sidecar(&self, index: u64, slot: u64, block_root: &BlockRoot) -> Vec { + self.gloas_sidecar_with_proofs(index, slot, block_root, index) + } + + fn gloas_sidecar_with_proofs( + &self, + index: u64, + slot: u64, + block_root: &BlockRoot, + proof_index: u64, + ) -> Vec { + let column = self.cells[index as usize].to_bytes(); + let mut out = vec![0u8; DATA_COLUMN_SIDECAR_GLOAS_MIN]; + out[0..8].copy_from_slice(&index.to_le_bytes()); + out[8..12].copy_from_slice(&(DATA_COLUMN_SIDECAR_GLOAS_MIN as u32).to_le_bytes()); + out[12..16].copy_from_slice( + &((DATA_COLUMN_SIDECAR_GLOAS_MIN + column.len()) as u32).to_le_bytes(), + ); + out[16..24].copy_from_slice(&slot.to_le_bytes()); + out[24..56].copy_from_slice(block_root); + out.extend_from_slice(&column); + out.extend_from_slice(&self.proofs[proof_index as usize].to_bytes().into_inner()); + out + } +} + +fn block_around(slot: u64, body: &[u8]) -> Vec { + let mut block = vec![0u8; SIGNED_BEACON_BLOCK_MIN + body.len()]; + block[0..4].copy_from_slice(&100u32.to_le_bytes()); + block[100..108].copy_from_slice(&slot.to_le_bytes()); + block[180..184].copy_from_slice(&84u32.to_le_bytes()); + block[SIGNED_BEACON_BLOCK_MIN..].copy_from_slice(body); + block +} + +fn fulu_body(commitments: &[u8]) -> Vec { + const FIXED: usize = BEACON_BLOCK_BODY_FIXED; + let mut body = vec![0u8; FIXED + commitments.len()]; + for off in [200usize, 204, 208, 212, 216, 380, 384, 388] { + body[off..off + 4].copy_from_slice(&(FIXED as u32).to_le_bytes()); + } + body[392..396].copy_from_slice(&((FIXED + commitments.len()) as u32).to_le_bytes()); + body[FIXED..].copy_from_slice(commitments); + body +} + +/// Gloas carries commitments in the payload bid. +fn gloas_body(commitments: &[u8]) -> Vec { + const FIXED: usize = BEACON_BLOCK_BODY_FIXED; + let mut bid = vec![0u8; EXECUTION_PAYLOAD_BID_MIN + commitments.len()]; + bid[188..192].copy_from_slice(&(EXECUTION_PAYLOAD_BID_MIN as u32).to_le_bytes()); + bid[EXECUTION_PAYLOAD_BID_MIN..].copy_from_slice(commitments); + + let mut signed_bid = vec![0u8; 100]; + signed_bid[0..4].copy_from_slice(&100u32.to_le_bytes()); + signed_bid.extend_from_slice(&bid); + + let end = FIXED + signed_bid.len(); + let mut body = vec![0u8; end]; + for off in [200usize, 204, 208, 212, 216, 380, 384] { + body[off..off + 4].copy_from_slice(&(FIXED as u32).to_le_bytes()); + } + for off in [388usize, 392] { + body[off..off + 4].copy_from_slice(&(end as u32).to_le_bytes()); + } + body[FIXED..].copy_from_slice(&signed_bid); + body +} + +impl Rig { + fn receive_column(&mut self, source: ColumnSource, index: u64, bytes: &[u8]) { + match source { + ColumnSource::Gossip => self.gossip_sidecar(index, bytes), + ColumnSource::Rpc => self.rpc_sidecar(bytes), + ColumnSource::El => unreachable!(), + } + } +} + +#[test] +fn column_publications_carry_metadata_for_gossip_and_following_rpc() { + const SLOT: u64 = 40; + let blob = BlockBlob::counting(); + let block = block_around(SLOT, &gloas_body(&blob.commitment)); + let block_root = util::block_root_gloas(&block); + for (source, following, index) in [ + (ColumnSource::Gossip, true, 3), + (ColumnSource::Gossip, true, 5), + (ColumnSource::Rpc, true, 3), + (ColumnSource::Rpc, true, 5), + (ColumnSource::Rpc, false, 3), + ] { + let mut rig = Rig::gloas(CUSTODY_COLUMNS); + if following { + rig.follow([0xAA; 32]); + } + rig.block(&block); + rig.drain(); + rig.receive_column(source, index, &blob.gloas_sidecar(index, SLOT, &block_root)); + rig.turn(); + let out = rig.drain(); + if following { + let column = GossipDataColumn { slot: SLOT, block_root, column_index: index }; + assert_eq!(out.column_publications(), [( + source, + GossipTopic::DataColumnSidecar(index), + column + )]); + } else { + assert!(out.persisted(block_root, index), "syncing still processes the column"); + assert!(out.publications.is_empty(), "syncing RPC columns do not request publication"); + } + } +} + +#[test] +fn fulu_column_publication_requires_a_resolved_proposer() { + let blob = BlockBlob::counting(); + // The empty state's lookahead covers the current and next epochs. + for (slot, relay_eligible) in [(7, true), (2 * SLOTS_PER_EPOCH + 1, false)] { + let block = block_around(slot, &fulu_body(&blob.commitment)); + let block_root = util::block_root_fulu(&block); + let mut rig = Rig::new(CUSTODY_COLUMNS); + rig.follow(*SignedBeaconBlockView::parent_root(&block)); + let sidecar = blob.fulu_sidecar(3, &block); + // Fixture bypass: the empty validator registry cannot verify signatures. + // This isolates proposer eligibility; the staged-parent case uses signed data. + rig.tile + .tracker + .set_signature(block_root, *DataColumnSidecarFuluView::block_signature(&sidecar)); + rig.gossip_sidecar(3, &sidecar); + rig.turn(); + let out = rig.drain(); + if relay_eligible { + let column = GossipDataColumn { slot, block_root, column_index: 3 }; + assert_eq!(out.column_publications(), [( + ColumnSource::Gossip, + GossipTopic::DataColumnSidecar(3), + column + )]); + } else { + assert!( + out.persisted(block_root, 3), + "an unresolved proposer does not prevent storage" + ); + assert!(out.publications.is_empty(), "an unresolved proposer prevents relay"); + } + } +} + +#[test] +fn held_columns_do_not_request_publication_again() { + const SLOT: u64 = 40; + let blob = BlockBlob::counting(); + let block = block_around(SLOT, &gloas_body(&blob.commitment)); + let block_root = util::block_root_gloas(&block); + let mut rig = Rig::gloas(CUSTODY_COLUMNS); + rig.follow([0xAA; 32]); + rig.block(&block); + rig.drain(); + let sidecar = blob.gloas_sidecar(3, SLOT, &block_root); + rig.gossip_sidecar(3, &sidecar); + rig.turn(); + let column = GossipDataColumn { slot: SLOT, block_root, column_index: 3 }; + assert_eq!(rig.drain().column_publications(), [( + ColumnSource::Gossip, + GossipTopic::DataColumnSidecar(3), + column + )]); + + for source in [ColumnSource::Gossip, ColumnSource::Rpc] { + rig.receive_column(source, 3, &sidecar); + rig.turn(); + assert!(rig.drain().publications.is_empty(), "{source:?}: a held copy is not republished"); + } +} + +#[test] +fn only_columns_with_valid_kzg_proofs_request_publication() { + const SLOT: u64 = 40; + let blob = BlockBlob::counting(); + let block = block_around(SLOT, &gloas_body(&blob.commitment)); + let block_root = util::block_root_gloas(&block); + let mut rig = Rig::gloas(CUSTODY_COLUMNS); + rig.follow([0xAA; 32]); + rig.block(&block); + rig.drain(); + rig.gossip_sidecar(3, &blob.gloas_sidecar(3, SLOT, &block_root)); + // Column 7 carries column 6's proofs: structural checks pass, KZG fails. + rig.gossip_sidecar(7, &blob.gloas_sidecar_with_proofs(7, SLOT, &block_root, 6)); + rig.turn(); + let column = GossipDataColumn { slot: SLOT, block_root, column_index: 3 }; + assert_eq!(rig.drain().column_publications(), [( + ColumnSource::Gossip, + GossipTopic::DataColumnSidecar(3), + column + )]); +} + +#[test] +fn buffered_gloas_columns_are_processed_without_publication() { + const SLOT: u64 = 40; + let blob = BlockBlob::counting(); + let block = block_around(SLOT, &gloas_body(&blob.commitment)); + let block_root = util::block_root_gloas(&block); + for source in [ColumnSource::Gossip, ColumnSource::Rpc] { + let mut rig = Rig::gloas(CUSTODY_COLUMNS); + rig.follow([0xAA; 32]); + rig.receive_column(source, 3, &blob.gloas_sidecar(3, SLOT, &block_root)); + rig.turn(); + assert!(rig.drain().publications.is_empty()); + rig.block(&block); + rig.turn(); + let out = rig.drain(); + assert!(out.persisted(block_root, 3), "the buffered column was processed"); + assert!(out.publications.is_empty(), "processing a buffered copy does not request relay"); + } +} + +#[test] +fn reconstructed_columns_do_not_request_publication() { + const SLOT: u64 = 40; + let blob = BlockBlob::counting(); + let block = block_around(SLOT, &fulu_body(&blob.commitment)); + let block_root = util::block_root_fulu(&block); + let mut rig = Rig::new(CUSTODY_COLUMNS); + rig.turn(); + rig.follow([0xAA; 32]); + rig.block(&block); + rig.drain(); + rig.engine_blobs(block_root, SLOT, &blob.el_frame()); + rig.turn(); + let out = rig.drain(); + assert!( + out.receipts.iter().any(|event| matches!(event, + DataColumnsEvent::Persist { source: ColumnSource::El, block_root: root, .. } + if *root == block_root + )), + "the EL response produced a reconstructed column" + ); + assert!(out.publications.is_empty()); +} diff --git a/crates/common/src/lib.rs b/crates/common/src/lib.rs index 045d723b..937794bf 100644 --- a/crates/common/src/lib.rs +++ b/crates/common/src/lib.rs @@ -23,17 +23,17 @@ pub use crate::{ EngineGetPayloadBodiesByRangeReq, EngineGetPayloadBodiesResp, EngineGetPayloadReq, EngineGetPayloadResp, EngineHealthEvent, EngineNewPayloadEnvelopeReq, EngineNewPayloadReq, EngineNewPayloadResp, EnginePreparePayloadReq, EngineReq, EngineResp, Error as TCacheError, - GossipBlock, GossipMsgIn, GossipMsgOut, IpBytes, LOCAL_GOSSIP_STREAM_ID, - LocalAttestationFailure, LocalAttestationResult, MAX_BLOBS_PER_BLOCK, - MAX_PAYLOAD_BODIES_PER_REQ, MULTISTREAM_V1, MultiProducer as TMultiProducer, NewGossipMsg, - P2pConnectionStats, P2pSend, P2pStreamId, PREFILL_SLOTS, PayloadValidationStatus, - PeerControl, PeerEvent, PeerScores, PeerStats, PeerStatus, PeerTopicScores, Prefill, - Producer as TProducer, REJECT_RESPONSE, RPC_PROTOCOLS, - RandomAccessConsumer as TRandomAccess, ReplayBlock, Reservation as TReservation, - RpcInbound, RpcOutbound, RpcRequest, RpcRequestInbound, RpcRequestOutbound, RpcResponse, - RpcResponseInbound, RpcResponseOutbound, RpcSeverity, SilverSpine, SilverSpineProducers, - StreamProtocol, SyncNeed, SyncUpdate, SyncingStrategy, TCache, TCacheProducer, TCacheRead, - TCacheRef, WithdrawalInline, + GossipBlock, GossipDataColumn, GossipMetadata, GossipMsgIn, GossipMsgOut, IpBytes, + LOCAL_GOSSIP_STREAM_ID, LocalAttestationFailure, LocalAttestationResult, + MAX_BLOBS_PER_BLOCK, MAX_PAYLOAD_BODIES_PER_REQ, MULTISTREAM_V1, + MultiProducer as TMultiProducer, NewGossipMsg, P2pConnectionStats, P2pSend, P2pStreamId, + PREFILL_SLOTS, PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, + PeerStatus, PeerTopicScores, Prefill, Producer as TProducer, REJECT_RESPONSE, + RPC_PROTOCOLS, RandomAccessConsumer as TRandomAccess, ReplayBlock, + Reservation as TReservation, RpcInbound, RpcOutbound, RpcRequest, RpcRequestInbound, + RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, RpcSeverity, + SilverSpine, SilverSpineProducers, StreamProtocol, SyncNeed, SyncUpdate, SyncingStrategy, + TCache, TCacheProducer, TCacheRead, TCacheRef, WithdrawalInline, }, util::{create_self_signed_certificate, decode_varint, encode_varint, hex32}, wheel::Wheel, diff --git a/crates/common/src/spine.rs b/crates/common/src/spine.rs index b391a05d..fc407147 100644 --- a/crates/common/src/spine.rs +++ b/crates/common/src/spine.rs @@ -7,13 +7,13 @@ pub use messages::{ EngineGetBlobsResp, EngineGetPayloadBodiesByHashReq, EngineGetPayloadBodiesByRangeReq, EngineGetPayloadBodiesResp, EngineGetPayloadReq, EngineGetPayloadResp, EngineHealthEvent, EngineNewPayloadEnvelopeReq, EngineNewPayloadReq, EngineNewPayloadResp, - EnginePreparePayloadReq, EngineReq, EngineResp, GossipBlock, GossipMsgIn, GossipMsgOut, - IpBytes, LocalAttestationFailure, LocalAttestationResult, MAX_BLOBS_PER_BLOCK, - MAX_PAYLOAD_BODIES_PER_REQ, NewGossipMsg, P2pConnectionStats, P2pSend, PREFILL_SLOTS, - PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, PeerStatus, - PeerTopicScores, Prefill, ReplayBlock, RpcInbound, RpcOutbound, RpcRequest, RpcRequestInbound, - RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, RpcSeverity, - SyncNeed, SyncUpdate, SyncingStrategy, WithdrawalInline, + EnginePreparePayloadReq, EngineReq, EngineResp, GossipBlock, GossipDataColumn, GossipMetadata, + GossipMsgIn, GossipMsgOut, IpBytes, LocalAttestationFailure, LocalAttestationResult, + MAX_BLOBS_PER_BLOCK, MAX_PAYLOAD_BODIES_PER_REQ, NewGossipMsg, P2pConnectionStats, P2pSend, + PREFILL_SLOTS, PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, + PeerStatus, PeerTopicScores, Prefill, ReplayBlock, RpcInbound, RpcOutbound, RpcRequest, + RpcRequestInbound, RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, + RpcSeverity, SyncNeed, SyncUpdate, SyncingStrategy, WithdrawalInline, }; pub use stream_id::{LOCAL_GOSSIP_STREAM_ID, P2pStreamId}; pub use stream_protocol::{ diff --git a/crates/common/src/spine/messages.rs b/crates/common/src/spine/messages.rs index ac3e6c66..40a618f8 100644 --- a/crates/common/src/spine/messages.rs +++ b/crates/common/src/spine/messages.rs @@ -325,6 +325,21 @@ pub struct GossipBlock { pub block_root: B256, } +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(C)] +pub struct GossipDataColumn { + pub slot: u64, + pub block_root: B256, + pub column_index: u64, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(C, u8)] +pub enum GossipMetadata { + Block(GossipBlock), + DataColumn(GossipDataColumn), +} + #[derive(Clone, Copy, Debug)] #[repr(C, u8)] #[allow(clippy::large_enum_variant)] @@ -455,15 +470,12 @@ pub enum PeerEvent { p2p_peer: usize, iwant: TCacheRead, }, - /// A data column sidecar validated from a non-gossip source (RPC - /// by-root / EL blobs). Control re-publishes it on its subnet: the - /// gossip handler wraps the SSZ (a ref into `incoming_rpc`) as - /// protobuf and PM fans it out to the topic mesh, excluding - /// `originator` (the peer that served it to us). + /// The SSZ handle refers to incoming RPC bytes, before gossip encoding. PublishDataColumn { originator: P2pStreamId, topic: GossipTopic, ssz: TCacheRead, + column: GossipDataColumn, }, /// Emitted in order to trigger sending of a gossip message. /// Peer manager will generate select peers to send to. @@ -473,7 +485,7 @@ pub enum PeerEvent { msg_hash: MessageId, recv_ts: Nanos, protobuf: TCacheRead, - block: Option, + metadata: Option, }, /// Misbehaviour observed on the RPC (req/resp) sub-protocol. The peer /// manager translates `severity` into a P5 application-score delta; diff --git a/crates/control/src/tile.rs b/crates/control/src/tile.rs index 199b2b1b..38a1bcbf 100644 --- a/crates/control/src/tile.rs +++ b/crates/control/src/tile.rs @@ -2,9 +2,10 @@ use std::time::{Duration, Instant}; use flux::{spine::SpineAdapter, tile::Tile}; use silver_common::{ - BeaconStateEvent, GossipTopic, Nanos, P2pSend, PeerControl, PeerEvent, PeerStats, RpcInbound, - RpcOutbound, RpcRequest, RpcRequestOutbound, RpcResponse, RpcResponseInbound, SilverSpine, - SilverSpineProducers, SyncNeed, SyncUpdate, TMultiProducer, TRandomAccess, + BeaconStateEvent, GossipMetadata, GossipTopic, Nanos, P2pSend, PeerControl, PeerEvent, + PeerStats, RpcInbound, RpcOutbound, RpcRequest, RpcRequestOutbound, RpcResponse, + RpcResponseInbound, SilverSpine, SilverSpineProducers, SyncNeed, SyncUpdate, TMultiProducer, + TRandomAccess, ssz_view::{METADATA_SIZE, STATUS_V2_SIZE, StatusView}, }; use silver_gossip::{GossipHandler, GossipHandlerEvent}; @@ -173,7 +174,7 @@ impl Tile for Controller { } adapter.consume(|event: PeerEvent, producers| { - if let PeerEvent::PublishDataColumn { originator, topic, ssz } = event { + if let PeerEvent::PublishDataColumn { originator, topic, ssz, column } = event { let read = self.rpc_ssz_consumer.acquire(ssz); match read.buffer() { Ok((bytes, _)) => { @@ -187,7 +188,7 @@ impl Tile for Controller { msg_hash, recv_ts: Nanos::now(), protobuf, - block: None, + metadata: Some(GossipMetadata::DataColumn(column)), }, now, &mut |evt| { @@ -214,7 +215,7 @@ impl Tile for Controller { msg_hash, recv_ts: _, protobuf, - block: _, + metadata: _, } = &event { self.gossip_handler.mcache_insert(*msg_hash, *topic, *protobuf); diff --git a/crates/control/src/tile/tests.rs b/crates/control/src/tile/tests.rs index de512aa2..0faa1fe5 100644 --- a/crates/control/src/tile/tests.rs +++ b/crates/control/src/tile/tests.rs @@ -8,8 +8,8 @@ use std::{ use silver_chain_spec::SpecConfig; use silver_common::{ - GossipBlock, GossipMsgIn, GossipMsgOut, IpBytes, Keypair, MessageId, P2pStreamId, PeerId, - StreamProtocol, TCache, TCacheProducer, TCacheRead, TProducer, + GossipBlock, GossipDataColumn, GossipMsgIn, GossipMsgOut, IpBytes, Keypair, MessageId, + P2pStreamId, PeerId, StreamProtocol, TCache, TCacheProducer, TCacheRead, TProducer, }; use silver_peer::SyncingConfig; @@ -20,6 +20,7 @@ struct GossipPublications { adapter: SpineAdapter, observer: SpineAdapter, incoming: TProducer, + rpc: TProducer, payload: TCacheRead, outbound: TRandomAccess, _spine: Box, @@ -71,7 +72,7 @@ impl GossipPublications { let mut observer = SpineAdapter::connect_tile(&Observer, &mut spine); observer.consume(|_: P2pSend, _| {}); let mut capture = - Self { controller, adapter, observer, incoming, payload, outbound, _spine: spine }; + Self { controller, adapter, observer, incoming, rpc, payload, outbound, _spine: spine }; capture.crank(); for peer in 1..=2u8 { capture.observer.produce(PeerEvent::P2pNewConnection { @@ -130,15 +131,29 @@ fn relay_metadata_preserves_routing_and_iwant_service() { // Forwarding and IWANT service treat the payload as opaque bytes. let bytes = b"relay payload"; let hash = MessageId { id: [0xCD; 20] }; - for block in [None, Some(GossipBlock { slot: 37, block_root: [0xAB; 32] })] { - let mut capture = GossipPublications::new(GossipTopic::BeaconBlock, bytes); + for (topic, metadata) in [ + (GossipTopic::BeaconBlock, None), + ( + GossipTopic::BeaconBlock, + Some(GossipMetadata::Block(GossipBlock { slot: 37, block_root: [0xAB; 32] })), + ), + ( + GossipTopic::DataColumnSidecar(5), + Some(GossipMetadata::DataColumn(GossipDataColumn { + slot: 38, + block_root: [0xCD; 32], + column_index: 5, + })), + ), + ] { + let mut capture = GossipPublications::new(topic, bytes); capture.observer.produce(PeerEvent::SendGossip { originator_stream_id: P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), - topic: GossipTopic::BeaconBlock, + topic, msg_hash: hash, recv_ts: Nanos::now(), protobuf: capture.payload, - block, + metadata, }); capture.crank(); assert_eq!(capture.sent(), [(2, bytes.to_vec())], "the sender is excluded"); @@ -151,3 +166,59 @@ fn relay_metadata_preserves_routing_and_iwant_service() { ); } } + +#[test] +fn column_publication_encodes_and_routes_without_another_spine_request() { + let topic = GossipTopic::DataColumnSidecar(5); + let column = GossipDataColumn { slot: 38, block_root: [0xCD; 32], column_index: 5 }; + // Transport decoding checks payload size; consensus validation is outside this + // fixture. + let bytes = vec![0x42; topic.min_uncompressed_size()]; + let mut capture = GossipPublications::new(topic, &[]); + capture.observer.consume(|_: PeerEvent, _| {}); + let ssz = write_bytes(&mut capture.rpc, &bytes); + capture.observer.produce(PeerEvent::PublishDataColumn { + originator: P2pStreamId::new(1, 0, StreamProtocol::DataColumnSidecarsByRoot, true), + topic, + ssz, + column, + }); + capture.crank(); + let sent = capture.sent(); + let [(peer, encoded)] = sent.as_slice() else { panic!("expected one gossip publication") }; + assert_eq!(*peer, 2, "the sender is excluded"); + + let decoded = TCache::producer("publication_decoded", 1 << 16); + let mut decoded_reader = + decoded.cache_ref().random_access("publication_decoded", true).unwrap(); + let mut receiver = GossipHandler::new( + capture.incoming.cache_ref().random_access("publication_receiver", true).unwrap(), + decoded, + TCache::producer("publication_received_protobuf", 1 << 16), + "00000000".to_owned(), + ) + .unwrap(); + let mut receiver_adapter = SpineAdapter::connect_tile(&Observer, &mut capture._spine); + receiver.spin(&mut receiver_adapter); + let stream = P2pStreamId::new(2, 0, StreamProtocol::GossipSub, true); + let mut packet = stream.as_ref().to_vec(); + packet.extend_from_slice(encoded); + let tcache = write_bytes(&mut capture.incoming, &packet); + capture.observer.produce(GossipMsgIn { p2p_id: stream, tcache }); + receiver.spin(&mut receiver_adapter); + let Some(GossipHandlerEvent::NewGossip(message)) = receiver.pop_event() else { + panic!("publication must decode as a gossip message") + }; + assert_eq!(message.topic, topic); + assert_eq!(decoded_reader.acquire(message.ssz).buffer().unwrap().0, bytes); + + capture.iwant(1, message.msg_hash); + assert_eq!(capture.sent(), [(1, encoded.clone())]); + let mut publications = 0; + capture.observer.consume(|event: PeerEvent, _| match event { + PeerEvent::PublishDataColumn { .. } => publications += 1, + PeerEvent::SendGossip { .. } => panic!("column conversion must stay local"), + _ => {} + }); + assert_eq!(publications, 1); +} diff --git a/crates/peer/src/manager/mod.rs b/crates/peer/src/manager/mod.rs index 81af2965..9ab5f959 100644 --- a/crates/peer/src/manager/mod.rs +++ b/crates/peer/src/manager/mod.rs @@ -424,7 +424,7 @@ impl PeerManager { msg_hash, recv_ts: _, protobuf, - block: _, + metadata: _, } => { // TODO recv_ts elapsed metric self.on_send_gossip(originator_stream_id.peer(), msg_hash, topic, protobuf, emit); diff --git a/crates/peer/src/manager/promises.rs b/crates/peer/src/manager/promises.rs index a024e592..7630d322 100644 --- a/crates/peer/src/manager/promises.rs +++ b/crates/peer/src/manager/promises.rs @@ -342,7 +342,7 @@ impl PeerManager { mod tests { use std::time::Duration; - use silver_common::{GossipBlock, PeerEvent, TCacheProducer}; + use silver_common::{GossipBlock, GossipMetadata, PeerEvent, TCacheProducer}; use silver_config::ScoreParams; use super::*; @@ -1026,7 +1026,10 @@ mod tests { msg_hash: hash, recv_ts: silver_common::Nanos::now(), protobuf: mk_tcache_read(), - block: Some(GossipBlock { slot: 37, block_root: [0xAB; 32] }), + metadata: Some(GossipMetadata::Block(GossipBlock { + slot: 37, + block_root: [0xAB; 32], + })), }, now, &mut |c| cap.0.push(c), @@ -1080,7 +1083,7 @@ mod tests { msg_hash: silver_common::MessageId { id: [0xCD; 20] }, recv_ts: silver_common::Nanos::now(), protobuf: mk_tcache_read(), - block: None, + metadata: None, }, now, &mut |event| cap.0.push(event), From 363c765a7bbd32970c9add1afec21eaea79527c4 Mon Sep 17 00:00:00 2001 From: Bronek Kozicki Date: Thu, 10 Sep 2026 20:13:10 +0100 Subject: [PATCH 2/5] Serve data_column_sidecar events over SSE Add the data_column_sidecar topic, renderer, and BeaconApi publication method. Emit block_root as lowercase hex and index and slot as decimal strings. Publish column metadata from SendGossip and PublishDataColumn through the existing peer-event consumer, outside the engine capacity gate. Select metadata or request kind without reading payload bytes or repeating the producer's topic check. SSE acknowledges publication requests following silver's gossip checks, including KZG; it does not guarantee delivery to peers. The ADR amendment records this narrower validation contract, excluded paths, repeated requests, and no replay. No additional queue or spine publication is introduced. Extend the existing topic parser test, retaining repeated parameters and URL-encoded commas. The renderer test checks required JSON fields with distinct slot and index values, allowing additional fields and different formatting. Extend three existing socket scenarios through spine injection and ApplicationBoundaryTile::loop_body. They cover both column request variants, topic isolation, repeated requests, late subscriptions, and delivery during engine saturation. Mixed subscriptions preserve each topic's sequence without requiring an order between topics. Trailing sentinels keep unexpected frames inside the observations. The saturation fixture fills the configured pool with its own unanswered FCU requests. Publication requests carry opaque fixture bytes; these socket tests do not establish validation or network delivery. Fixtures keep topic and metadata consistent, so they cannot distinguish an additional topic guard. The changed tests passed with reordered JSON keys, extra whitespace, an additional nested field, and the peer-event drain preceding the beacon-event drain. Those temporary changes were restored. The full control runs encountered SIGBUS failures in other tests at default concurrency; no negative controls or size measurements were repeated. Final gates: just fmt-check, just clippy, git diff --check, and NEXTEST_TEST_THREADS=4 just nextest exited 0. The workspace passed 1,369 tests and skipped five. Assisted-by: Codex:gpt-6-astra --- crates/application_boundary/src/lib.rs | 18 +- crates/application_boundary/tests/tile.rs | 238 ++++++++++++++++------ crates/beacon_api/src/events.rs | 19 +- crates/beacon_api/src/json.rs | 25 +++ crates/beacon_api/src/server.rs | 11 + docs/adr/0004-sync-materialized-api.md | 16 ++ 6 files changed, 257 insertions(+), 70 deletions(-) diff --git a/crates/application_boundary/src/lib.rs b/crates/application_boundary/src/lib.rs index 3bac9be1..e8f76373 100644 --- a/crates/application_boundary/src/lib.rs +++ b/crates/application_boundary/src/lib.rs @@ -99,14 +99,20 @@ impl ApplicationBoundaryTile { } => beacon.publish_block(slot, &block_root), _ => {} }); - adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { + adapter.consume(|event: PeerEvent, _| match event { + PeerEvent::SendGossip { metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root })), .. - } = event - { - beacon.publish_block_gossip(slot, &block_root); - } + } => beacon.publish_block_gossip(slot, &block_root), + PeerEvent::SendGossip { + metadata: Some(GossipMetadata::DataColumn(column)), .. + } | + PeerEvent::PublishDataColumn { column, .. } => beacon.publish_data_column_sidecar( + &column.block_root, + column.column_index, + column.slot, + ), + _ => {} }); let status = beacon.node_status_mut(); adapter.consume(|update: SyncUpdate, _| { diff --git a/crates/application_boundary/tests/tile.rs b/crates/application_boundary/tests/tile.rs index 40ac8519..c6ec5ba2 100644 --- a/crates/application_boundary/tests/tile.rs +++ b/crates/application_boundary/tests/tile.rs @@ -8,7 +8,7 @@ use std::{ }; use flux::{spine::SpineAdapter, tile::Tile, timing::Nanos}; -use serde_json::Value; +use serde_json::{Value, json}; use silver_application_boundary::ApplicationBoundaryTile; use silver_beacon_api::SlotStatus; use silver_beacon_state_data::{BeaconStateOwner, SpecConfig}; @@ -16,7 +16,7 @@ use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ELSyncStatus, EngineFcuReq, EngineReq, EngineResp, Enr, GossipBlock, GossipDataColumn, GossipMetadata, GossipTopic, Identify, Keypair, MessageId, P2pStreamId, PayloadValidationStatus, PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, - TCache, TCacheProducer, ssz_view::STATUS_V2_SIZE, + TCache, TCacheProducer, TCacheRead, ssz_view::STATUS_V2_SIZE, }; use silver_config::EngineConfig; use silver_engine_api::test_el::{FCU_VALID_RESULT, FakeEl, write_jwt}; @@ -147,24 +147,48 @@ fn block_received(slot: u64, byte: u8, stage: BlockStage) -> BeaconStateEvent { } } -fn block_relay(slot: u64, byte: u8) -> PeerEvent { +fn publication_payload() -> TCacheRead { // SSE uses the metadata, so the fixture needs no encoded gossip object. let payload = b"opaque relay payload"; let mut producer = TCache::producer("cs_relay_metadata", 1 << 12); let mut reservation = producer.reserve(payload.len(), false).unwrap(); reservation.write_all(payload).unwrap(); reservation.flush().unwrap(); - let protobuf = reservation.read(); + reservation.read() +} + +fn block_relay(slot: u64, byte: u8) -> PeerEvent { PeerEvent::SendGossip { originator_stream_id: P2pStreamId::new(0, 0, StreamProtocol::GossipSub, false), topic: GossipTopic::BeaconBlock, msg_hash: MessageId { id: [byte; 20] }, recv_ts: Nanos::now(), - protobuf, + protobuf: publication_payload(), metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root: [byte; 32] })), } } +fn column_publications(slot: u64, byte: u8, column_index: u64) -> [PeerEvent; 2] { + let column = GossipDataColumn { slot, block_root: [byte; 32], column_index }; + let topic = GossipTopic::DataColumnSidecar(column_index); + [ + PeerEvent::SendGossip { + originator_stream_id: P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), + topic, + msg_hash: MessageId { id: [byte; 20] }, + recv_ts: Nanos::now(), + protobuf: publication_payload(), + metadata: Some(GossipMetadata::DataColumn(column)), + }, + PeerEvent::PublishDataColumn { + originator: P2pStreamId::new(2, 0, StreamProtocol::DataColumnSidecarsByRange, true), + topic, + ssz: publication_payload(), + column, + }, + ] +} + #[derive(Debug)] struct SseEvent { name: String, @@ -172,6 +196,34 @@ struct SseEvent { } impl SseEvent { + fn block(slot: u64, byte: u8) -> Self { + Self { + name: "block".to_owned(), + data: json!({"slot": slot.to_string(), "block": format!("0x{}", hex::encode([byte; 32]))}), + } + } + + fn block_gossip(slot: u64, byte: u8) -> Self { + Self { + name: "block_gossip".to_owned(), + data: json!({"slot": slot.to_string(), "block": format!("0x{}", hex::encode([byte; 32]))}), + } + } + + fn column(slot: u64, byte: u8, index: u64) -> Self { + Self { + name: "data_column_sidecar".to_owned(), + data: json!({"block_root": format!("0x{}", hex::encode([byte; 32])), "index": index.to_string(), "slot": slot.to_string()}), + } + } + + fn assert_matches(&self, expected: &Self) { + assert_eq!(self.name, expected.name); + for (key, value) in expected.data.as_object().unwrap() { + assert_eq!(self.data.get(key), Some(value), "field {key} in {}", self.name); + } + } + fn assert_block(&self, name: &str, slot: u64, byte: u8) { assert_eq!(self.name, name); assert_eq!(self.data["slot"], slot.to_string()); @@ -235,6 +287,18 @@ impl EventsSubscriber { fn next(&self, pump: impl FnMut()) -> SseEvent { receive_while_pumping(&self.events, pump) } + + fn assert_topic_sequences(&self, expected: &[SseEvent], mut pump: impl FnMut()) { + let mut actual = (0..expected.len()).map(|_| self.next(&mut pump)).collect::>(); + let mut expected = expected.iter().collect::>(); + // Stable sorting preserves each topic's sequence without fixing their + // interleaving. + actual.sort_by(|a, b| a.name.cmp(&b.name)); + expected.sort_by(|a, b| a.name.cmp(&b.name)); + for (actual, expected) in actual.iter().zip(expected) { + actual.assert_matches(expected); + } + } } fn receive_while_pumping(receiver: &Receiver, mut pump: impl FnMut()) -> T { @@ -819,13 +883,13 @@ fn an_applied_block_on_the_spine_reaches_an_events_subscriber() { } #[test] -fn block_subscriptions_select_imports_and_preserve_repeated_relay_requests() { +fn subscriptions_select_their_topics_and_preserve_repeated_requests() { let base = TempDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ - "cs_two_streams_gossip", - "cs_two_streams_rpc", - "cs_two_streams_resp", + "cs_subscriptions_gossip", + "cs_subscriptions_rpc", + "cs_subscriptions_resp", ]); let mut adapter = SpineAdapter::connect_tile(&tile, &mut *spine); let mut inj = SpineAdapter::connect_tile(&Injector, &mut *spine); @@ -834,59 +898,83 @@ fn block_subscriptions_select_imports_and_preserve_repeated_relay_requests() { let [Bind::Tcp(addr)] = tile.beacon.local_addrs()[..] else { panic!("expected one tcp bind") }; let deadline = Instant::now() + Duration::from_secs(10); let mut crank = || { - assert!(Instant::now() < deadline, "timeout: block subscription routing"); + assert!(Instant::now() < deadline, "timeout: subscription routing"); tile.loop_body(&mut adapter); std::thread::sleep(Duration::from_millis(1)); }; let block = EventsSubscriber::new(addr, "block", 2, &mut crank); let gossip = EventsSubscriber::new(addr, "block_gossip", 3, &mut crank); - let mixed = EventsSubscriber::new(addr, "block,block_gossip", 5, &mut crank); + let column = EventsSubscriber::new(addr, "data_column_sidecar", 5, &mut crank); + let mixed = + EventsSubscriber::new(addr, "block,block_gossip,data_column_sidecar", 10, &mut crank); let mut unrelated = block_relay(9, 0xaf); - let PeerEvent::SendGossip { topic, metadata, protobuf, .. } = &mut unrelated else { - unreachable!() - }; + let PeerEvent::SendGossip { topic, metadata, .. } = &mut unrelated else { unreachable!() }; *topic = GossipTopic::BeaconAttestation(0); *metadata = None; - let ssz = *protobuf; inj.produce(unrelated); + inj.produce(PeerEvent::EarliestSlot(99)); - let column = GossipDataColumn { slot: 13, block_root: [0xaf; 32], column_index: 5 }; - let mut column_relay = unrelated; - let PeerEvent::SendGossip { topic, metadata, .. } = &mut column_relay else { unreachable!() }; - *topic = GossipTopic::DataColumnSidecar(5); - *metadata = Some(GossipMetadata::DataColumn(column)); - inj.produce(column_relay); - inj.produce(PeerEvent::PublishDataColumn { - originator: P2pStreamId::new(1, 0, StreamProtocol::DataColumnSidecarsByRange, true), - topic: GossipTopic::DataColumnSidecar(5), - ssz, - column, - }); - inj.produce(block_relay(10, 0xac)); - inj.produce(block_relay(10, 0xac)); - inj.produce(block_received(11, 0xab, BlockStage::Applied)); - - // Observe both topics before sending sentinels, keeping leaks inside the events - // read. The two queues promise no ordering relative to each other. - let mut initial = (0..3).map(|_| mixed.next(&mut crank)).collect::>(); - initial.sort_by(|a, b| a.name.cmp(&b.name)); - initial[0].assert_block("block", 11, 0xab); - initial[1].assert_block("block_gossip", 10, 0xac); - initial[2].assert_block("block_gossip", 10, 0xac); + let [relay, _] = column_publications(10, 0xac, 3); + let [_, rpc] = column_publications(11, 0xad, 5); + inj.produce(relay); + inj.produce(relay); + inj.produce(rpc); + inj.produce(block_relay(12, 0xae)); inj.produce(block_relay(12, 0xae)); - inj.produce(block_received(14, 0xb0, BlockStage::Applied)); - - block.next(&mut crank).assert_block("block", 11, 0xab); - block.next(&mut crank).assert_block("block", 14, 0xb0); - gossip.next(&mut crank).assert_block("block_gossip", 10, 0xac); - gossip.next(&mut crank).assert_block("block_gossip", 10, 0xac); - gossip.next(&mut crank).assert_block("block_gossip", 12, 0xae); - let mut trailing = [mixed.next(&mut crank), mixed.next(&mut crank)]; - trailing.sort_by(|a, b| a.name.cmp(&b.name)); - trailing[0].assert_block("block", 14, 0xb0); - trailing[1].assert_block("block_gossip", 12, 0xae); - for subscriber in [block, gossip, mixed] { + inj.produce(block_received(13, 0xab, BlockStage::Applied)); + + // Observe every topic before sending sentinels, so leaks remain inside the + // events read. + mixed.assert_topic_sequences( + &[ + SseEvent::block(13, 0xab), + SseEvent::block_gossip(12, 0xae), + SseEvent::block_gossip(12, 0xae), + SseEvent::column(10, 0xac, 3), + SseEvent::column(10, 0xac, 3), + SseEvent::column(11, 0xad, 5), + ], + &mut crank, + ); + inj.produce(relay); + let [_, sentinel] = column_publications(14, 0xaf, 7); + inj.produce(sentinel); + inj.produce(block_relay(15, 0xb0)); + inj.produce(block_received(16, 0xb1, BlockStage::Applied)); + + block.assert_topic_sequences( + &[SseEvent::block(13, 0xab), SseEvent::block(16, 0xb1)], + &mut crank, + ); + gossip.assert_topic_sequences( + &[ + SseEvent::block_gossip(12, 0xae), + SseEvent::block_gossip(12, 0xae), + SseEvent::block_gossip(15, 0xb0), + ], + &mut crank, + ); + column.assert_topic_sequences( + &[ + SseEvent::column(10, 0xac, 3), + SseEvent::column(10, 0xac, 3), + SseEvent::column(11, 0xad, 5), + SseEvent::column(10, 0xac, 3), + SseEvent::column(14, 0xaf, 7), + ], + &mut crank, + ); + mixed.assert_topic_sequences( + &[ + SseEvent::block(16, 0xb1), + SseEvent::block_gossip(15, 0xb0), + SseEvent::column(10, 0xac, 3), + SseEvent::column(14, 0xaf, 7), + ], + &mut crank, + ); + for subscriber in [block, gossip, column, mixed] { subscriber.client.join().unwrap(); } } @@ -911,19 +999,42 @@ fn a_late_subscriber_receives_only_relay_requests_published_after_it() { tile.loop_body(&mut adapter); std::thread::sleep(Duration::from_millis(1)); }; - let early = EventsSubscriber::new(addr, "block_gossip", 1, &mut crank); + let topics = "block_gossip,data_column_sidecar"; + let early = EventsSubscriber::new(addr, topics, 3, &mut crank); inj.produce(block_relay(20, 0x11)); - early.next(&mut crank).assert_block("block_gossip", 20, 0x11); + let [relay, _] = column_publications(20, 0x11, 3); + let [_, rpc] = column_publications(21, 0x12, 5); + inj.produce(relay); + inj.produce(rpc); + early.assert_topic_sequences( + &[ + SseEvent::block_gossip(20, 0x11), + SseEvent::column(20, 0x11, 3), + SseEvent::column(21, 0x12, 5), + ], + &mut crank, + ); early.client.join().unwrap(); - let late = EventsSubscriber::new(addr, "block_gossip", 1, &mut crank); - inj.produce(block_relay(21, 0x22)); - late.next(&mut crank).assert_block("block_gossip", 21, 0x22); + let late = EventsSubscriber::new(addr, topics, 3, &mut crank); + inj.produce(block_relay(22, 0x22)); + let [relay, _] = column_publications(22, 0x22, 7); + let [_, rpc] = column_publications(23, 0x23, 9); + inj.produce(relay); + inj.produce(rpc); + late.assert_topic_sequences( + &[ + SseEvent::block_gossip(22, 0x22), + SseEvent::column(22, 0x22, 7), + SseEvent::column(23, 0x23, 9), + ], + &mut crank, + ); late.client.join().unwrap(); } #[test] -fn a_gossip_event_is_served_while_the_engine_pool_is_saturated() { +fn gossip_events_are_served_while_the_engine_pool_is_saturated() { let base = TempDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); @@ -973,9 +1084,20 @@ fn a_gossip_event_is_served_while_the_engine_pool_is_saturated() { let mut pump = || { crank(&mut tile, &mut el); }; - let client = EventsSubscriber::new(addr, "block_gossip", 1, &mut pump); + let client = EventsSubscriber::new(addr, "block_gossip,data_column_sidecar", 3, &mut pump); inj.produce(block_relay(30, 0x33)); - client.next(&mut pump).assert_block("block_gossip", 30, 0x33); + let [relay, _] = column_publications(31, 0x34, 7); + let [_, rpc] = column_publications(32, 0x35, 9); + inj.produce(relay); + inj.produce(rpc); + client.assert_topic_sequences( + &[ + SseEvent::block_gossip(30, 0x33), + SseEvent::column(31, 0x34, 7), + SseEvent::column(32, 0x35, 9), + ], + &mut pump, + ); client.client.join().unwrap(); assert_eq!(pending, capacity, "the additional FCU stays queued while SSE is served"); } diff --git a/crates/beacon_api/src/events.rs b/crates/beacon_api/src/events.rs index d82a0f8c..b2edb86a 100644 --- a/crates/beacon_api/src/events.rs +++ b/crates/beacon_api/src/events.rs @@ -14,6 +14,7 @@ pub(crate) const KEEP_ALIVE: &[u8] = b": keep-alive\n\n"; pub(crate) enum Channel { Block, BlockGossip, + DataColumnSidecar, } #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] @@ -70,6 +71,7 @@ fn channel(topic: &str) -> Option { match topic { "block" => Some(Channel::Block), "block_gossip" => Some(Channel::BlockGossip), + "data_column_sidecar" => Some(Channel::DataColumnSidecar), _ => None, } } @@ -132,14 +134,19 @@ mod tests { gossip.insert(Channel::BlockGossip); assert_eq!(topics("topics=block_gossip"), Ok(gossip)); - let mut both = gossip; - both.insert(Channel::Block); + let mut columns = ChannelSet::default(); + columns.insert(Channel::DataColumnSidecar); + assert_eq!(topics("topics=data_column_sidecar"), Ok(columns)); + + let mut all = columns; + all.insert(Channel::Block); + all.insert(Channel::BlockGossip); for query in [ - "topics=block,block_gossip", - "topics=block_gossip&topics=block", - "topics=block%2Cblock_gossip", + "topics=block,block_gossip,data_column_sidecar", + "topics=data_column_sidecar&topics=block_gossip&topics=block", + "topics=block%2Cdata_column_sidecar%2Cblock_gossip", ] { - assert_eq!(topics(query), Ok(both), "{query}"); + assert_eq!(topics(query), Ok(all), "{query}"); } } diff --git a/crates/beacon_api/src/json.rs b/crates/beacon_api/src/json.rs index b86107b7..a79c3688 100644 --- a/crates/beacon_api/src/json.rs +++ b/crates/beacon_api/src/json.rs @@ -240,6 +240,22 @@ impl Json<'_> { self.end_object(); } + pub(crate) fn data_column_sidecar_event( + &mut self, + block_root: &[u8; 32], + column_index: u64, + slot: u64, + ) { + self.begin_object(); + self.key("block_root"); + self.hex(block_root); + self.key("index"); + self.quoted_u64(column_index); + self.key("slot"); + self.quoted_u64(slot); + self.end_object(); + } + pub(crate) fn finality_checkpoints(&mut self, checkpoints: &FinalityCheckpoints) { self.begin_object(); self.key("previous_justified"); @@ -508,4 +524,13 @@ mod tests { ); assert_eq!(out, expected.as_bytes()); } + + #[test] + fn data_column_sidecar_event_carries_the_root_index_and_slot() { + let body = write(|json| json.data_column_sidecar_event(&[0x9a; 32], 3, 10)); + let parsed: serde_json::Value = serde_json::from_str(&body).expect("valid JSON"); + assert_eq!(parsed["block_root"], format!("0x{}", hex::encode([0x9a; 32]))); + assert_eq!(parsed["index"], "3"); + assert_eq!(parsed["slot"], "10"); + } } diff --git a/crates/beacon_api/src/server.rs b/crates/beacon_api/src/server.rs index a30bad2c..849d9df3 100644 --- a/crates/beacon_api/src/server.rs +++ b/crates/beacon_api/src/server.rs @@ -404,6 +404,17 @@ impl BeaconApi { self.publish(Channel::BlockGossip, "block_gossip", &data); } + pub fn publish_data_column_sidecar( + &mut self, + block_root: &[u8; 32], + column_index: u64, + slot: u64, + ) { + let mut data = Vec::new(); + Json::new(&mut data).data_column_sidecar_event(block_root, column_index, slot); + self.publish(Channel::DataColumnSidecar, "data_column_sidecar", &data); + } + fn publish(&mut self, channel: Channel, event: &str, data: &[u8]) { let mut frame = Vec::new(); events::frame(&mut frame, event, data); diff --git a/docs/adr/0004-sync-materialized-api.md b/docs/adr/0004-sync-materialized-api.md index d30f621b..b0aaf9e3 100644 --- a/docs/adr/0004-sync-materialized-api.md +++ b/docs/adr/0004-sync-materialized-api.md @@ -92,3 +92,19 @@ its payload; producers own topic consistency. The `block` topic still follows Repeated requests for one root are not deduplicated, and late subscribers receive no replay. Subscribers to both topics can reach the existing send cap sooner. + +Amended 2026-09-10: `/eth/v1/events` also serves `data_column_sidecar` for +column publication requests following silver's gossip checks, including KZG. +This deliberately narrows the Beacon API's validation contract: validation +without a publication request produces no event. The boundary selects +column metadata on `SendGossip` or `PublishDataColumn`, without reading +payload bytes or filtering by custody. Producers own topic consistency. +Control's converted request stays off the spine, avoiding a second +notification. These events acknowledge requests, including RPC requests +that can fail before encoding; they do not guarantee delivery to peers. +Buffered copies, RPC columns processed while syncing, held copies, and EL +reconstruction remain silent under the existing publication policy. +`Persist` and `Available` retain their existing meaning and selection. +Repeated requests are not deduplicated, and late subscribers receive no +replay. The `beacon_events` and `peer_events` queues establish no shared +ordering. Additional subscriptions can reach the existing send cap sooner. From 5d92d9d5717ace94b1eb7118fde6979caee67baf Mon Sep 17 00:00:00 2001 From: Bronek Kozicki Date: Fri, 11 Sep 2026 15:33:50 +0100 Subject: [PATCH 3/5] Clean up temporary test directories and shared memory Several fixtures create directories without retaining a cleanup owner. Others remove directories only after their assertions succeed. Retain TempDir ownership for storage, configuration, counters, telemetry, and panic-log fixtures so cleanup also runs during unwinding. Deleting Flux link files alone leaves their POSIX shared-memory objects behind. Add ShmemDir under silver_common's test-util feature to call cleanup_shmem before its TempDir drops. Use it for tile spines, boundary fixtures, Surfer queues, and the E2E harnesses and examples. Keep directory owners alive alongside the resources they support. The beacon-state producer fixture now owns its spine and directory together. Existing behavior assertions remain unchanged. One regression test checks that the directory disappears and its shared memory cannot be reopened after normal return or unwinding. Before the cleanup call, that test failed because the shared-memory object survived; the other 1,369 tests passed. An isolated Controller test previously left 119 shared-memory objects. It now leaves no fixture objects and 28 process-global timer objects. Those timers remain outside the private fixture directories. Forgotten Flux mappings remain live until process exit; this change unlinks their names without changing the mapping lifetime. Destructor cleanup cannot cover aborts or kills that skip unwinding. Final gates: just fmt-check, just clippy, git diff --check, and NEXTEST_TEST_THREADS=4 just nextest exited 0. The workspace passed 1,370 tests and skipped five. Assisted-by: Codex:gpt-6-astra --- Cargo.lock | 11 ++-- crates/application_boundary/Cargo.toml | 2 +- crates/application_boundary/tests/tile.rs | 29 +++++----- crates/beacon_state/tile/Cargo.toml | 1 - crates/beacon_state/tile/src/tile/tests.rs | 33 +++++------ .../tile/src/tile/tests/block_relay.rs | 4 +- crates/beacon_state/tile/tests/common.rs | 26 +++------ .../tile/tests/ef_gossip_validation.rs | 7 +-- crates/columns/Cargo.toml | 2 +- crates/columns/src/tile.rs | 6 +- crates/common/Cargo.toml | 7 ++- crates/common/src/lib.rs | 2 + crates/common/src/test_util.rs | 58 +++++++++++++++++++ crates/common/tests/panic_log.rs | 11 ++-- crates/config/Cargo.toml | 3 + crates/config/src/lib.rs | 42 ++++++-------- crates/control/src/tile/tests.rs | 33 +++++------ crates/e2e/Cargo.toml | 3 +- crates/e2e/examples/gossip_oneway.rs | 7 ++- crates/e2e/examples/gossip_oneway_lh.rs | 7 +-- crates/e2e/examples/gossip_oneway_lh_b.rs | 7 ++- crates/e2e/src/bin/da_replay.rs | 6 +- crates/e2e/src/harness.rs | 9 +-- crates/e2e/src/utils.rs | 6 +- crates/e2e/tests/lh_common/mod.rs | 6 +- crates/e2e/tests/lh_rpc.rs | 5 +- crates/e2e/tests/rpc_multipart.rs | 7 +-- crates/metrics/Cargo.toml | 1 + crates/metrics/src/lib.rs | 7 +-- crates/storage/Cargo.toml | 2 + crates/storage/src/store/checkpoint.rs | 19 +++--- crates/storage/src/store/tests.rs | 55 +++++++----------- crates/storage/src/tile.rs | 30 +++++----- crates/surfer/Cargo.toml | 4 ++ crates/surfer/src/sources/counters.rs | 9 ++- crates/surfer/src/sources/tilemetrics.rs | 30 ++++------ crates/telemetry/Cargo.toml | 1 + crates/telemetry/src/collector.rs | 45 ++++++-------- 38 files changed, 276 insertions(+), 267 deletions(-) create mode 100644 crates/common/src/test_util.rs diff --git a/Cargo.lock b/Cargo.lock index 6ad2bf88..de4310ed 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4545,7 +4545,6 @@ dependencies = [ "silver_config", "silver_engine_api", "silver_httpcore", - "tempfile", "ureq", ] @@ -4585,7 +4584,6 @@ dependencies = [ "silver_ssz", "silver_storage", "snap 0.1.2", - "tempfile", "thiserror 1.0.69", "tracing", "tracing-subscriber", @@ -4635,7 +4633,6 @@ dependencies = [ "silver_beacon_state_data", "silver_common", "snap 1.1.1", - "tempfile", "tracing", ] @@ -4667,11 +4664,13 @@ dependencies = [ "serde", "serde_json", "sha3", + "shared_memory", "silver_beacon_state_data", "silver_chain_spec", "silver_metrics", "silver_ssz", "snap 1.1.1", + "tempfile", "thiserror 1.0.69", "toml", "tracing", @@ -4689,6 +4688,7 @@ dependencies = [ "serde_yml", "silver_chain_spec", "silver_common", + "tempfile", "toml", "tracing", ] @@ -4759,7 +4759,6 @@ dependencies = [ "silver_network", "silver_peer", "snap 1.1.1", - "tempfile", "tokio", "tracing", "tracing-subscriber", @@ -4829,6 +4828,7 @@ dependencies = [ "rustc-hash", "serde", "serde_json", + "tempfile", ] [[package]] @@ -4913,6 +4913,7 @@ dependencies = [ "rand 0.8.6", "silver_beacon_state_data", "silver_common", + "tempfile", "tracing", ] @@ -4935,6 +4936,7 @@ dependencies = [ "silver_peer", "silver_stages", "silver_storage", + "tempfile", ] [[package]] @@ -4954,6 +4956,7 @@ dependencies = [ "silver_common", "silver_config", "silver_stages", + "tempfile", "toml", "tracing", "ureq", diff --git a/crates/application_boundary/Cargo.toml b/crates/application_boundary/Cargo.toml index 6ff861b9..a6c1adaf 100644 --- a/crates/application_boundary/Cargo.toml +++ b/crates/application_boundary/Cargo.toml @@ -15,10 +15,10 @@ silver_engine_api.workspace = true silver_httpcore.workspace = true [dev-dependencies] +silver_common = { workspace = true, features = ["test-util"] } hex.workspace = true serde_json.workspace = true silver_engine_api = { workspace = true, features = ["test-el"] } -tempfile = "3" ureq.workspace = true [lints] diff --git a/crates/application_boundary/tests/tile.rs b/crates/application_boundary/tests/tile.rs index c6ec5ba2..ea7dd03c 100644 --- a/crates/application_boundary/tests/tile.rs +++ b/crates/application_boundary/tests/tile.rs @@ -16,12 +16,11 @@ use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ELSyncStatus, EngineFcuReq, EngineReq, EngineResp, Enr, GossipBlock, GossipDataColumn, GossipMetadata, GossipTopic, Identify, Keypair, MessageId, P2pStreamId, PayloadValidationStatus, PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, - TCache, TCacheProducer, TCacheRead, ssz_view::STATUS_V2_SIZE, + TCache, TCacheProducer, TCacheRead, ssz_view::STATUS_V2_SIZE, test_util::ShmemDir, }; use silver_config::EngineConfig; use silver_engine_api::test_el::{FCU_VALID_RESULT, FakeEl, write_jwt}; use silver_httpcore::Bind; -use tempfile::TempDir; struct Injector; impl Tile for Injector { @@ -325,7 +324,7 @@ fn status_event(head_slot: u64, wall_slot: u64, head_optimistic: bool) -> Beacon #[test] fn serves_identity_over_tcp() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_tcp_gossip", @@ -350,7 +349,7 @@ fn serves_identity_over_tcp() { #[test] fn serves_identity_over_uds() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let socket = base.path().join("beacon_api.sock"); let mut tile = boundary_tile(&Bind::Unix(socket.clone()), no_el(), [ @@ -382,7 +381,7 @@ fn serves_identity_over_uds() { /// once the response arrives. #[test] fn serves_beacon_api_while_engine_call_in_flight() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); @@ -458,7 +457,7 @@ fn serves_beacon_api_while_engine_call_in_flight() { /// connection, and completions must correlate out of order. #[test] fn pool_cap_gates_spine_intake() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); @@ -539,7 +538,7 @@ fn pool_cap_gates_spine_intake() { /// not the one after. #[test] fn an_engine_request_reaches_the_el_in_the_iteration_that_takes_it() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); @@ -617,7 +616,7 @@ fn an_engine_request_reaches_the_el_in_the_iteration_that_takes_it() { /// first iteration on. #[test] fn node_status_tracks_the_spine_once_the_cursor_snaps() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_status_gossip", @@ -663,7 +662,7 @@ fn node_status_tracks_the_spine_once_the_cursor_snaps() { /// loses its whole backlog. #[test] fn node_status_updates_while_the_engine_pool_is_at_cap() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); @@ -735,7 +734,7 @@ fn node_status_updates_while_the_engine_pool_is_at_cap() { /// at the same time. #[test] fn concurrent_clients_and_engine_calls_keep_their_own_sockets() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); @@ -827,7 +826,7 @@ fn concurrent_clients_and_engine_calls_keep_their_own_sockets() { /// had one to itself. #[test] fn serves_concurrent_clients_with_no_engine_registered() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_noel_gossip", @@ -852,7 +851,7 @@ fn serves_concurrent_clients_with_no_engine_registered() { #[test] fn an_applied_block_on_the_spine_reaches_an_events_subscriber() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_sse_gossip", @@ -884,7 +883,7 @@ fn an_applied_block_on_the_spine_reaches_an_events_subscriber() { #[test] fn subscriptions_select_their_topics_and_preserve_repeated_requests() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_subscriptions_gossip", @@ -981,7 +980,7 @@ fn subscriptions_select_their_topics_and_preserve_repeated_requests() { #[test] fn a_late_subscriber_receives_only_relay_requests_published_after_it() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ "cs_late_gossip", @@ -1035,7 +1034,7 @@ fn a_late_subscriber_receives_only_relay_requests_published_after_it() { #[test] fn gossip_events_are_served_while_the_engine_pool_is_saturated() { - let base = TempDir::new().unwrap(); + let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let (mut el, endpoint) = FakeEl::tcp(); let jwt_path = write_jwt(base.path()); diff --git a/crates/beacon_state/tile/Cargo.toml b/crates/beacon_state/tile/Cargo.toml index e389f96e..395784a9 100644 --- a/crates/beacon_state/tile/Cargo.toml +++ b/crates/beacon_state/tile/Cargo.toml @@ -34,7 +34,6 @@ serde_yml.workspace = true # EF gossip_validation: the data_column_sidecar topic is the columns tile's. silver_columns = { workspace = true, features = ["ef_tests"] } snap = "0.1" -tempfile = "3" tracing-subscriber.workspace = true [[bench]] diff --git a/crates/beacon_state/tile/src/tile/tests.rs b/crates/beacon_state/tile/src/tile/tests.rs index a92e4f62..f5e0ccbb 100644 --- a/crates/beacon_state/tile/src/tile/tests.rs +++ b/crates/beacon_state/tile/src/tile/tests.rs @@ -21,6 +21,7 @@ use silver_common::{ SIGNED_EXECUTION_PAYLOAD_ENVELOPE_MIN, SIGNED_VOLUNTARY_EXIT_SIZE, SINGLE_ATT_SIZE, SignedAggregateAndProofView, SignedBeaconBlockView, SingleAttestationView, StatusView, }, + test_util::ShmemDir, }; use silver_ssz::ssz_view::{EXECUTION_PAYLOAD_ENVELOPE_MIN, SyncCommitteeContributionView}; @@ -583,7 +584,7 @@ fn anchor_child(block_root: B256, state_id: StateId) -> BlockImport { fn a_block_already_in_fork_choice_is_reported_already_known() { let (mut tile, mut gp, _rp, mut spine, mut adapter) = tile_with_producers(200); seed_tile(&mut tile, 4, 10); - let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine); + let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine.spine); sink.consume(|_: BeaconStateEvent, _| {}); let mut bytes = empty_block(); @@ -626,7 +627,7 @@ fn a_block_is_applied_once_and_already_known_on_repeat() { let block_slot = SignedBeaconBlockView::slot(&block_ssz); let (mut tile, mut gp, _rp, mut spine, mut adapter) = tile_with_producers_on(block_slot + 1, state); - let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine); + let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine.spine); sink.consume(|_: BeaconStateEvent, _| {}); // The checkpoint was loaded without decompressed pubkeys, so this import @@ -853,19 +854,24 @@ fn pending_admission_window_bounds() { assert!(!tile.within_pending_window(50 + tol + 1)); } +struct TestSpine { + spine: Box, + _dir: ShmemDir, +} + /// Tile (seed separately) plus a spine + adapter, so tests can drive /// `buffer_orphan`, which produces into `adapter.producers`. The spine is /// returned to keep it alive for the adapter. fn tile_with_producers( wall_slot: u64, -) -> (BeaconStateTile, TProducer, TProducer, Box, SpineAdapter) { +) -> (BeaconStateTile, TProducer, TProducer, TestSpine, SpineAdapter) { tile_with_producers_on(wall_slot, BeaconState::empty_test(0)) } fn tile_with_producers_on( wall_slot: u64, state: BeaconState, -) -> (BeaconStateTile, TProducer, TProducer, Box, SpineAdapter) { +) -> (BeaconStateTile, TProducer, TProducer, TestSpine, SpineAdapter) { let (tile, gp, rp) = make_tile_with_gossip(wall_slot, state); let (spine, adapter) = spine_adapter(&tile); (tile, gp, rp, spine, adapter) @@ -873,18 +879,11 @@ fn tile_with_producers_on( /// A spine plus the tile's adapter on it, so tests can hand `adapter.producers` /// to methods that produce. The spine is returned to keep the adapter alive. -fn spine_adapter(tile: &BeaconStateTile) -> (Box, SpineAdapter) { - use std::sync::atomic::{AtomicU64, Ordering}; - static SEQ: AtomicU64 = AtomicU64::new(0); - let base = std::env::temp_dir().join(format!( - "silver-pending-{}-{}", - std::process::id(), - SEQ.fetch_add(1, Ordering::Relaxed) - )); - std::fs::create_dir_all(&base).expect("temp base"); - let mut spine = Box::new(SilverSpine::new_with_base_dir(&base, None)); +fn spine_adapter(tile: &BeaconStateTile) -> (TestSpine, SpineAdapter) { + let dir = ShmemDir::new().expect("temp base"); + let mut spine = Box::new(SilverSpine::new_with_base_dir(dir.path(), None)); let adapter = SpineAdapter::connect_tile(tile, &mut spine); - (spine, adapter) + (TestSpine { spine, _dir: dir }, adapter) } fn root_with(idx: u64, tag: u8) -> B256 { @@ -960,7 +959,7 @@ fn missing_blocks(sink: &mut SpineAdapter) -> Vec<(B256, u64)> { fn lapped_orphan_is_re_requested_on_replay() { let (mut tile, _gp, mut rp, mut spine, mut adapter) = tile_with_producers(200); seed_tile(&mut tile, 4, 10); - let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine); + let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine.spine); sink.consume(|_: SyncNeed, _| {}); let parent_root = [0xAAu8; 32]; @@ -2917,7 +2916,7 @@ fn el_invalid_drops_staged_block() { const S_ROOT: B256 = [0x05; 32]; let mut forks = ThreeForks::new(); let (mut spine, mut adapter) = spine_adapter(&forks.tile); - let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine); + let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine.spine); sink.consume(|_: BeaconStateEvent, _| {}); let mut producer = TCache::producer("test_el_invalid", 1 << 12); forks.stage(&mut producer, S_ROOT, D_ROOT, forks.d_id, 3); diff --git a/crates/beacon_state/tile/src/tile/tests/block_relay.rs b/crates/beacon_state/tile/src/tile/tests/block_relay.rs index 1122b5d8..c18c98a3 100644 --- a/crates/beacon_state/tile/src/tile/tests/block_relay.rs +++ b/crates/beacon_state/tile/src/tile/tests/block_relay.rs @@ -68,7 +68,7 @@ struct BlockPublications { replay: TProducer, adapter: SpineAdapter, sink: SpineAdapter, - _spine: Box, + _spine: TestSpine, } impl BlockPublications { @@ -80,7 +80,7 @@ impl BlockPublications { make_tile_with_producers(block_slot + 1, state, fulu_from_genesis()); tile.sync_target = target; let (mut spine, adapter) = spine_adapter(&tile); - let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine); + let mut sink = SpineAdapter::connect_tile(&Sink, &mut spine.spine); sink.consume(|_: BeaconStateEvent, _| {}); sink.consume(|_: PeerEvent, _| {}); Self { tile, gossip, rpc, replay, adapter, sink, _spine: spine } diff --git a/crates/beacon_state/tile/tests/common.rs b/crates/beacon_state/tile/tests/common.rs index d9c9515f..34dd02c8 100644 --- a/crates/beacon_state/tile/tests/common.rs +++ b/crates/beacon_state/tile/tests/common.rs @@ -1,11 +1,7 @@ use std::{ fs, path::{Path, PathBuf}, - process, - sync::{ - Arc, - atomic::{AtomicU64, Ordering}, - }, + sync::Arc, time::{Duration, SystemTime, UNIX_EPOCH}, }; @@ -21,6 +17,7 @@ use silver_common::{ P2pStreamId, PeerEvent, RpcInbound, RpcResponseInbound, SilverSpine, StreamProtocol, SyncNeed, SyncUpdate, TCache, TCacheProducer, TProducer, TRandomAccess, hex32, ssz_view::{STATUS_V2_SIZE, SignedBeaconBlockView}, + test_util::ShmemDir, ticker::SlotTicker, }; use silver_config::SyncingConfig; @@ -88,7 +85,6 @@ impl Tile for Injector { } pub struct Harness { - _spine: Box, tile: BeaconStateTile, tile_adapter: SpineAdapter, inj_adapter: SpineAdapter, @@ -97,7 +93,8 @@ pub struct Harness { // Kept alive to back the tile's replay consumer; unused by these tests. _replay_in_producer: TProducer, outbound_log: Vec, - _base_dir: PathBuf, // owned to keep temp files around for the run + _spine: Box, + _base_dir: ShmemDir, } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -209,17 +206,8 @@ impl Harness { TRandomAccess, ) -> BeaconStateTile, { - static SEQ: AtomicU64 = AtomicU64::new(0); - let seq = SEQ.fetch_add(1, Ordering::Relaxed); - let base = std::env::temp_dir().join(format!( - "silver-ef-{}-{}-{}", - process::id(), - seq, - rand::random::() - )); - fs::create_dir_all(&base).expect("create temp base"); - - let mut spine = Box::new(SilverSpine::new_with_base_dir(&base, None)); + let base = ShmemDir::new().expect("create temp base"); + let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); // Ticker: genesis positioned so `current_slot()` == wall_slot at // construction. @@ -262,7 +250,6 @@ impl Harness { inj_adapter.consume(|_: SyncNeed, _| {}); Self { - _spine: spine, tile, tile_adapter, inj_adapter, @@ -270,6 +257,7 @@ impl Harness { rpc_in_producer, _replay_in_producer: replay_in_producer, outbound_log: Vec::new(), + _spine: spine, _base_dir: base, } } diff --git a/crates/beacon_state/tile/tests/ef_gossip_validation.rs b/crates/beacon_state/tile/tests/ef_gossip_validation.rs index 4d66cfcd..fc223fee 100644 --- a/crates/beacon_state/tile/tests/ef_gossip_validation.rs +++ b/crates/beacon_state/tile/tests/ef_gossip_validation.rs @@ -29,9 +29,8 @@ use silver_beacon_state_data::{ use silver_columns::tile::{ColumnConsumers, DataColumnsTile, EfVerdict}; use silver_common::{ PayloadValidationStatus, SilverSpine, TCache, TCacheProducer, TCacheRead, TProducer, - ssz_view::SignedBeaconBlockView, + ssz_view::SignedBeaconBlockView, test_util::ShmemDir, }; -use tempfile::TempDir; const HANDLED_TOPICS: &[&str] = &[ "beacon_block", @@ -157,7 +156,7 @@ struct ColumnsRig { tile: DataColumnsTile, gossip: TProducer, _spine: Box, - _dir: TempDir, + _dir: ShmemDir, } impl ColumnsRig { @@ -185,7 +184,7 @@ impl ColumnsRig { TCache::producer("ef_columns_el", 1 << 16), ticker, ); - let dir = TempDir::new().unwrap(); + let dir = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(dir.path(), None)); let adapter = SpineAdapter::connect_tile(&tile, &mut spine); Self { adapter, tile, gossip, _spine: spine, _dir: dir } diff --git a/crates/columns/Cargo.toml b/crates/columns/Cargo.toml index 1556c930..04140a62 100644 --- a/crates/columns/Cargo.toml +++ b/crates/columns/Cargo.toml @@ -25,6 +25,6 @@ ef_tests = ["silver_common/test-util"] workspace = true [dev-dependencies] +silver_common = { workspace = true, features = ["test-util"] } rand.workspace = true snap.workspace = true -tempfile = "3" diff --git a/crates/columns/src/tile.rs b/crates/columns/src/tile.rs index 1f846c71..d1a21fee 100644 --- a/crates/columns/src/tile.rs +++ b/crates/columns/src/tile.rs @@ -786,8 +786,8 @@ mod tests { DATA_COLUMN_SIDECAR_MIN, DataColumnSidecarFuluView, NUMBER_OF_COLUMNS, SIGNED_BEACON_BLOCK_MIN, }, + test_util::ShmemDir, }; - use tempfile::TempDir; use super::*; @@ -807,7 +807,7 @@ mod tests { rpc_p: TProducer, engine_p: TProducer, _spine: Box, - _dir: TempDir, + _dir: ShmemDir, } struct Injector; @@ -866,7 +866,7 @@ mod tests { SlotTicker::new(0, Duration::from_secs(12), Duration::from_secs(4)), ); - let dir = tempfile::tempdir().unwrap(); + let dir = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(dir.path(), None)); let conn = SpineAdapter::connect_tile(&tile, &mut spine); let mut inj = SpineAdapter::connect_tile(&Injector, &mut spine); diff --git a/crates/common/Cargo.toml b/crates/common/Cargo.toml index 1999f7b6..b84a9a84 100644 --- a/crates/common/Cargo.toml +++ b/crates/common/Cargo.toml @@ -35,6 +35,7 @@ serde_json.workspace = true toml.workspace = true sha3.workspace = true snap.workspace = true +tempfile = { version = "3", optional = true } thiserror.workspace = true tracing.workspace = true tracing-appender.workspace = true @@ -44,11 +45,15 @@ tracing-subscriber.workspace = true alloc-profile = ["silver_metrics/alloc-profile"] # Forwards `#[timed]`'s hardware-counter dimension (rdpmc) to silver_metrics. perf = ["silver_metrics/perf"] -test-util = [] +test-util = ["dep:tempfile"] thread_park = ["flux/park"] [build-dependencies] buffa-build = "0.2.0" +[dev-dependencies] +shared_memory = "0.12" +tempfile = "3" + [lints] workspace = true diff --git a/crates/common/src/lib.rs b/crates/common/src/lib.rs index 937794bf..efeb73a3 100644 --- a/crates/common/src/lib.rs +++ b/crates/common/src/lib.rs @@ -56,6 +56,8 @@ mod identity; mod spine; pub use silver_beacon_state_data::SLOTS_PER_EPOCH; pub use silver_ssz::{merkle, progressive, ssz_hash, ssz_hash_gloas, ssz_view}; +#[cfg(feature = "test-util")] +pub mod test_util; pub mod ticker; pub mod tracing; mod util; diff --git a/crates/common/src/test_util.rs b/crates/common/src/test_util.rs new file mode 100644 index 00000000..b75a6eea --- /dev/null +++ b/crates/common/src/test_util.rs @@ -0,0 +1,58 @@ +use std::{io, path::Path}; + +use flux::communication::cleanup_shmem; +use tempfile::TempDir; + +/// Removing a Flux link file alone leaves its POSIX shared-memory object +/// behind. +pub struct ShmemDir(TempDir); + +impl ShmemDir { + pub fn new() -> io::Result { + TempDir::new().map(Self) + } + + pub fn path(&self) -> &Path { + self.0.path() + } +} + +impl Drop for ShmemDir { + fn drop(&mut self) { + cleanup_shmem(self.path()); + } +} + +#[cfg(test)] +mod tests { + use std::panic::{AssertUnwindSafe, catch_unwind}; + + use flux::communication::queue::{Queue, QueueType}; + use shared_memory::ShmemConf; + + use super::*; + + #[test] + fn shared_memory_is_removed_on_return_and_unwind() { + for unwind in [false, true] { + let mut resource = None; + let result = catch_unwind(AssertUnwindSafe(|| { + let dir = ShmemDir::new().unwrap(); + let link = dir.path().join("queue"); + let _queue: Queue = Queue::create_or_open_shared(&link, 8, QueueType::SPMC); + let mapping = ShmemConf::new().flink(&link).open().unwrap(); + resource = Some((dir.path().to_owned(), mapping.get_os_id().to_owned())); + if unwind { + panic!("exercise fixture unwinding"); + } + })); + assert_eq!(result.is_err(), unwind); + let (path, id) = resource.unwrap(); + assert!(!path.exists(), "fixture directory survives teardown"); + assert!( + ShmemConf::new().os_id(id).open().is_err(), + "shared-memory object survives teardown" + ); + } + } +} diff --git a/crates/common/tests/panic_log.rs b/crates/common/tests/panic_log.rs index ee9b4621..1df7763f 100644 --- a/crates/common/tests/panic_log.rs +++ b/crates/common/tests/panic_log.rs @@ -1,20 +1,19 @@ -use std::{fs, panic, path::PathBuf}; +use std::{fs, panic}; use silver_common::tracing::initialise_tracing_log; +use tempfile::TempDir; #[test] fn panic_reaches_the_log_file() { - let dir = PathBuf::from(env!("CARGO_TARGET_TMPDIR")).join("panic-log"); - let _ = fs::remove_dir_all(&dir); - fs::create_dir_all(&dir).unwrap(); - unsafe { std::env::set_var("LOG_PATH", &dir) }; + let dir = TempDir::new().unwrap(); + unsafe { std::env::set_var("LOG_PATH", dir.path()) }; let guard = initialise_tracing_log("smoke", 1, None, false); let caught = panic::catch_unwind(|| panic!("marker-from-a-tile")); assert!(caught.is_err()); drop(guard); - let log = fs::read_dir(&dir).unwrap().next().unwrap().unwrap().path(); + let log = fs::read_dir(dir.path()).unwrap().next().unwrap().unwrap().path(); let body = fs::read_to_string(&log).unwrap(); assert!(body.contains("marker-from-a-tile"), "{body}"); assert!(body.contains("Full backtrace"), "{body}"); diff --git a/crates/config/Cargo.toml b/crates/config/Cargo.toml index 6be5a89c..fddd90e8 100644 --- a/crates/config/Cargo.toml +++ b/crates/config/Cargo.toml @@ -17,3 +17,6 @@ tracing.workspace = true [lints] workspace = true + +[dev-dependencies] +tempfile = "3" diff --git a/crates/config/src/lib.rs b/crates/config/src/lib.rs index 93733ca8..b1aecba2 100644 --- a/crates/config/src/lib.rs +++ b/crates/config/src/lib.rs @@ -533,6 +533,8 @@ impl Config { #[cfg(test)] mod tests { + use tempfile::TempDir; + use super::*; #[test] @@ -566,8 +568,8 @@ mod tests { /// only the operator can say which half is the typo. #[test] fn a_config_name_contradicting_its_fork_version_still_loads() { - let path = - std::env::temp_dir().join(format!("silver_misnamed_{}.toml", std::process::id())); + let dir = TempDir::new().unwrap(); + let path = dir.path().join("misnamed.toml"); std::fs::write( &path, r#" @@ -583,7 +585,6 @@ mod tests { .unwrap(); let cfg = Config::from_file(&path).unwrap(); - std::fs::remove_file(&path).unwrap(); assert_eq!(cfg.chain_config.spec.misnamed_network(), Some("hoodi")); assert_eq!(cfg.chain_config.spec.network_name(), "mainnet"); @@ -671,24 +672,17 @@ mod tests { path.to_str().unwrap().to_owned() } - fn temp_dir(name: &str) -> std::path::PathBuf { - let dir = std::env::temp_dir().join(format!("silver-config-{name}")); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); - dir - } - /// The whole in-enclave contract: a config naming only the two files a /// network publishes resolves to the same digest a hand-written one /// spells out. #[test] fn spec_file_and_anchor_resolve_the_digest_and_genesis() { - let dir = temp_dir("spec-file"); + let dir = TempDir::new().unwrap(); let gvr = [7u8; 32]; let genesis = 1_600_000_000; - let anchor = write_anchor(&dir, genesis, gvr); + let anchor = write_anchor(dir.path(), genesis, gvr); let spec_file = write_file( - &dir, + dir.path(), "config.yaml", "CONFIG_NAME: kurtosis\n\ GENESIS_FORK_VERSION: 0x10000038\n\ @@ -700,7 +694,7 @@ mod tests { GLOAS_FORK_EPOCH: 18446744073709551615\n", ); let config_file = write_file( - &dir, + dir.path(), "silver.toml", &format!( "secret_key = \"{}\"\n\ @@ -728,17 +722,17 @@ mod tests { /// assert a digest or a genesis that contradicts the state it boots on. #[test] fn the_anchor_outranks_the_files_literals() { - let dir = temp_dir("anchor-wins"); + let dir = TempDir::new().unwrap(); let genesis = 1_600_000_000; let gvr = [7u8; 32]; - let anchor = write_anchor(&dir, genesis, gvr); + let anchor = write_anchor(dir.path(), genesis, gvr); let spec_file = write_file( - &dir, + dir.path(), "config.yaml", "FULU_FORK_VERSION: 0x70000038\nFULU_FORK_EPOCH: 0\nELECTRA_FORK_EPOCH: 0\n", ); let config_file = write_file( - &dir, + dir.path(), "silver.toml", &format!( "secret_key = \"{}\"\n\ @@ -767,18 +761,18 @@ mod tests { /// gossips on. #[test] fn spec_reading_earlier_than_fulu_is_refused() { - let dir = temp_dir("pre-fulu"); + let dir = TempDir::new().unwrap(); // A genesis an hour ago, like a devnet's: the wall epoch is then far // below mainnet's fulu_fork_epoch, which is the default in force here. let recent = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs() - 3600; - let anchor = write_anchor(&dir, recent, [7u8; 32]); + let anchor = write_anchor(dir.path(), recent, [7u8; 32]); let spec_file = write_file( - &dir, + dir.path(), "config.yaml", "FULU_FORK_VERSION: 0x70000038\nELECTRA_FORK_EPOCH: 0\n", ); let config_file = write_file( - &dir, + dir.path(), "silver.toml", &format!( "secret_key = \"{}\"\n\ @@ -798,9 +792,9 @@ mod tests { /// literals stand — this is the default mainnet run. #[test] fn without_an_anchor_the_files_literals_stand() { - let dir = temp_dir("no-anchor"); + let dir = TempDir::new().unwrap(); let config_file = write_file( - &dir, + dir.path(), "silver.toml", &format!("secret_key = \"{}\"\nfork_digest = \"8c9f62fe\"\n", "11".repeat(32)), ); diff --git a/crates/control/src/tile/tests.rs b/crates/control/src/tile/tests.rs index 0faa1fe5..641637b2 100644 --- a/crates/control/src/tile/tests.rs +++ b/crates/control/src/tile/tests.rs @@ -1,15 +1,10 @@ -use std::{ - io::Write, - sync::{ - Arc, - atomic::{AtomicU64, Ordering}, - }, -}; +use std::{io::Write, sync::Arc}; use silver_chain_spec::SpecConfig; use silver_common::{ GossipBlock, GossipDataColumn, GossipMsgIn, GossipMsgOut, IpBytes, Keypair, MessageId, P2pStreamId, PeerId, StreamProtocol, TCache, TCacheProducer, TCacheRead, TProducer, + test_util::ShmemDir, }; use silver_peer::SyncingConfig; @@ -24,6 +19,7 @@ struct GossipPublications { payload: TCacheRead, outbound: TRandomAccess, _spine: Box, + _dir: ShmemDir, } struct Observer; @@ -60,19 +56,22 @@ impl GossipPublications { rpc.cache_ref().random_access("publication_rpc", true).unwrap(), SyncEngine::new(SyncingConfig::default(), false, 0, Arc::new(SpecConfig::mainnet())), ); - static SEQUENCE: AtomicU64 = AtomicU64::new(0); - let base = std::env::temp_dir().join(format!( - "silver-publications-{}-{}", - std::process::id(), - SEQUENCE.fetch_add(1, Ordering::Relaxed), - )); - std::fs::create_dir_all(&base).unwrap(); - let mut spine = Box::new(SilverSpine::new_with_base_dir(&base, None)); + let dir = ShmemDir::new().unwrap(); + let mut spine = Box::new(SilverSpine::new_with_base_dir(dir.path(), None)); let adapter = SpineAdapter::connect_tile(&controller, &mut spine); let mut observer = SpineAdapter::connect_tile(&Observer, &mut spine); observer.consume(|_: P2pSend, _| {}); - let mut capture = - Self { controller, adapter, observer, incoming, rpc, payload, outbound, _spine: spine }; + let mut capture = Self { + controller, + adapter, + observer, + incoming, + rpc, + payload, + outbound, + _spine: spine, + _dir: dir, + }; capture.crank(); for peer in 1..=2u8 { capture.observer.produce(PeerEvent::P2pNewConnection { diff --git a/crates/e2e/Cargo.toml b/crates/e2e/Cargo.toml index f01e688b..3ede6756 100644 --- a/crates/e2e/Cargo.toml +++ b/crates/e2e/Cargo.toml @@ -18,7 +18,7 @@ serde_json.workspace = true silver_beacon_state.workspace = true silver_beacon_state_data.workspace = true silver_columns.workspace = true -silver_common.workspace = true +silver_common = { workspace = true, features = ["test-util"] } silver_config.workspace = true silver_metrics.workspace = true silver_control.workspace = true @@ -27,7 +27,6 @@ silver_gossip.workspace = true silver_network.workspace = true silver_peer.workspace = true snap.workspace = true -tempfile = "3" tracing.workspace = true tracing-subscriber.workspace = true diff --git a/crates/e2e/examples/gossip_oneway.rs b/crates/e2e/examples/gossip_oneway.rs index 534b4850..209ac437 100644 --- a/crates/e2e/examples/gossip_oneway.rs +++ b/crates/e2e/examples/gossip_oneway.rs @@ -42,13 +42,14 @@ use mimalloc::MiMalloc; use rand::{Rng, RngCore}; #[cfg(feature = "alloc-profile")] use silver_common::metrics::CountingAllocator; -use silver_common::{GossipMsgOut, GossipTopic, NewGossipMsg, P2pSend, PeerEvent, TRandomAccess}; +use silver_common::{ + GossipMsgOut, GossipTopic, NewGossipMsg, P2pSend, PeerEvent, TRandomAccess, test_util::ShmemDir, +}; use silver_e2e::{ EchoCompressionHalf, EchoNetworkHalf, EchoStack, PublisherStack, Stats, inject::{build_publish_frame, snappy_compress}, keypair_from_seed, }; -use tempfile::TempDir; const DEFAULT_DURATION_S: u64 = 3; const DEFAULT_RATE_HZ: u64 = 500; @@ -71,7 +72,7 @@ fn main() { let args = parse_args(); assert!(args.payload_size >= 8, "payload-size must be >= 8 for the timestamp prefix"); - let tempdir = TempDir::new().expect("tempdir"); + let tempdir = ShmemDir::new().expect("tempdir"); let echo_addr = loopback_ephemeral(); let echo_disc_addr = loopback_ephemeral(); let pub_addr = loopback_ephemeral(); diff --git a/crates/e2e/examples/gossip_oneway_lh.rs b/crates/e2e/examples/gossip_oneway_lh.rs index 746a3ad2..78deb5d7 100644 --- a/crates/e2e/examples/gossip_oneway_lh.rs +++ b/crates/e2e/examples/gossip_oneway_lh.rs @@ -31,13 +31,12 @@ use flux::{tile::Tile, timing::Nanos}; use rand::{Rng, RngCore}; #[cfg(feature = "alloc-profile")] use silver_common::metrics::CountingAllocator; -use silver_common::{GossipMsgOut, GossipTopic, P2pSend, PeerEvent, PeerId}; +use silver_common::{GossipMsgOut, GossipTopic, P2pSend, PeerEvent, PeerId, test_util::ShmemDir}; use silver_e2e::{ LhGossipClient, PublisherStack, Stats, inject::{build_publish_frame, snappy_compress}, keypair_from_seed, }; -use tempfile::TempDir; const DEFAULT_DURATION_S: u64 = 3; const DEFAULT_RATE_HZ: u64 = 500; @@ -195,8 +194,8 @@ fn subscriber_thread( stats_tx.send(sub.stats).expect("send stats"); } -fn build_silver_publisher() -> io::Result<(PublisherStack, TempDir)> { - let tempdir = TempDir::new()?; +fn build_silver_publisher() -> io::Result<(PublisherStack, ShmemDir)> { + let tempdir = ShmemDir::new()?; let addr = loopback_ephemeral()?; let disc_addr = loopback_ephemeral()?; let kp = keypair_from_seed(11); diff --git a/crates/e2e/examples/gossip_oneway_lh_b.rs b/crates/e2e/examples/gossip_oneway_lh_b.rs index 2b57b469..4199a3f0 100644 --- a/crates/e2e/examples/gossip_oneway_lh_b.rs +++ b/crates/e2e/examples/gossip_oneway_lh_b.rs @@ -39,11 +39,12 @@ use std::{ use flux::{tile::Tile, timing::Nanos}; use rand::RngCore; -use silver_common::{GossipTopic, NewGossipMsg, PeerControl, PeerEvent, PeerId, TRandomAccess}; +use silver_common::{ + GossipTopic, NewGossipMsg, PeerControl, PeerEvent, PeerId, TRandomAccess, test_util::ShmemDir, +}; use silver_e2e::{ EchoCompressionHalf, EchoNetworkHalf, EchoStack, LhGossipClient, Stats, keypair_from_seed, }; -use tempfile::TempDir; const DEFAULT_DURATION_S: u64 = 3; const DEFAULT_RATE_HZ: u64 = 500; @@ -57,7 +58,7 @@ fn main() { let args = parse_args(); assert!(args.payload_size >= 8, "payload-size must be >= 8 for the timestamp prefix"); - let tempdir = TempDir::new().expect("tempdir"); + let tempdir = ShmemDir::new().expect("tempdir"); let echo_addr = loopback_ephemeral().expect("port"); let echo_disc_addr = loopback_ephemeral().expect("port"); let echo_kp = keypair_from_seed(12); diff --git a/crates/e2e/src/bin/da_replay.rs b/crates/e2e/src/bin/da_replay.rs index d2aad66b..4bca5bb7 100644 --- a/crates/e2e/src/bin/da_replay.rs +++ b/crates/e2e/src/bin/da_replay.rs @@ -34,6 +34,7 @@ use silver_common::{ TCache, TCacheProducer, TProducer, profiler::InProcessReader, ssz_view::{DataColumnSidecarFuluView, NUMBER_OF_COLUMNS, STATUS_V2_SIZE}, + test_util::ShmemDir, ticker::SlotTicker, }; use silver_e2e::{ @@ -44,7 +45,6 @@ use silver_metrics::{ fold_stats, table::{Column, Table}, }; -use tempfile::TempDir; #[cfg(not(feature = "alloc-profile"))] #[global_allocator] @@ -78,7 +78,7 @@ struct Node { gossip_p: TProducer, _state: BeaconStateOwner, _spine: Box, - _base: TempDir, + _base: ShmemDir, } #[derive(Default)] @@ -100,7 +100,7 @@ impl ReplayCost { impl Node { fn boot(state_ssz: &[u8], custody: u128) -> Self { - let base = TempDir::new().expect("tempdir"); + let base = ShmemDir::new().expect("tempdir"); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let spec = Arc::new(SpecConfig::mainnet()); diff --git a/crates/e2e/src/harness.rs b/crates/e2e/src/harness.rs index 37f76f78..62931671 100644 --- a/crates/e2e/src/harness.rs +++ b/crates/e2e/src/harness.rs @@ -11,8 +11,9 @@ use std::{ }; use flux::{tile::Tile, timing::Nanos}; -use silver_common::{GossipMsgOut, GossipTopic, NewGossipMsg, P2pSend, PeerEvent}; -use tempfile::TempDir; +use silver_common::{ + GossipMsgOut, GossipTopic, NewGossipMsg, P2pSend, PeerEvent, test_util::ShmemDir, +}; use crate::{ inject::{InjectError, build_publish_frame, snappy_compress}, @@ -37,7 +38,7 @@ pub struct TwoStackHarness { last_msg: Option, /// Kept alive so tempdir is retained. - _tempdir: TempDir, + _tempdir: ShmemDir, } impl TwoStackHarness { @@ -45,7 +46,7 @@ impl TwoStackHarness { /// keypairs from seeds 1 (publisher) and 2 (echo). pub fn new(fork_digest_hex: impl Into) -> io::Result { let fork_digest_hex: String = fork_digest_hex.into(); - let tempdir = TempDir::new()?; + let tempdir = ShmemDir::new()?; let publisher_kp = keypair_from_seed(1); let echo_kp = keypair_from_seed(2); diff --git a/crates/e2e/src/utils.rs b/crates/e2e/src/utils.rs index aa6197a3..f8a0a7f8 100644 --- a/crates/e2e/src/utils.rs +++ b/crates/e2e/src/utils.rs @@ -16,13 +16,13 @@ use silver_common::{ BeaconBlocksByRangeRequestView, METADATA_SIZE, STATUS_V2_SIZE, SignedBeaconBlockView, StatusView, }, + test_util::ShmemDir, ticker::SlotTicker, }; use silver_config::{ScoreParams, SyncingConfig}; use silver_control::{Controller, sync_engine::SyncEngine}; use silver_gossip::GossipHandler; use silver_peer::PeerManager; -use tempfile::TempDir; use crate::perf::BlockFixtures; @@ -108,7 +108,7 @@ pub struct PmBsHarness { rpc_p: TProducer, _gossip_p: TProducer, _spine: Box, - _base: TempDir, + _base: ShmemDir, local: StatusBytes, fork_digest: [u8; 4], } @@ -117,7 +117,7 @@ impl PmBsHarness { /// The rpc-inbound cache is sized to hold `n_blocks` mainnet blocks /// (~300 KB each). pub fn new(checkpoint: &[u8], n_blocks: usize) -> Self { - let base = TempDir::new().expect("tempdir"); + let base = ShmemDir::new().expect("tempdir"); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); // Position genesis so `current_slot() ≈ checkpoint slot`; otherwise diff --git a/crates/e2e/tests/lh_common/mod.rs b/crates/e2e/tests/lh_common/mod.rs index aa3c18b6..9dcb4a76 100644 --- a/crates/e2e/tests/lh_common/mod.rs +++ b/crates/e2e/tests/lh_common/mod.rs @@ -13,9 +13,9 @@ use flux::tile::Tile; use silver_common::{ Identify, P2pSend, PeerEvent, PeerId, RpcOutbound, RpcRequestOutbound, ssz_view::{METADATA_SIZE, STATUS_V2_SIZE}, + test_util::ShmemDir, }; use silver_e2e::{LhClient, PublisherStack, keypair_from_seed}; -use tempfile::TempDir; pub fn pick_free_port() -> u16 { let s = std::net::UdpSocket::bind(("127.0.0.1", 0)).expect("bind"); @@ -26,8 +26,8 @@ pub fn pick_free_port() -> u16 { /// kept-alive tempdir. Disables the controller's heartbeat-driven /// outbound Ping fan-out so tests assert against deterministic RPC /// traffic only. -pub fn build_silver_listener(seed: u8) -> (PublisherStack, TempDir) { - let tempdir = TempDir::new().expect("tempdir"); +pub fn build_silver_listener(seed: u8) -> (PublisherStack, ShmemDir) { + let tempdir = ShmemDir::new().expect("tempdir"); let addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), pick_free_port()); let disc_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), pick_free_port()); let kp = keypair_from_seed(seed); diff --git a/crates/e2e/tests/lh_rpc.rs b/crates/e2e/tests/lh_rpc.rs index c3822bcf..dce59dad 100644 --- a/crates/e2e/tests/lh_rpc.rs +++ b/crates/e2e/tests/lh_rpc.rs @@ -16,6 +16,7 @@ use lh_common::{ use silver_common::{ RpcRequest, ssz_view::{METADATA_SIZE, STATUS_V2_SIZE}, + test_util::ShmemDir, }; use silver_e2e::{LhClient, lh_client}; @@ -124,9 +125,7 @@ fn silver_responds_to_metadata() { // silver's stream completes — silver's reception of that response is // covered separately by controller unit tests. -fn drive_silver_dialer( - seed: u8, -) -> (silver_e2e::PublisherStack, LhClient, usize, tempfile::TempDir) { +fn drive_silver_dialer(seed: u8) -> (silver_e2e::PublisherStack, LhClient, usize, ShmemDir) { let (mut silver, td) = build_silver_listener(seed); let mut client = LhClient::new_listener(); client.set_auto_response(PING_PROTOCOL, vec![0u8; 8]); diff --git a/crates/e2e/tests/rpc_multipart.rs b/crates/e2e/tests/rpc_multipart.rs index 7289bd26..1180b4df 100644 --- a/crates/e2e/tests/rpc_multipart.rs +++ b/crates/e2e/tests/rpc_multipart.rs @@ -20,10 +20,9 @@ use flux::tile::Tile; use silver_common::{ P2pSend, P2pStreamId, PeerEvent, RpcInbound, RpcOutbound, RpcRequest, RpcRequestInbound, RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, StreamProtocol, - TCacheProducer, ssz_view::BLOCKS_BY_RANGE_REQ_SIZE, + TCacheProducer, ssz_view::BLOCKS_BY_RANGE_REQ_SIZE, test_util::ShmemDir, }; use silver_e2e::{PublisherStack, keypair_from_seed}; -use tempfile::TempDir; const CHUNK_BYTES: usize = 2 * 1024 * 1024; const CHUNK_COUNT: usize = 3; @@ -37,7 +36,7 @@ fn pick_free_port() -> u16 { .port() } -fn build_stack(td: &TempDir, suffix: &str, seed: u8) -> PublisherStack { +fn build_stack(td: &ShmemDir, suffix: &str, seed: u8) -> PublisherStack { let addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), pick_free_port()); let disc = SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), pick_free_port()); let kp = keypair_from_seed(seed); @@ -130,7 +129,7 @@ fn synth_block_bytes(chunk_index: u8, len: usize) -> Vec { fn silver_receives_multipart_blocks_by_range_response() { tracing_subscriber::fmt().with_max_level(tracing::Level::INFO).try_init().ok(); - let td = TempDir::new().expect("tempdir"); + let td = ShmemDir::new().expect("tempdir"); let mut requester = build_stack(&td, "_req", 21); let mut responder = build_stack(&td, "_resp", 22); diff --git a/crates/metrics/Cargo.toml b/crates/metrics/Cargo.toml index 5c97a6d4..308434d5 100644 --- a/crates/metrics/Cargo.toml +++ b/crates/metrics/Cargo.toml @@ -14,6 +14,7 @@ serde.workspace = true serde_json.workspace = true [dev-dependencies] +tempfile = "3" flux-profiler = { workspace = true, features = ["test-util"] } [lints] diff --git a/crates/metrics/src/lib.rs b/crates/metrics/src/lib.rs index 625cc293..60dd81c7 100644 --- a/crates/metrics/src/lib.rs +++ b/crates/metrics/src/lib.rs @@ -250,6 +250,7 @@ macro_rules! declare_counters { #[cfg(test)] mod tests { + use tempfile::TempDir; crate::declare_counters! { TestCounters => "test_metrics" { Alpha, @@ -260,8 +261,8 @@ mod tests { #[test] fn round_trip() { - let tmp = std::env::temp_dir().join(format!("silver_metrics_test_{}", std::process::id())); - TestCounters::init_with_base(&tmp, "round_trip").unwrap(); + let tmp = TempDir::new().unwrap(); + TestCounters::init_with_base(tmp.path(), "round_trip").unwrap(); TestCounters::Alpha.set(0); TestCounters::Beta.set(0); @@ -277,7 +278,5 @@ mod tests { assert_eq!(TestCounters::NAMES, &["Alpha", "Beta", "Gamma"]); assert_eq!(TestCounters::COUNT, 3); - - std::fs::remove_dir_all(&tmp).ok(); } } diff --git a/crates/storage/Cargo.toml b/crates/storage/Cargo.toml index a2c48c9b..3db9b0d6 100644 --- a/crates/storage/Cargo.toml +++ b/crates/storage/Cargo.toml @@ -20,4 +20,6 @@ tracing.workspace = true workspace = true [dev-dependencies] +silver_common = { workspace = true, features = ["test-util"] } rand.workspace = true +tempfile = "3" diff --git a/crates/storage/src/store/checkpoint.rs b/crates/storage/src/store/checkpoint.rs index e6dbe2fc..18399f5f 100644 --- a/crates/storage/src/store/checkpoint.rs +++ b/crates/storage/src/store/checkpoint.rs @@ -350,6 +350,7 @@ mod tests { }; use silver_beacon_state_data::{BeaconState, BeaconStateOwner, SpecConfig}; + use tempfile::TempDir; use super::{FINALIZED_CHECKPOINTS_DIR, Store}; use crate::tile::IoEvent; @@ -426,9 +427,9 @@ mod tests { // the idempotency guard (the slot stays fixed — same per-slot dir, // overwritten in place). Unique subdir under SILVER_BENCH_DIR (default // `/tmp`; point it at the data-store disk — tmpfs understates fsync). - // Removed at the end. let base = std::env::var("SILVER_BENCH_DIR").unwrap_or_else(|_| "/tmp".to_string()); - let dir = format!("{base}/silver_bench_persist_{}", rand::random::()); + let temp = TempDir::new_in(base).unwrap(); + let dir = temp.path().to_str().unwrap().to_owned(); let mut store = Store::load(dir.clone(), crate::store::test_spec(u64::MAX), 0).unwrap(); const ITERS: u64 = 23; @@ -504,8 +505,6 @@ mod tests { for (idx, total) in per_section.iter().enumerate() { println!(" {}: {:?}", SECTION_NAMES[idx], *total / counted); } - - let _ = std::fs::remove_dir_all(&dir); } fn report(name: &str, samples: &mut [Duration], bytes: usize) { @@ -523,8 +522,8 @@ mod tests { #[test] fn checkpoint_persist_retention_and_load() { - let dir = format!("/tmp/silver_storage_cp_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&dir); + let temp = TempDir::new().unwrap(); + let dir = temp.path().to_str().unwrap().to_owned(); let mut store = Store::load(dir.clone(), crate::store::test_spec(u64::MAX), 0).unwrap(); assert_eq!(store.last_persisted_finalized_slot(), 0); @@ -556,8 +555,6 @@ mod tests { assert_eq!(reloaded.last_persisted_finalized_slot(), 13); assert!(!cp.join("99").exists(), "incomplete checkpoint dropped on load"); assert!(cp.join("13").exists()); - - let _ = std::fs::remove_dir_all(&dir); } /// End-to-end streamed persist: `begin_checkpoint` arms a job, `file_io` @@ -573,8 +570,8 @@ mod tests { let owner = published_owner(silver_beacon_state_data::BeaconState::empty_test(64)); let reader = owner.reader(); - let dir = format!("/tmp/silver_storage_streamcp_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&dir); + let temp = TempDir::new().unwrap(); + let dir = temp.path().to_str().unwrap().to_owned(); let mut store = Store::load(dir.clone(), crate::store::test_spec(u64::MAX), 0).unwrap(); assert!(!store.checkpoint_in_flight()); @@ -608,7 +605,5 @@ mod tests { // Sidecar committed alongside; zero validators => empty pubkey vec. let pk = std::fs::read(slot_dir.join("64.pubkeys")).unwrap(); assert!(decode_checkpoint_pubkeys(&pk).unwrap().is_empty()); - - let _ = std::fs::remove_dir_all(&dir); } } diff --git a/crates/storage/src/store/tests.rs b/crates/storage/src/store/tests.rs index 5878c5c7..18c036a1 100644 --- a/crates/storage/src/store/tests.rs +++ b/crates/storage/src/store/tests.rs @@ -9,6 +9,7 @@ use std::{ }; use silver_common::{Prefill, SyncNeed, SyncUpdate}; +use tempfile::TempDir; use super::{column_path, envelope_path, slot_dir}; @@ -36,8 +37,8 @@ use crate::tile::IoEvent; #[test] fn concurrent_read_write() { - let path = format!("/tmp/silver_storage_rw_{}.txt", rand::random::()); - let _ = std::fs::remove_file(&path); + let dir = TempDir::new().unwrap(); + let path = dir.path().join("storage.txt").to_str().unwrap().to_owned(); let mut file = super::io::open_file_write(&path, false).unwrap(); let mut handles = vec![]; @@ -93,8 +94,8 @@ fn fork_tree_persist_serve_promote() { RpcResponseOutbound, StreamProtocol, TCache, TCacheProducer, }; - let store_path = format!("/tmp/test_store_fork_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); // Two competing blocks at slot 42 sharing parent CC: A canonical, B fork. @@ -248,8 +249,6 @@ fn fork_tree_persist_serve_promote() { let reloaded = load_fulu(store_path.clone()); assert_eq!(reloaded.finalized.slot_of(&root_a), Some(slot)); assert!(reloaded.unfinalized.is_empty()); - - let _ = std::fs::remove_dir_all(&store_path); } // Envelopes: persist unfinalized, promote the canonical one to the flat @@ -258,8 +257,8 @@ fn fork_tree_persist_serve_promote() { fn envelope_persist_promote_prune() { use silver_common::{TCache, TCacheProducer}; - let store_path = format!("/tmp/test_store_env_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_gloas(store_path.clone()); // Canonical block A + fork block B at slot 42, shared parent CC; each @@ -350,8 +349,6 @@ fn envelope_persist_promote_prune() { // Reload rebuilds the (now empty) unfinalized envelope index. let reloaded = load_gloas(store_path.clone()); assert!(reloaded.unfinalized_envelopes.is_empty()); - - let _ = std::fs::remove_dir_all(&store_path); } // A self-parenting block (cycle) must not hang the canonical walk. @@ -361,8 +358,8 @@ fn range_query_terminates_on_cycle() { P2pStreamId, RpcRequest, RpcRequestInbound, StreamProtocol, TCache, TCacheProducer, }; - let store_path = format!("/tmp/test_store_cycle_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); // parent_root == block_root: a self-loop in the fork tree. @@ -397,8 +394,6 @@ fn range_query_terminates_on_cycle() { }); // Reaching here proves the walk terminated. assert!(!store.query_queue.is_empty()); - - let _ = std::fs::remove_dir_all(&store_path); } #[test] @@ -409,8 +404,8 @@ fn envelope_range_request_served_empty() { ssz_view::EXECUTION_PAYLOAD_ENVELOPES_BY_RANGE_REQ_SIZE, }; - let store_path = format!("/tmp/test_store_env_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); // We don't persist envelopes, but an inbound range request must still get @@ -455,8 +450,8 @@ fn column_fork_persist_serve_promote() { RpcResponseOutbound, StreamProtocol, TCache, TCacheProducer, ssz_view::DC_BY_RANGE_REQ_MAX, }; - let store_path = format!("/tmp/test_store_colfork_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); let ucol_dir = store.unfinalized_dir(super::Payload::Column); let flat_dir = store.finalized_slot_dir(super::Payload::Column, 42); @@ -636,14 +631,12 @@ fn column_fork_persist_serve_promote() { let reloaded = load_fulu(store_path.clone()); assert!(reloaded.unfinalized_columns.is_empty()); assert_eq!(reloaded.finalized.slot_of(&root_a), Some(slot)); - - let _ = std::fs::remove_dir_all(&store_path); } #[test] fn backfill_block_persists_its_index_record() { - let store_path = format!("/tmp/test_store_backfill_index_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let temp = TempDir::new().unwrap(); + let store_path = temp.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); let slot = 64u64; @@ -670,14 +663,12 @@ fn backfill_block_persists_its_index_record() { let dir = store.finalized_slot_dir(super::Payload::Block, slot); assert_eq!(index_records(&dir), vec![super::block_index::Record { block_root, slot }]); - - let _ = std::fs::remove_dir_all(&store_path); } #[test] fn column_already_on_disk_is_not_written_again() { - let store_path = format!("/tmp/test_store_col_dedupe_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); let (slot, block_root) = (9u64, [3u8; 32]); @@ -690,7 +681,6 @@ fn column_already_on_disk_is_not_written_again() { assert_eq!(store.write_queue.len(), 1, "the second is already on disk"); drain(&mut store).unwrap(); - let _ = std::fs::remove_dir_all(&store_path); } /// A failing write is dropped, not retried. The coverage and the report @@ -747,8 +737,8 @@ fn failed_write_is_neither_held_nor_reported() { /// store can drop the second copy. #[test] fn envelope_already_on_disk_is_not_written_again() { - let store_path = format!("/tmp/test_store_env_dedupe_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_gloas(store_path.clone()); let (slot, block_root, parent_root) = (9u64, [3u8; 32], [0u8; 32]); @@ -767,7 +757,6 @@ fn envelope_already_on_disk_is_not_written_again() { assert_eq!(store.write_queue.len(), 2, "the second is already on disk"); drain(&mut store).unwrap(); - let _ = std::fs::remove_dir_all(&store_path); } /// Synthetic fulu `SignedBeaconBlock` carrying blob commitments, so the @@ -1121,8 +1110,8 @@ fn range_queries_interleave_fairly() { RpcResponseOutbound, StreamProtocol, TCache, TCacheProducer, }; - let store_path = format!("/tmp/test_store_fair_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_path); + let dir = TempDir::new().unwrap(); + let store_path = dir.path().to_str().unwrap().to_owned(); let mut store = load_fulu(store_path.clone()); // Chain of two unfinalized blocks: slot 10 (parent CC) ← slot 11. @@ -1195,8 +1184,6 @@ fn range_queries_interleave_fairly() { }; assert!(matches!(response, RpcResponse::Complete)); } - - let _ = std::fs::remove_dir_all(&store_path); } /// An unfinalized file that vanished before finality is not promoted. The diff --git a/crates/storage/src/tile.rs b/crates/storage/src/tile.rs index 9d6fcd5a..9933b7ca 100644 --- a/crates/storage/src/tile.rs +++ b/crates/storage/src/tile.rs @@ -479,7 +479,8 @@ impl IoEvent { #[cfg(test)] mod tests { use silver_beacon_state_data::BeaconStateOwner; - use silver_common::{DataColumnsEvent, TCache, TCacheProducer}; + use silver_common::{DataColumnsEvent, TCache, TCacheProducer, test_util::ShmemDir}; + use tempfile::TempDir; use super::*; @@ -509,34 +510,33 @@ mod tests { // Only the unavailable block is dropped; replay continues past it and // ends with Done so the peer manager resyncs the gap. let custody = (1u128 << 3) | (1u128 << 7); - let store_dir = format!("/tmp/test_storage_replay_da_{}", rand::random::()); - let _ = std::fs::remove_dir_all(&store_dir); + let store_dir = TempDir::new().unwrap(); // Committed-checkpoint marker → last_persisted_finalized_slot = 32. - let ckpt = format!("{store_dir}/finalized_checkpoints/32"); + let ckpt = store_dir.path().join("finalized_checkpoints/32"); std::fs::create_dir_all(&ckpt).unwrap(); - std::fs::write(format!("{ckpt}/32.ssz"), b"x").unwrap(); + std::fs::write(ckpt.join("32.ssz"), b"x").unwrap(); // Unfinalized blocks: `__.ssz`. The root in the // name keys the column bitmask; needs-columns is parsed from the bytes. - let unfin = format!("{store_dir}/unfinalized"); + let unfin = store_dir.path().join("unfinalized"); std::fs::create_dir_all(&unfin).unwrap(); let (root_a, root_b, root_c) = ("a".repeat(64), "b".repeat(64), "c".repeat(64)); for (slot, root, dc) in [(33, &root_a, true), (34, &root_b, true), (35, &root_c, false)] { std::fs::write( - format!("{unfin}/{slot}_{}_{}.ssz", "0".repeat(64), root), + unfin.join(format!("{slot}_{}_{}.ssz", "0".repeat(64), root)), make_block(slot, dc), ) .unwrap(); } // Custody columns on disk: `__.ssz`. - let cols = format!("{store_dir}/unfinalized_columns"); + let cols = store_dir.path().join("unfinalized_columns"); std::fs::create_dir_all(&cols).unwrap(); for col in [3, 7] { - std::fs::write(format!("{cols}/33_{root_a}_{col}.ssz"), b"c").unwrap(); + std::fs::write(cols.join(format!("33_{root_a}_{col}.ssz")), b"c").unwrap(); } - std::fs::write(format!("{cols}/34_{root_b}_3.ssz"), b"c").unwrap(); // partial + std::fs::write(cols.join(format!("34_{root_b}_3.ssz")), b"c").unwrap(); // partial let pg_tc = TCache::producer("pg", 1 << 20); let rpc_tc = TCache::producer("r", 1 << 20); @@ -553,15 +553,14 @@ mod tests { BeaconStateOwner::empty_test(0).reader(), custody, Arc::new(SpecConfig::mainnet()), - store_dir.clone(), + store_dir.path().to_str().unwrap().to_owned(), true, ); assert_eq!(tile.replay_steps.len(), 3, "skip decided at replay, not load"); // Spine + injector: the tile produces, the injector drains. - let base = std::env::temp_dir().join(format!("silver-replay-da-{}", rand::random::())); - std::fs::create_dir_all(&base).unwrap(); - let mut spine = Box::new(SilverSpine::new_with_base_dir(&base, None)); + let base = ShmemDir::new().unwrap(); + let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); let mut tile_adapter = SpineAdapter::connect_tile(&tile, &mut spine); let inj = Injector; let mut inj_adapter = SpineAdapter::connect_tile(&inj, &mut spine); @@ -585,9 +584,6 @@ mod tests { assert_eq!(blocks, 2, "slots 33 and 35 replayed; 34 skipped"); assert_eq!(done, 1, "replay terminated with Done"); assert!(tile.replay_done); - - let _ = std::fs::remove_dir_all(&store_dir); - let _ = std::fs::remove_dir_all(&base); } struct Injector; diff --git a/crates/surfer/Cargo.toml b/crates/surfer/Cargo.toml index 874b8ea0..d740a659 100644 --- a/crates/surfer/Cargo.toml +++ b/crates/surfer/Cargo.toml @@ -24,3 +24,7 @@ silver_storage.workspace = true [lints] workspace = true + +[dev-dependencies] +tempfile = "3" +silver_common = { workspace = true, features = ["test-util"] } diff --git a/crates/surfer/src/sources/counters.rs b/crates/surfer/src/sources/counters.rs index 50b9afe1..35813ad6 100644 --- a/crates/surfer/src/sources/counters.rs +++ b/crates/surfer/src/sources/counters.rs @@ -278,6 +278,7 @@ fn mmap_readonly_bytes(path: &Path, bytes: usize) -> io::Result<*const u8> { #[cfg(test)] mod tests { use silver_common::declare_counters; + use tempfile::TempDir; declare_counters! { SurferTestCounters => "surfer_smoke" { @@ -289,8 +290,8 @@ mod tests { #[test] fn discover_open_sample() { - let tmp = std::env::temp_dir().join(format!("surfer_smoke_{}", std::process::id())); - SurferTestCounters::init_with_base(&tmp, "surfer_test").unwrap(); + let tmp = TempDir::new().unwrap(); + SurferTestCounters::init_with_base(tmp.path(), "surfer_test").unwrap(); SurferTestCounters::Alpha.set(0); SurferTestCounters::Beta.set(0); @@ -298,7 +299,7 @@ mod tests { SurferTestCounters::Alpha.add(11); SurferTestCounters::Beta.set(42); - let sources = crate::discovery::discover(&tmp, "surfer_test").unwrap(); + let sources = crate::discovery::discover(tmp.path(), "surfer_test").unwrap(); let file = sources.counters.iter().find(|f| f.name == "surfer_smoke").unwrap(); let mut set = super::CounterSet::open(file).unwrap(); @@ -315,8 +316,6 @@ mod tests { set.sample(); assert_eq!(set.current[0], 16); assert_eq!(set.previous[0], 11); - - std::fs::remove_dir_all(&tmp).ok(); } } diff --git a/crates/surfer/src/sources/tilemetrics.rs b/crates/surfer/src/sources/tilemetrics.rs index ca5cb530..4bb86c91 100644 --- a/crates/surfer/src/sources/tilemetrics.rs +++ b/crates/surfer/src/sources/tilemetrics.rs @@ -129,6 +129,7 @@ mod tests { tile::metrics::TileMetrics, timing::{IngestionTime, Nanos}, }; + use silver_common::test_util::ShmemDir; use super::*; use crate::discovery::TileMetricsFile; @@ -146,12 +147,9 @@ mod tests { /// Stand up a real shmem queue + consumer at a unique path. Open the /// consumer BEFORE producing: broadcast cursors start at head, so a late /// join skips earlier messages. - fn rig(tag: &str) -> (Producer, TileMetricsSet, std::path::PathBuf) { - let tmp = - std::env::temp_dir().join(format!("surfer_tileutil_{tag}_{}", std::process::id())); - std::fs::remove_dir_all(&tmp).ok(); - std::fs::create_dir_all(&tmp).unwrap(); - let path = tmp.join(format!("tilemetrics-{tag}")); + fn rig(tag: &str) -> (ShmemDir, Producer, TileMetricsSet) { + let tmp = ShmemDir::new().unwrap(); + let path = tmp.path().join(format!("tilemetrics-{tag}")); let queue: Queue = Queue::create_or_open_shared(&path, 4096, QueueType::SPMC); let producer = Producer::from(queue); let file = TileMetricsFile { name: tag.into(), path: path.clone() }; @@ -160,14 +158,14 @@ mod tests { // anchors on first consume, so a pre-produce drain avoids skipping the // backlog (mirrors main.rs's startup drain). set.drain(); - (producer, set, tmp) + (tmp, producer, set) } /// Drives drain → roll_bucket → util_avg/util_peak end-to-end through the /// queue, isolating the surfer arithmetic from the flux producer path. #[test] fn util_calculation_over_buckets() { - let (mut producer, mut set, tmp) = rig("calc"); + let (_tmp, mut producer, mut set) = rig("calc"); // No buckets rolled yet. assert_eq!(set.util_avg(), 0.0); @@ -190,15 +188,13 @@ mod tests { assert_eq!(set.total_busy, 900); assert_eq!(set.total_ticks, 3000); assert_eq!(set.samples_seen, 3); - - std::fs::remove_dir_all(&tmp).ok(); } /// A nonzero-busy sample MUST produce nonzero util. If the live TUI shows /// zeros, busy_ticks is zero upstream (flux/did_work), not here. #[test] fn nonzero_busy_yields_nonzero_util() { - let (mut producer, mut set, tmp) = rig("nz"); + let (_tmp, mut producer, mut set) = rig("nz"); producer.produce(&mk(1, 1_000_000)); set.drain(); @@ -207,8 +203,6 @@ mod tests { assert!(set.util_avg() > 0.0, "avg={}", set.util_avg()); assert!(set.util_peak() > 0.0, "peak={}", set.util_peak()); assert_eq!(set.total_busy, 1); - - std::fs::remove_dir_all(&tmp).ok(); } /// Producer-side bracket: drive the real flux @@ -218,15 +212,13 @@ mod tests { /// the flux timing attribution itself works. #[test] fn flux_producer_attributes_busy() { - let tmp = std::env::temp_dir().join(format!("surfer_fluxprod_{}", std::process::id())); - std::fs::remove_dir_all(&tmp).ok(); - std::fs::create_dir_all(&tmp).unwrap(); + let tmp = ShmemDir::new().unwrap(); // TileMetrics::new creates the queue under the app's shmem dir. - let mut tm = TileMetrics::new(&tmp, "fluxprodapp", "fluxprod"); + let mut tm = TileMetrics::new(tmp.path(), "fluxprodapp", "fluxprod"); // Attach the consumer before producing (prime cursor at head). - let sources = crate::discovery::discover(&tmp, "fluxprodapp").unwrap(); + let sources = crate::discovery::discover(tmp.path(), "fluxprodapp").unwrap(); let file = sources.tilemetrics.iter().find(|f| f.name == "fluxprod").unwrap(); let mut set = TileMetricsSet::open(file).unwrap(); set.drain(); @@ -248,7 +240,5 @@ mod tests { assert!(set.samples_seen >= 1, "no sample emitted"); assert!(set.total_busy > 0, "flux attributed zero busy despite did_work=true"); assert!(set.util_avg() > 0.0, "avg={}", set.util_avg()); - - std::fs::remove_dir_all(&tmp).ok(); } } diff --git a/crates/telemetry/Cargo.toml b/crates/telemetry/Cargo.toml index ecd200f2..2d189a99 100644 --- a/crates/telemetry/Cargo.toml +++ b/crates/telemetry/Cargo.toml @@ -24,6 +24,7 @@ tracing.workspace = true ureq.workspace = true [dev-dependencies] +tempfile = "3" flux-profiler = { workspace = true, features = ["test-util"] } [lints] diff --git a/crates/telemetry/src/collector.rs b/crates/telemetry/src/collector.rs index e0c0e6d6..03fb330a 100644 --- a/crates/telemetry/src/collector.rs +++ b/crates/telemetry/src/collector.rs @@ -273,6 +273,7 @@ mod tests { use flate2::read::MultiGzDecoder; use flux_profiler::{enable_profiler, test_shmem::ShmemGuard, timed}; + use tempfile::TempDir; use super::*; @@ -328,10 +329,8 @@ mod tests { let mut reader = CrossProcessReader::attach(guard.app()).expect("pid published"); while reader.poll() {} - let dir = std::env::temp_dir().join(format!("segments-{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); - let mut collector = collector(reader, dir.clone()); + let dir = TempDir::new().unwrap(); + let mut collector = collector(reader, dir.path().to_owned()); let started_at = Nanos::from_secs(STARTED_AT); collector.append(started_at); @@ -340,15 +339,13 @@ mod tests { let pid = collector.reader.pid(); assert_eq!( - names(&dir), + names(dir.path()), [ format!("{APP_NAME}_2001-09-09_01-00-00_pid{pid}.fxt.gz"), format!("{APP_NAME}_2001-09-09_02-00-00_pid{pid}.fxt.gz"), ], "both mid-hour appends share 01-00-00; the one an hour on opens 02-00-00" ); - - std::fs::remove_dir_all(&dir).unwrap(); } #[test] @@ -357,13 +354,11 @@ mod tests { enable_profiler(guard.app()); let mut reader = CrossProcessReader::attach(guard.app()).expect("pid published"); - let dir = std::env::temp_dir().join(format!("appends-{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); + let dir = TempDir::new().unwrap(); traced_work(); while reader.poll() {} - let mut collector = collector(reader, dir.clone()); + let mut collector = collector(reader, dir.path().to_owned()); let at = Nanos::from_secs(STARTED_AT); collector.append(at); @@ -373,7 +368,7 @@ mod tests { traced_work(); while collector.reader.poll() {} collector.append(at); - assert_eq!(names(&dir).len(), 1, "both appends went to the interval's own file"); + assert_eq!(names(dir.path()).len(), 1, "both appends went to the interval's own file"); assert!( std::fs::metadata(&path).unwrap().len() > after_first, "the second append extended the file" @@ -388,8 +383,6 @@ mod tests { 2, "one self-contained trace per append" ); - - std::fs::remove_dir_all(&dir).unwrap(); } fn segment(hour: u64) -> String { @@ -398,43 +391,39 @@ mod tests { /// Three equal segments, oldest first, and an older file that is not ours: /// counting or dropping `notes.txt` changes what the budget leaves behind. - fn segment_dir(name: &str) -> PathBuf { - let dir = std::env::temp_dir().join(format!("{name}-{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); + fn segment_dir() -> TempDir { + let dir = TempDir::new().unwrap(); - backdate(&dir.join("notes.txt"), 100, 5); + backdate(&dir.path().join("notes.txt"), 100, 5); for hour in 1..=3 { - backdate(&dir.join(segment(hour)), 100, 4 - hour); + backdate(&dir.path().join(segment(hour)), 100, 4 - hour); } dir } #[test] fn prunes_oldest_first_to_the_budget() { - let dir = segment_dir("prune-budget"); + let dir = segment_dir(); - TraceCollector::prune(&dir, 250); + TraceCollector::prune(dir.path(), 250); assert_eq!( - names(&dir), + names(dir.path()), ["notes.txt".to_owned(), segment(2), segment(3)], "300 bytes against a 250 budget drops the oldest, then stops once it fits" ); - std::fs::remove_dir_all(&dir).unwrap(); } #[test] fn keeps_the_newest_and_what_is_not_ours() { - let dir = segment_dir("prune-floor"); + let dir = segment_dir(); - TraceCollector::prune(&dir, 0); + TraceCollector::prune(dir.path(), 0); assert_eq!( - names(&dir), + names(dir.path()), ["notes.txt".to_owned(), segment(3)], "a budget under one segment still keeps the last cut" ); - std::fs::remove_dir_all(&dir).unwrap(); } } From 17ba40a84e4d79ed14b0891c9ac4be1ebea210e2 Mon Sep 17 00:00:00 2001 From: Bronek Kozicki Date: Fri, 11 Sep 2026 16:07:29 +0100 Subject: [PATCH 4/5] Enable all Cargo features in rust-analyzer Match the feature selection used by just clippy so analysis includes feature-gated test support such as silver_common's test-util. Assisted-by: Claude:claude-fable-5-1 Assisted-by: Codex:gpt-6-astra --- rust-analyzer.toml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 rust-analyzer.toml diff --git a/rust-analyzer.toml b/rust-analyzer.toml new file mode 100644 index 00000000..14df3063 --- /dev/null +++ b/rust-analyzer.toml @@ -0,0 +1,3 @@ +# Match the feature selection used by `just clippy`. +[cargo] +features = "all" From cc9488bcd0443acdf39e90b4d18d916ec0cc24f4 Mon Sep 17 00:00:00 2001 From: Bronek Kozicki Date: Mon, 14 Sep 2026 10:49:58 +0100 Subject: [PATCH 5/5] Carry decompressed SSZ handles on publication requests Replace SendGossip's per-topic metadata with a decompressed SSZ handle for every topic. Remove the column metadata from PublishDataColumn and delete GossipBlock, GossipDataColumn and GossipMetadata. Consumers derive the fields they need from the referenced bytes. The application boundary reads gossip and RPC caches to build block_gossip and data_column_sidecar events. It computes block roots from block bytes and extracts sidecar identities from Fulu or Gloas layouts. Cache read failures and unrecognized sidecar layouts produce warnings and no event. Block roots are computed even without subscribers. ADR-0004 documents these costs and failure conditions. Publish the boundary consumers' tails on every loop iteration so idle consumers can release cache space even when no publication requests arrive. A boundary test fills both caches and checks that producers can reserve space again after the idle interval and another tile iteration. Producer fixtures use distinct protobuf and SSZ payloads to expose incorrect handle forwarding. Boundary fixtures include a fixed block-body prefix to exercise fork-specific hashing. Tests cover topic selection, repeated requests, late subscriptions and engine saturation. Controller tests check routing and IWANT service with distinct payloads. Assisted-by: Claude:claude-fable-5-1 Assisted-by: Codex:gpt-6-astra --- Cargo.lock | 1 + crates/application_boundary/Cargo.toml | 1 + crates/application_boundary/src/lib.rs | 56 ++- crates/application_boundary/tests/tile.rs | 333 +++++++++++------- crates/beacon_state/tile/src/tile/block.rs | 9 +- crates/beacon_state/tile/src/tile/gossip.rs | 29 +- .../beacon_state/tile/src/tile/orphan_pool.rs | 2 +- crates/beacon_state/tile/src/tile/tests.rs | 76 ++-- .../tile/src/tile/tests/block_relay.rs | 39 +- crates/bin/src/main.rs | 6 + crates/columns/src/tile.rs | 63 ++-- crates/columns/src/tile/tests/publication.rs | 22 +- crates/common/src/column_util.rs | 28 +- crates/common/src/lib.rs | 5 +- crates/common/src/spine.rs | 14 +- crates/common/src/spine/messages.rs | 28 +- crates/control/src/tile.rs | 13 +- crates/control/src/tile/tests.rs | 31 +- crates/peer/src/manager/mod.rs | 2 +- crates/peer/src/manager/promises.rs | 9 +- docs/adr/0004-sync-materialized-api.md | 29 +- 21 files changed, 446 insertions(+), 350 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index de4310ed..23b7da8e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4545,6 +4545,7 @@ dependencies = [ "silver_config", "silver_engine_api", "silver_httpcore", + "tracing", "ureq", ] diff --git a/crates/application_boundary/Cargo.toml b/crates/application_boundary/Cargo.toml index a6c1adaf..a4bb267b 100644 --- a/crates/application_boundary/Cargo.toml +++ b/crates/application_boundary/Cargo.toml @@ -13,6 +13,7 @@ silver_common.workspace = true silver_config.workspace = true silver_engine_api.workspace = true silver_httpcore.workspace = true +tracing.workspace = true [dev-dependencies] silver_common = { workspace = true, features = ["test-util"] } diff --git a/crates/application_boundary/src/lib.rs b/crates/application_boundary/src/lib.rs index e8f76373..67f7bdab 100644 --- a/crates/application_boundary/src/lib.rs +++ b/crates/application_boundary/src/lib.rs @@ -4,8 +4,10 @@ use flux::{spine::SpineAdapter, tile::Tile}; use silver_beacon_api::{BeaconApi, SlotStatus}; use silver_beacon_state_data::{BeaconStateReader, SpecConfig}; use silver_common::{ - BeaconStateEvent, BlockStage, Enr, GossipBlock, GossipMetadata, Identify, Keypair, PeerEvent, - SilverSpine, SyncUpdate, TProducer, TRandomAccess, + BeaconStateEvent, BlockStage, Enr, GossipTopic, Identify, Keypair, PeerEvent, SilverSpine, + SyncUpdate, TProducer, TRandomAccess, TRead, + column_util::{SidecarIdentity, block_root}, + ssz_view::SignedBeaconBlockView, }; use silver_config::EngineConfig; use silver_engine_api::EngineApi; @@ -21,6 +23,9 @@ pub struct ApplicationBoundaryTile { readiness: Readiness, pub beacon: BeaconApi, engine: EngineApi, + spec: SpecConfig, + relayed_gossip: TRandomAccess, + relayed_rpc: TRandomAccess, } impl Tile for ApplicationBoundaryTile { @@ -50,6 +55,8 @@ impl ApplicationBoundaryTile { gossip_consumer: TRandomAccess, rpc_consumer: TRandomAccess, resp_producer: TProducer, + relayed_gossip: TRandomAccess, + relayed_rpc: TRandomAccess, ) -> Self { // A batch too small for every socket the tile can register leaves the // rest of a busy iteration's readiness for the next one. @@ -77,11 +84,14 @@ impl ApplicationBoundaryTile { rpc_consumer, resp_producer, ); - Self { readiness, beacon, engine } + Self { readiness, beacon, engine, spec: spec.clone(), relayed_gossip, relayed_rpc } } fn consume_spine_events(&mut self, adapter: &mut SpineAdapter) { - let beacon = &mut self.beacon; + let Self { beacon, spec, relayed_gossip, relayed_rpc, .. } = self; + // Publish tails even without reads so idle consumers can release cache space. + relayed_gossip.free(); + relayed_rpc.free(); // A consumer's first consume starts at the producer's write head. // Keep both event queues active during engine saturation; delaying @@ -100,18 +110,22 @@ impl ApplicationBoundaryTile { _ => {} }); adapter.consume(|event: PeerEvent, _| match event { - PeerEvent::SendGossip { - metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root })), - .. - } => beacon.publish_block_gossip(slot, &block_root), - PeerEvent::SendGossip { - metadata: Some(GossipMetadata::DataColumn(column)), .. - } | - PeerEvent::PublishDataColumn { column, .. } => beacon.publish_data_column_sidecar( - &column.block_root, - column.column_index, - column.slot, - ), + PeerEvent::SendGossip { topic: GossipTopic::BeaconBlock, ssz, .. } => { + match relayed_gossip.acquire(ssz).buffer() { + Ok((block, _)) => { + let slot = SignedBeaconBlockView::slot(block); + let block_root = block_root(block, spec.is_gloas_at_slot(slot)); + beacon.publish_block_gossip(slot, &block_root); + } + Err(e) => tracing::warn!(?e, "relayed block unavailable to block_gossip"), + } + } + PeerEvent::SendGossip { topic: GossipTopic::DataColumnSidecar(_), ssz, .. } => { + publish_data_column_sidecar(beacon, relayed_gossip.acquire(ssz)) + } + PeerEvent::PublishDataColumn { ssz, .. } => { + publish_data_column_sidecar(beacon, relayed_rpc.acquire(ssz)) + } _ => {} }); let status = beacon.node_status_mut(); @@ -122,3 +136,13 @@ impl ApplicationBoundaryTile { status.el = self.engine.sync_status(); } } + +fn publish_data_column_sidecar(beacon: &mut BeaconApi, sidecar: TRead) { + match sidecar.buffer().map(|(bytes, _)| SidecarIdentity::of(bytes)) { + Ok(Some(column)) => { + beacon.publish_data_column_sidecar(&column.block_root, column.column_index, column.slot) + } + Ok(None) => tracing::warn!("published sidecar fits no layout data_column_sidecar reads"), + Err(e) => tracing::warn!(?e, "published sidecar unavailable to data_column_sidecar"), + } +} diff --git a/crates/application_boundary/tests/tile.rs b/crates/application_boundary/tests/tile.rs index ea7dd03c..ec4d0a14 100644 --- a/crates/application_boundary/tests/tile.rs +++ b/crates/application_boundary/tests/tile.rs @@ -14,9 +14,15 @@ use silver_beacon_api::SlotStatus; use silver_beacon_state_data::{BeaconStateOwner, SpecConfig}; use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ELSyncStatus, EngineFcuReq, EngineReq, EngineResp, - Enr, GossipBlock, GossipDataColumn, GossipMetadata, GossipTopic, Identify, Keypair, MessageId, - P2pStreamId, PayloadValidationStatus, PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, - TCache, TCacheProducer, TCacheRead, ssz_view::STATUS_V2_SIZE, test_util::ShmemDir, + Enr, GossipTopic, Identify, Keypair, MessageId, P2pStreamId, PayloadValidationStatus, + PeerEvent, SilverSpine, StreamProtocol, SyncUpdate, TCache, TCacheProducer, TCacheRead, + TProducer, + column_util::{block_root_from_sidecar, block_root_fulu}, + ssz_view::{ + BEACON_BLOCK_BODY_FIXED, DATA_COLUMN_SIDECAR_GLOAS_MIN, DATA_COLUMN_SIDECAR_MIN, + SIGNED_BEACON_BLOCK_MIN, STATUS_V2_SIZE, + }, + test_util::ShmemDir, }; use silver_config::EngineConfig; use silver_engine_api::test_el::{FCU_VALID_RESULT, FakeEl, write_jwt}; @@ -32,12 +38,20 @@ fn boundary_tile( engine_config: EngineConfig, tcache_names: [&'static str; 3], ) -> ApplicationBoundaryTile { + boundary_tile_with_objects(bind, engine_config, tcache_names).0 +} + +fn boundary_tile_with_objects( + bind: &Bind, + engine_config: EngineConfig, + tcache_names: [&'static str; 3], +) -> (ApplicationBoundaryTile, TProducer, TProducer) { let keypair = Keypair::from_secret(&[1u8; 32]).unwrap(); let local_enr = Enr::empty(keypair.secret_key()).unwrap(); - let gossip_p = TCache::producer(tcache_names[0], 1 << 12); - let rpc_p = TCache::producer(tcache_names[1], 1 << 12); + let gossip_p = TCache::producer(tcache_names[0], 1 << 16); + let rpc_p = TCache::producer(tcache_names[1], 1 << 16); let resp_p = TCache::producer(tcache_names[2], 1 << 12); - ApplicationBoundaryTile::new( + let tile = ApplicationBoundaryTile::new( std::slice::from_ref(bind), 64, Duration::from_secs(75), @@ -50,7 +64,10 @@ fn boundary_tile( gossip_p.cache_ref().random_access("t", true).unwrap(), rpc_p.cache_ref().random_access("t", true).unwrap(), resp_p, - ) + gossip_p.cache_ref().random_access("t_events", true).unwrap(), + rpc_p.cache_ref().random_access("t_events", true).unwrap(), + ); + (tile, gossip_p, rpc_p) } fn identity_client(addr: SocketAddr) -> JoinHandle { @@ -146,49 +163,98 @@ fn block_received(slot: u64, byte: u8, stage: BlockStage) -> BeaconStateEvent { } } -fn publication_payload() -> TCacheRead { - // SSE uses the metadata, so the fixture needs no encoded gossip object. - let payload = b"opaque relay payload"; - let mut producer = TCache::producer("cs_relay_metadata", 1 << 12); - let mut reservation = producer.reserve(payload.len(), false).unwrap(); - reservation.write_all(payload).unwrap(); +fn write_object(producer: &mut TProducer, bytes: &[u8]) -> TCacheRead { + let mut reservation = producer.reserve(bytes.len(), false).unwrap(); + reservation.write_all(bytes).unwrap(); reservation.flush().unwrap(); reservation.read() } -fn block_relay(slot: u64, byte: u8) -> PeerEvent { +/// Synthetic block for field extraction, not consensus validation. The fixed +/// body prefix exercises fork-specific hashing instead of the short-body +/// fallback. Varying the state root gives fixtures distinct block roots. +fn block_bytes(slot: u64, byte: u8) -> Vec { + let mut body = vec![0u8; BEACON_BLOCK_BODY_FIXED]; + for offset in [200, 204, 208, 212, 216, 380, 384, 388, 392] { + body[offset..offset + 4].copy_from_slice(&(BEACON_BLOCK_BODY_FIXED as u32).to_le_bytes()); + } + let mut block = vec![0u8; SIGNED_BEACON_BLOCK_MIN]; + block[0..4].copy_from_slice(&100u32.to_le_bytes()); + block[100..108].copy_from_slice(&slot.to_le_bytes()); + block[148..180].copy_from_slice(&[byte; 32]); + block[180..184].copy_from_slice(&84u32.to_le_bytes()); + block.extend_from_slice(&body); + block +} + +/// Empty Fulu sidecar for field extraction; consensus validation is outside +/// this fixture. +fn fulu_sidecar_bytes(slot: u64, byte: u8, index: u64) -> Vec { + let mut sidecar = vec![0u8; DATA_COLUMN_SIDECAR_MIN]; + sidecar[0..8].copy_from_slice(&index.to_le_bytes()); + for offset in [8, 12, 16] { + sidecar[offset..offset + 4] + .copy_from_slice(&(DATA_COLUMN_SIDECAR_MIN as u32).to_le_bytes()); + } + sidecar[20..28].copy_from_slice(&slot.to_le_bytes()); + sidecar[68..100].copy_from_slice(&[byte; 32]); + sidecar +} + +/// Empty Gloas sidecar for field extraction; consensus validation is outside +/// this fixture. +fn gloas_sidecar_bytes(slot: u64, byte: u8, index: u64) -> Vec { + let mut sidecar = vec![0u8; DATA_COLUMN_SIDECAR_GLOAS_MIN]; + sidecar[0..8].copy_from_slice(&index.to_le_bytes()); + for offset in [8, 12] { + sidecar[offset..offset + 4] + .copy_from_slice(&(DATA_COLUMN_SIDECAR_GLOAS_MIN as u32).to_le_bytes()); + } + sidecar[16..24].copy_from_slice(&slot.to_le_bytes()); + sidecar[24..56].copy_from_slice(&[byte; 32]); + sidecar +} + +fn send_gossip(topic: GossipTopic, byte: u8, ssz: TCacheRead) -> PeerEvent { PeerEvent::SendGossip { originator_stream_id: P2pStreamId::new(0, 0, StreamProtocol::GossipSub, false), - topic: GossipTopic::BeaconBlock, + topic, msg_hash: MessageId { id: [byte; 20] }, recv_ts: Nanos::now(), - protobuf: publication_payload(), - metadata: Some(GossipMetadata::Block(GossipBlock { slot, block_root: [byte; 32] })), + // The boundary does not read protobuf, so no encoded payload is needed. + protobuf: ssz, + ssz, } } -fn column_publications(slot: u64, byte: u8, column_index: u64) -> [PeerEvent; 2] { - let column = GossipDataColumn { slot, block_root: [byte; 32], column_index }; - let topic = GossipTopic::DataColumnSidecar(column_index); - [ - PeerEvent::SendGossip { - originator_stream_id: P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), - topic, - msg_hash: MessageId { id: [byte; 20] }, - recv_ts: Nanos::now(), - protobuf: publication_payload(), - metadata: Some(GossipMetadata::DataColumn(column)), - }, - PeerEvent::PublishDataColumn { - originator: P2pStreamId::new(2, 0, StreamProtocol::DataColumnSidecarsByRange, true), - topic, - ssz: publication_payload(), - column, - }, - ] +fn block_relay(gossip: &mut TProducer, slot: u64, byte: u8) -> (PeerEvent, SseEvent) { + let block = block_bytes(slot, byte); + let event = send_gossip(GossipTopic::BeaconBlock, byte, write_object(gossip, &block)); + (event, SseEvent::block_gossip(slot, &block_root_fulu(&block))) +} + +fn column_relay(gossip: &mut TProducer, slot: u64, byte: u8, index: u64) -> (PeerEvent, SseEvent) { + let sidecar = fulu_sidecar_bytes(slot, byte, index); + let topic = GossipTopic::DataColumnSidecar(index); + let event = send_gossip(topic, byte, write_object(gossip, &sidecar)); + (event, SseEvent::column(slot, &block_root_from_sidecar(&sidecar), index)) } -#[derive(Debug)] +fn column_publication( + rpc: &mut TProducer, + slot: u64, + byte: u8, + index: u64, +) -> (PeerEvent, SseEvent) { + let event = PeerEvent::PublishDataColumn { + originator: P2pStreamId::new(2, 0, StreamProtocol::DataColumnSidecarsByRange, true), + topic: GossipTopic::DataColumnSidecar(index), + ssz: write_object(rpc, &gloas_sidecar_bytes(slot, byte, index)), + }; + (event, SseEvent::column(slot, &[byte; 32], index)) +} + +#[derive(Clone, Debug)] struct SseEvent { name: String, data: Value, @@ -202,17 +268,17 @@ impl SseEvent { } } - fn block_gossip(slot: u64, byte: u8) -> Self { + fn block_gossip(slot: u64, block_root: &[u8; 32]) -> Self { Self { name: "block_gossip".to_owned(), - data: json!({"slot": slot.to_string(), "block": format!("0x{}", hex::encode([byte; 32]))}), + data: json!({"slot": slot.to_string(), "block": format!("0x{}", hex::encode(block_root))}), } } - fn column(slot: u64, byte: u8, index: u64) -> Self { + fn column(slot: u64, block_root: &[u8; 32], index: u64) -> Self { Self { name: "data_column_sidecar".to_owned(), - data: json!({"block_root": format!("0x{}", hex::encode([byte; 32])), "index": index.to_string(), "slot": slot.to_string()}), + data: json!({"block_root": format!("0x{}", hex::encode(block_root)), "index": index.to_string(), "slot": slot.to_string()}), } } @@ -885,11 +951,12 @@ fn an_applied_block_on_the_spine_reaches_an_events_subscriber() { fn subscriptions_select_their_topics_and_preserve_repeated_requests() { let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); - let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ - "cs_subscriptions_gossip", - "cs_subscriptions_rpc", - "cs_subscriptions_resp", - ]); + let (mut tile, mut gossip, mut rpc) = + boundary_tile_with_objects(&Bind::parse("127.0.0.1:0"), no_el(), [ + "cs_subscriptions_gossip", + "cs_subscriptions_rpc", + "cs_subscriptions_resp", + ]); let mut adapter = SpineAdapter::connect_tile(&tile, &mut *spine); let mut inj = SpineAdapter::connect_tile(&Injector, &mut *spine); tile.loop_body(&mut adapter); @@ -902,25 +969,25 @@ fn subscriptions_select_their_topics_and_preserve_repeated_requests() { std::thread::sleep(Duration::from_millis(1)); }; let block = EventsSubscriber::new(addr, "block", 2, &mut crank); - let gossip = EventsSubscriber::new(addr, "block_gossip", 3, &mut crank); + let block_gossip = EventsSubscriber::new(addr, "block_gossip", 3, &mut crank); let column = EventsSubscriber::new(addr, "data_column_sidecar", 5, &mut crank); let mixed = EventsSubscriber::new(addr, "block,block_gossip,data_column_sidecar", 10, &mut crank); - let mut unrelated = block_relay(9, 0xaf); - let PeerEvent::SendGossip { topic, metadata, .. } = &mut unrelated else { unreachable!() }; - *topic = GossipTopic::BeaconAttestation(0); - *metadata = None; - inj.produce(unrelated); + // Topic selection must exclude this request even though its bytes resemble a + // block. + let unrelated = write_object(&mut gossip, &block_bytes(9, 0xaf)); + inj.produce(send_gossip(GossipTopic::BeaconAttestation(0), 0xaf, unrelated)); inj.produce(PeerEvent::EarliestSlot(99)); - let [relay, _] = column_publications(10, 0xac, 3); - let [_, rpc] = column_publications(11, 0xad, 5); + let (relay, relayed) = column_relay(&mut gossip, 10, 0xac, 3); + let (published, publication) = column_publication(&mut rpc, 11, 0xad, 5); + let (block_relayed, relayed_block) = block_relay(&mut gossip, 12, 0xae); inj.produce(relay); inj.produce(relay); - inj.produce(rpc); - inj.produce(block_relay(12, 0xae)); - inj.produce(block_relay(12, 0xae)); + inj.produce(published); + inj.produce(block_relayed); + inj.produce(block_relayed); inj.produce(block_received(13, 0xab, BlockStage::Applied)); // Observe every topic before sending sentinels, so leaks remain inside the @@ -928,52 +995,44 @@ fn subscriptions_select_their_topics_and_preserve_repeated_requests() { mixed.assert_topic_sequences( &[ SseEvent::block(13, 0xab), - SseEvent::block_gossip(12, 0xae), - SseEvent::block_gossip(12, 0xae), - SseEvent::column(10, 0xac, 3), - SseEvent::column(10, 0xac, 3), - SseEvent::column(11, 0xad, 5), + relayed_block.clone(), + relayed_block.clone(), + relayed.clone(), + relayed.clone(), + publication.clone(), ], &mut crank, ); inj.produce(relay); - let [_, sentinel] = column_publications(14, 0xaf, 7); + let (sentinel, sentinel_publication) = column_publication(&mut rpc, 14, 0xaf, 7); inj.produce(sentinel); - inj.produce(block_relay(15, 0xb0)); + let (last_block, last_relayed_block) = block_relay(&mut gossip, 15, 0xb0); + inj.produce(last_block); inj.produce(block_received(16, 0xb1, BlockStage::Applied)); block.assert_topic_sequences( &[SseEvent::block(13, 0xab), SseEvent::block(16, 0xb1)], &mut crank, ); - gossip.assert_topic_sequences( - &[ - SseEvent::block_gossip(12, 0xae), - SseEvent::block_gossip(12, 0xae), - SseEvent::block_gossip(15, 0xb0), - ], + block_gossip.assert_topic_sequences( + &[relayed_block.clone(), relayed_block.clone(), last_relayed_block.clone()], &mut crank, ); column.assert_topic_sequences( &[ - SseEvent::column(10, 0xac, 3), - SseEvent::column(10, 0xac, 3), - SseEvent::column(11, 0xad, 5), - SseEvent::column(10, 0xac, 3), - SseEvent::column(14, 0xaf, 7), + relayed.clone(), + relayed.clone(), + publication.clone(), + relayed.clone(), + sentinel_publication.clone(), ], &mut crank, ); mixed.assert_topic_sequences( - &[ - SseEvent::block(16, 0xb1), - SseEvent::block_gossip(15, 0xb0), - SseEvent::column(10, 0xac, 3), - SseEvent::column(14, 0xaf, 7), - ], + &[SseEvent::block(16, 0xb1), last_relayed_block, relayed, sentinel_publication], &mut crank, ); - for subscriber in [block, gossip, column, mixed] { + for subscriber in [block, block_gossip, column, mixed] { subscriber.client.join().unwrap(); } } @@ -982,11 +1041,12 @@ fn subscriptions_select_their_topics_and_preserve_repeated_requests() { fn a_late_subscriber_receives_only_relay_requests_published_after_it() { let base = ShmemDir::new().unwrap(); let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); - let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), no_el(), [ - "cs_late_gossip", - "cs_late_rpc", - "cs_late_resp", - ]); + let (mut tile, mut gossip, mut rpc) = + boundary_tile_with_objects(&Bind::parse("127.0.0.1:0"), no_el(), [ + "cs_late_gossip", + "cs_late_rpc", + "cs_late_resp", + ]); let mut adapter = SpineAdapter::connect_tile(&tile, &mut *spine); let mut inj = SpineAdapter::connect_tile(&Injector, &mut *spine); tile.loop_body(&mut adapter); @@ -1000,38 +1060,60 @@ fn a_late_subscriber_receives_only_relay_requests_published_after_it() { }; let topics = "block_gossip,data_column_sidecar"; let early = EventsSubscriber::new(addr, topics, 3, &mut crank); - inj.produce(block_relay(20, 0x11)); - let [relay, _] = column_publications(20, 0x11, 3); - let [_, rpc] = column_publications(21, 0x12, 5); + let (block, relayed_block) = block_relay(&mut gossip, 20, 0x11); + let (relay, relayed) = column_relay(&mut gossip, 20, 0x11, 3); + let (published, publication) = column_publication(&mut rpc, 21, 0x12, 5); + inj.produce(block); inj.produce(relay); - inj.produce(rpc); - early.assert_topic_sequences( - &[ - SseEvent::block_gossip(20, 0x11), - SseEvent::column(20, 0x11, 3), - SseEvent::column(21, 0x12, 5), - ], - &mut crank, - ); + inj.produce(published); + early.assert_topic_sequences(&[relayed_block, relayed, publication], &mut crank); early.client.join().unwrap(); let late = EventsSubscriber::new(addr, topics, 3, &mut crank); - inj.produce(block_relay(22, 0x22)); - let [relay, _] = column_publications(22, 0x22, 7); - let [_, rpc] = column_publications(23, 0x23, 9); + let (block, relayed_block) = block_relay(&mut gossip, 22, 0x22); + let (relay, relayed) = column_relay(&mut gossip, 22, 0x22, 7); + let (published, publication) = column_publication(&mut rpc, 23, 0x23, 9); + inj.produce(block); inj.produce(relay); - inj.produce(rpc); - late.assert_topic_sequences( - &[ - SseEvent::block_gossip(22, 0x22), - SseEvent::column(22, 0x22, 7), - SseEvent::column(23, 0x23, 9), - ], - &mut crank, - ); + inj.produce(published); + late.assert_topic_sequences(&[relayed_block, relayed, publication], &mut crank); late.client.join().unwrap(); } +#[test] +fn an_idle_boundary_lets_the_object_rings_evict_its_consumers() { + let base = ShmemDir::new().unwrap(); + let mut spine = Box::new(SilverSpine::new_with_base_dir(base.path(), None)); + let (mut tile, mut gossip, mut rpc) = + boundary_tile_with_objects(&Bind::parse("127.0.0.1:0"), no_el(), [ + "cs_idle_gossip", + "cs_idle_rpc", + "cs_idle_resp", + ]); + let mut adapter = SpineAdapter::connect_tile(&tile, &mut *spine); + tile.loop_body(&mut adapter); + + let chunk = [0u8; 1 << 10]; + let fill = |producer: &mut TProducer| { + let mut written = 0; + while let Some(mut reservation) = producer.reserve(chunk.len(), false) { + reservation.write_all(&chunk).unwrap(); + reservation.flush().unwrap(); + written += 1; + assert!(written < 1_000, "the ring never filled"); + } + }; + fill(&mut gossip); + fill(&mut rpc); + + // Tail advancement requires inactivity beyond the cache's five-second idle + // interval. + std::thread::sleep(Duration::from_millis(5_100)); + tile.loop_body(&mut adapter); + assert!(gossip.reserve(chunk.len(), false).is_some(), "the gossip ring is held by the tile"); + assert!(rpc.reserve(chunk.len(), false).is_some(), "the RPC ring is held by the tile"); +} + #[test] fn gossip_events_are_served_while_the_engine_pool_is_saturated() { let base = ShmemDir::new().unwrap(); @@ -1045,11 +1127,12 @@ fn gossip_events_are_served_while_the_engine_pool_is_saturated() { max_connections: capacity, ..EngineConfig::default() }; - let mut tile = boundary_tile(&Bind::parse("127.0.0.1:0"), config, [ - "cs_gsat_gossip", - "cs_gsat_rpc", - "cs_gsat_resp", - ]); + let (mut tile, mut gossip, mut rpc) = + boundary_tile_with_objects(&Bind::parse("127.0.0.1:0"), config, [ + "cs_gsat_gossip", + "cs_gsat_rpc", + "cs_gsat_resp", + ]); let mut adapter = SpineAdapter::connect_tile(&tile, &mut *spine); let mut inj = SpineAdapter::connect_tile(&Injector, &mut *spine); tile.loop_body(&mut adapter); @@ -1084,19 +1167,13 @@ fn gossip_events_are_served_while_the_engine_pool_is_saturated() { crank(&mut tile, &mut el); }; let client = EventsSubscriber::new(addr, "block_gossip,data_column_sidecar", 3, &mut pump); - inj.produce(block_relay(30, 0x33)); - let [relay, _] = column_publications(31, 0x34, 7); - let [_, rpc] = column_publications(32, 0x35, 9); + let (block, relayed_block) = block_relay(&mut gossip, 30, 0x33); + let (relay, relayed) = column_relay(&mut gossip, 31, 0x34, 7); + let (published, publication) = column_publication(&mut rpc, 32, 0x35, 9); + inj.produce(block); inj.produce(relay); - inj.produce(rpc); - client.assert_topic_sequences( - &[ - SseEvent::block_gossip(30, 0x33), - SseEvent::column(31, 0x34, 7), - SseEvent::column(32, 0x35, 9), - ], - &mut pump, - ); + inj.produce(published); + client.assert_topic_sequences(&[relayed_block, relayed, publication], &mut pump); client.client.join().unwrap(); assert_eq!(pending, capacity, "the additional FCU stays queued while SSE is served"); } diff --git a/crates/beacon_state/tile/src/tile/block.rs b/crates/beacon_state/tile/src/tile/block.rs index 41a26521..da497c14 100644 --- a/crates/beacon_state/tile/src/tile/block.rs +++ b/crates/beacon_state/tile/src/tile/block.rs @@ -6,7 +6,7 @@ use silver_beacon_state_data::{ }; use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, EngineFcuReq, EngineNewPayloadReq, EngineReq, - GossipBlock, SyncNeed, SyncUpdate, TCacheRead, TRandomAccess, hex32, + SyncNeed, SyncUpdate, TCacheRead, TRandomAccess, hex32, ssz_view::{ self, BEACON_BLOCK_BODY_FIXED, BeaconBlockBodyFuluView, BeaconBlockBodyGloasView, SignedBeaconBlockView, @@ -96,7 +96,7 @@ impl BeaconStateTile { source: BlockSource, pre_verified: bool, producers: &mut Producers, - mut send_gossip: impl FnMut(&mut Producers, GossipBlock), + mut send_gossip: impl FnMut(&mut Producers), ) -> Feedback { if let Err(e) = Self::check_block_size(data) { tracing::warn!(?source, "{e}"); @@ -107,10 +107,7 @@ impl BeaconStateTile { let parsed = match self.parse_and_verify_block(data, pre_verified) { Ok(parsed) => { if parsed.relay_eligible { - send_gossip(producers, GossipBlock { - slot: block_slot, - block_root: parsed.block_root, - }); + send_gossip(producers); } parsed } diff --git a/crates/beacon_state/tile/src/tile/gossip.rs b/crates/beacon_state/tile/src/tile/gossip.rs index 4ed5353a..76a88a63 100644 --- a/crates/beacon_state/tile/src/tile/gossip.rs +++ b/crates/beacon_state/tile/src/tile/gossip.rs @@ -6,8 +6,8 @@ use silver_beacon_state_data::{ }; use silver_common::{ ATTESTATION_SUBNETS, BeaconStateEvent, BlockSource, EngineNewPayloadEnvelopeReq, EngineReq, - GossipBlock, GossipMetadata, GossipTopic, LOCAL_GOSSIP_STREAM_ID, MAX_BLOBS_PER_BLOCK, - NewGossipMsg, PeerEvent, SyncNeed, TCacheRead, TRead, hex32, + GossipTopic, LOCAL_GOSSIP_STREAM_ID, MAX_BLOBS_PER_BLOCK, NewGossipMsg, PeerEvent, SyncNeed, + TCacheRead, TRead, hex32, metrics::timed, ssz_view::{ AttestationDataView, AttesterSlashingView, ExecutionPayloadEnvelopeView as Envelope, @@ -428,7 +428,7 @@ impl BeaconStateTile { committed_ptc = true; } } - Self::relay_gossip(&m, None, producers); + Self::relay_gossip(&m, producers); accepted = true; } else { Self::reject_gossip(&m, producers); @@ -1220,14 +1220,9 @@ impl BeaconStateTile { let feedback = match m.topic { GossipTopic::BeaconBlock if !self.sync_target.is_following() => { match self.parse_and_verify_block(data, pre_verified) { - Ok(parsed) if do_relay && parsed.relay_eligible => Self::relay_gossip( - &m, - Some(GossipBlock { - slot: parsed.header.slot, - block_root: parsed.block_root, - }), - producers, - ), + Ok(parsed) if do_relay && parsed.relay_eligible => { + Self::relay_gossip(&m, producers) + } Err(err) if matches!(err.feedback(), Feedback::Reject(_)) => { producers.produce(PeerEvent::P2pGossipInvalidMsg { p2p_peer: m.stream_id.peer(), @@ -1246,9 +1241,9 @@ impl BeaconStateTile { BlockSource::Gossip, pre_verified, producers, - |p, block| { + |p| { if do_relay { - Self::relay_gossip(&m, Some(block), p); + Self::relay_gossip(&m, p); } }, ); @@ -1278,7 +1273,7 @@ impl BeaconStateTile { }), Feedback::Accept(block_root) => { if do_relay { - Self::relay_gossip(&m, None, producers); + Self::relay_gossip(&m, producers); } self.on_accept(block_root, producers); } @@ -1287,7 +1282,7 @@ impl BeaconStateTile { } Feedback::AwaitParentPayload { .. } => { if do_relay { - Self::relay_gossip(&m, None, producers); + Self::relay_gossip(&m, producers); } self.park_block(feedback, BlockSourceMsg::Gossip(m), data, producers); } @@ -1299,14 +1294,14 @@ impl BeaconStateTile { true } - fn relay_gossip(m: &NewGossipMsg, block: Option, producers: &mut Producers) { + fn relay_gossip(m: &NewGossipMsg, producers: &mut Producers) { producers.produce(PeerEvent::SendGossip { originator_stream_id: m.stream_id, topic: m.topic, msg_hash: m.msg_hash, recv_ts: m.recv_ts, protobuf: m.protobuf, - metadata: block.map(GossipMetadata::Block), + ssz: m.ssz, }); } diff --git a/crates/beacon_state/tile/src/tile/orphan_pool.rs b/crates/beacon_state/tile/src/tile/orphan_pool.rs index 93a3878c..8f670c7a 100644 --- a/crates/beacon_state/tile/src/tile/orphan_pool.rs +++ b/crates/beacon_state/tile/src/tile/orphan_pool.rs @@ -233,7 +233,7 @@ impl BeaconStateTile { } let feedback = - self.apply_block(data, read, BlockSource::Rpc, pre_verified, producers, |_, _| {}); + self.apply_block(data, read, BlockSource::Rpc, pre_verified, producers, |_| {}); match feedback { Feedback::Accept(block_root) => self.on_accept(block_root, producers), Feedback::Reject(_) => producers.produce(PeerEvent::RpcMisbehaviour { diff --git a/crates/beacon_state/tile/src/tile/tests.rs b/crates/beacon_state/tile/src/tile/tests.rs index f5e0ccbb..8b911408 100644 --- a/crates/beacon_state/tile/src/tile/tests.rs +++ b/crates/beacon_state/tile/src/tile/tests.rs @@ -10,9 +10,8 @@ use silver_beacon_state_data::{ StateReadView, ValSeed, Withdrawals, }; use silver_common::{ - BlockStage, EngineNewPayloadResp, GossipBlock, GossipMetadata, GossipTopic, - LOCAL_GOSSIP_STREAM_ID, MessageId, P2pStreamId, PeerEvent, StreamProtocol, SyncNeed, TCache, - TCacheProducer, TCacheRead, TProducer, + BlockStage, EngineNewPayloadResp, GossipTopic, LOCAL_GOSSIP_STREAM_ID, MessageId, P2pStreamId, + PeerEvent, StreamProtocol, SyncNeed, TCache, TCacheProducer, TCacheRead, TProducer, column_util::block_root_fulu, ssz_view::{ ATTESTATION_DATA_SIZE, AttestationView, BEACON_BLOCK_BODY_FIXED, BYTES_PER_KZG_COMMITMENT, @@ -510,7 +509,7 @@ fn short_gossip_block_rejected_before_any_field_read() { BlockSource::Gossip, false, &mut adapter.producers, - |_, _| panic!("a malformed block must never be relayed"), + |_| panic!("a malformed block must never be relayed"), ); assert!(matches!(feedback, Feedback::Reject(None)), "len {len}: {feedback:?}"); } @@ -521,14 +520,10 @@ fn short_gossip_block_rejected_before_any_field_read() { bytes[100..108].copy_from_slice(&11u64.to_le_bytes()); bytes[116] = 0xFF; // unknown parent_root let (data, read) = publish_block_bytes(&mut gp, &bytes); - let feedback = tile.apply_block( - &data, - read, - BlockSource::Gossip, - false, - &mut adapter.producers, - |_, _| panic!("an unimportable block must never be relayed"), - ); + let feedback = + tile.apply_block(&data, read, BlockSource::Gossip, false, &mut adapter.producers, |_| { + panic!("an unimportable block must never be relayed") + }); assert!(matches!(feedback, Feedback::RequestParent { .. }), "{feedback:?}"); } @@ -602,7 +597,7 @@ fn a_block_already_in_fork_choice_is_reported_already_known() { let (data, read) = publish_block_bytes(&mut gp, &bytes); let feedback = - tile.apply_block(&data, read, BlockSource::Rpc, false, &mut adapter.producers, |_, _| { + tile.apply_block(&data, read, BlockSource::Rpc, false, &mut adapter.producers, |_| { panic!("a repeat is never relayed") }); assert_eq!(feedback, Feedback::AlreadyKnown(block_root)); @@ -634,13 +629,13 @@ fn a_block_is_applied_once_and_already_known_on_repeat() { // bypasses proposer-signature verification. let (data, read) = publish_block_bytes(&mut gp, &block_ssz); let feedback = - tile.apply_block(&data, read, BlockSource::Gossip, true, &mut adapter.producers, |_, _| {}); + tile.apply_block(&data, read, BlockSource::Gossip, true, &mut adapter.producers, |_| {}); let Feedback::Accept(Some(block_root)) = feedback else { panic!("{feedback:?}") }; assert_eq!(block_stages(&mut sink), [(block_root, BlockStage::Applied)]); let (data, read) = publish_block_bytes(&mut gp, &block_ssz); let feedback = - tile.apply_block(&data, read, BlockSource::Rpc, false, &mut adapter.producers, |_, _| { + tile.apply_block(&data, read, BlockSource::Rpc, false, &mut adapter.producers, |_| { panic!("a repeat is never relayed") }); assert_eq!(feedback, Feedback::AlreadyKnown(block_root)); @@ -679,7 +674,7 @@ fn payload_timestamp_and_blob_count_are_checked_before_relay() { BlockSource::Gossip, true, // pre_verified: the signature is not what these cases are about &mut adapter.producers, - |_, _| relayed = true, + |_| relayed = true, ); // The well-formed case still fails the STF (synthetic parent root), so @@ -759,7 +754,7 @@ fn non_canonical_body_is_rejected_before_relay() { BlockSource::Gossip, true, &mut adapter.producers, - |_, _| relayed = true, + |_| relayed = true, ); assert_eq!(matches!(feedback, Feedback::Reject(None)), want_reject, "{feedback:?}"); @@ -789,7 +784,7 @@ fn block_at_the_finalized_start_slot_is_ignored() { BlockSource::Gossip, true, &mut adapter.producers, - |_, _| {}, + |_| {}, ); assert_eq!(matches!(feedback, Feedback::Ignore), want_ignore, "slot {slot}: {feedback:?}"); @@ -818,19 +813,19 @@ fn block_relay_requires_a_resolved_proposer() { let stamp = slot * SpecConfig::mainnet().seconds_per_slot(); let bytes = fulu_block_with_payload(slot, stamp, 0); let msg = gossip_msg(&mut gp, &bytes, GossipTopic::BeaconBlock); + let msg_seq = msg.ssz.seq(); assert!(tile.handle_gossip(msg.ssz, msg, true, true, &mut adapter.producers)); let mut relays = Vec::new(); adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, metadata, .. } = event { + if let PeerEvent::SendGossip { topic, ssz, .. } = event { assert_eq!(topic, GossipTopic::BeaconBlock); - relays.push(metadata); + relays.push(ssz.seq()); } }); - let expected = GossipBlock { slot, block_root: block_root_fulu(&bytes) }; assert_eq!( relays, - if want_relay { vec![Some(GossipMetadata::Block(expected))] } else { vec![] }, + if want_relay { vec![msg_seq] } else { vec![] }, "{target:?}, slot {slot}" ); } @@ -1474,24 +1469,31 @@ fn attestation_updates_vote_tracker() { assert_eq!(tile.fork_choice.vote_tracker.votes[0].latest_epoch, want_epoch); } -/// Fixture bypass: `protobuf` reuses the SSZ handle; beacon-state forwards it -/// without reading its bytes. +/// Distinct payloads expose relays that substitute the protobuf handle for SSZ. +/// The protobuf bytes are a placeholder; beacon-state forwards them without +/// decoding. fn gossip_msg(producer: &mut TProducer, bytes: &[u8], topic: GossipTopic) -> NewGossipMsg { - let mut r = producer.reserve(bytes.len(), true).expect("reserve"); - r.buffer().unwrap()[..bytes.len()].copy_from_slice(bytes); - r.increment_offset(bytes.len()); - let read = r.read(); - producer.publish_head(); + let ssz = tcache_write(producer, bytes); + let protobuf = tcache_write(producer, b"encoded frame"); NewGossipMsg { stream_id: P2pStreamId::new(0, 0, StreamProtocol::Unset, false), topic, msg_hash: MessageId { id: [0u8; 20] }, recv_ts: Nanos(0), - ssz: read, - protobuf: read, + ssz, + protobuf, } } +fn tcache_write(producer: &mut TProducer, bytes: &[u8]) -> TCacheRead { + let mut r = producer.reserve(bytes.len(), true).expect("reserve"); + r.buffer().unwrap()[..bytes.len()].copy_from_slice(bytes); + r.increment_offset(bytes.len()); + let read = r.read(); + producer.publish_head(); + read +} + fn gossip_att_msg( producer: &mut TProducer, att: &[u8; SINGLE_ATT_SIZE], @@ -3267,8 +3269,7 @@ mod block_relay; fn non_block_relays(adapter: &mut SpineAdapter) -> Vec { let mut topics = Vec::new(); adapter.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, metadata, .. } = event { - assert_eq!(metadata, None, "{topic:?} carries no SSE metadata"); + if let PeerEvent::SendGossip { topic, .. } = event { topics.push(topic); } }); @@ -3281,7 +3282,14 @@ fn assert_non_block_relay(tile: &mut BeaconStateTile, bytes: &[u8], topic: Gossi let (_spine, mut adapter) = spine_adapter(tile); adapter.consume(|_: PeerEvent, _| {}); let msg = gossip_msg(&mut gossip, bytes, topic); + let msg_seq = msg.ssz.seq(); tile.on_gossip(msg, &mut adapter.producers); tile.flush_votes(&mut adapter.producers); - assert_eq!(non_block_relays(&mut adapter), [topic]); + let mut relays = Vec::new(); + adapter.consume(|event: PeerEvent, _| { + if let PeerEvent::SendGossip { topic, ssz, .. } = event { + relays.push((topic, ssz.seq())); + } + }); + assert_eq!(relays, [(topic, msg_seq)], "the relay names the message's own decompressed bytes"); } diff --git a/crates/beacon_state/tile/src/tile/tests/block_relay.rs b/crates/beacon_state/tile/src/tile/tests/block_relay.rs index c18c98a3..39c82b5b 100644 --- a/crates/beacon_state/tile/src/tile/tests/block_relay.rs +++ b/crates/beacon_state/tile/src/tile/tests/block_relay.rs @@ -8,23 +8,28 @@ struct Receipt { source: BlockSource, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct Relayed { + slot: Slot, + block_root: B256, +} + struct Published { events: Vec, - relays: Vec, + relays: Vec, } impl Published { - fn drain(sink: &mut SpineAdapter) -> Self { + fn drain(sink: &mut SpineAdapter, gossip: &mut TRandomAccess) -> Self { let mut events = Vec::new(); sink.consume(|event: BeaconStateEvent, _| events.push(event)); let mut relays = Vec::new(); sink.consume(|event: PeerEvent, _| { - if let PeerEvent::SendGossip { topic, metadata, .. } = event { + if let PeerEvent::SendGossip { topic, ssz, .. } = event { assert_eq!(topic, GossipTopic::BeaconBlock); - let Some(GossipMetadata::Block(block)) = metadata else { - panic!("every block relay carries block metadata") - }; - relays.push(block); + let relayed = gossip.acquire(ssz); + let (bytes, _) = relayed.buffer().expect("relayed bytes readable"); + relays.push(fulu_relayed(bytes)); } }); Self { events, relays } @@ -107,7 +112,7 @@ impl BlockPublications { } fn drain(&mut self) -> Published { - Published::drain(&mut self.sink) + Published::drain(&mut self.sink, &mut self.tile.gossip_consumer) } } @@ -115,14 +120,14 @@ fn stages_of(receipts: &[Receipt]) -> Vec { receipts.iter().map(|r| r.stage).collect() } -fn fulu_gossip_block(bytes: &[u8]) -> GossipBlock { - GossipBlock { slot: SignedBeaconBlockView::slot(bytes), block_root: block_root_fulu(bytes) } +fn fulu_relayed(bytes: &[u8]) -> Relayed { + Relayed { slot: SignedBeaconBlockView::slot(bytes), block_root: block_root_fulu(bytes) } } #[test] -fn gossip_relay_carries_block_metadata_in_either_sync_mode() { +fn a_gossip_relay_names_the_block_in_either_sync_mode() { let (pre_ssz, block_ssz) = sanity_fixture("attestation"); - let expected = fulu_gossip_block(&block_ssz); + let expected = fulu_relayed(&block_ssz); for target in [SyncUpdate::Following, SyncUpdate::SyncingHead { head_slot: 400, head_root: [9; 32] }] { @@ -145,7 +150,7 @@ fn gossip_relay_carries_block_metadata_in_either_sync_mode() { #[test] fn an_rpc_block_is_imported_without_a_gossip_notification() { let (pre_ssz, block_ssz) = sanity_fixture("attestation"); - let expected = fulu_gossip_block(&block_ssz); + let expected = fulu_relayed(&block_ssz); for target in [SyncUpdate::Following, SyncUpdate::SyncingHead { head_slot: 400, head_root: [9; 32] }] { @@ -168,7 +173,7 @@ fn an_rpc_block_is_imported_without_a_gossip_notification() { fn a_blob_block_is_relayed_once_across_staging_and_import() { let (pre_ssz, block_ssz) = sanity_fixture("one_blob"); let mut rig = BlockPublications::new(&pre_ssz, &block_ssz, SyncUpdate::Following); - let expected = fulu_gossip_block(&block_ssz); + let expected = fulu_relayed(&block_ssz); rig.on_gossip(&block_ssz); let published = rig.drain(); @@ -194,7 +199,7 @@ fn a_blob_block_is_relayed_once_across_staging_and_import() { #[test] fn disabling_relay_suppresses_the_gossip_notification() { let (pre_ssz, block_ssz) = sanity_fixture("attestation"); - let expected = fulu_gossip_block(&block_ssz); + let expected = fulu_relayed(&block_ssz); for target in [SyncUpdate::Following, SyncUpdate::SyncingHead { head_slot: 400, head_root: [9; 32] }] { @@ -230,7 +235,7 @@ fn a_parked_block_is_relayed_by_the_retry_that_admits_it() { rig.on_gossip(&first); let released = rig.drain(); - assert_eq!(released.relays, [fulu_gossip_block(&first), fulu_gossip_block(&second)]); + assert_eq!(released.relays, [fulu_relayed(&first), fulu_relayed(&second)]); let of_child = released.receipts().into_iter().filter(|r| r.block_root == child).collect::>(); assert_eq!(stages_of(&of_child), [BlockStage::Applied]); @@ -244,7 +249,7 @@ fn a_relay_request_does_not_imply_successful_import() { rig.on_gossip(&block_ssz); let published = rig.drain(); - let expected = fulu_gossip_block(&block_ssz); + let expected = fulu_relayed(&block_ssz); assert_eq!(published.relays, [expected]); assert!(published.receipts().is_empty()); assert!( diff --git a/crates/bin/src/main.rs b/crates/bin/src/main.rs index 23044dce..b213e255 100644 --- a/crates/bin/src/main.rs +++ b/crates/bin/src/main.rs @@ -74,6 +74,8 @@ fn main() -> Result<(), Box> { ssz_gossip_producer.cache_ref().random_access("dc_persist_ssz_gossip", true)?; let ssz_gossip_consumer_eng = ssz_gossip_producer.cache_ref().random_access("eng_ssz_gossip", true)?; + let ssz_gossip_consumer_api = + ssz_gossip_producer.cache_ref().random_access("api_ssz_gossip", true)?; let outgoing_gossip_producer = TCache::producer("outgoing_gossip", config.outgoing_gossip_tcache_size()); let incoming_rpc_producer = TCache::producer("incoming_rpc", config.incoming_rpc_tcache_size()); @@ -89,6 +91,8 @@ fn main() -> Result<(), Box> { incoming_rpc_producer.cache_ref().random_access("dc_persist_incoming_rpc", true)?; let incoming_rpc_consumer_eng = incoming_rpc_producer.cache_ref().random_access("eng_incoming_rpc", true)?; + let incoming_rpc_consumer_api = + incoming_rpc_producer.cache_ref().random_access("api_incoming_rpc", true)?; let incoming_rpc_consumer_ctl = incoming_rpc_producer.cache_ref().random_access("ctl_incoming_rpc", true)?; let incoming_engine_resp_producer = TCache::producer( @@ -301,6 +305,8 @@ fn main() -> Result<(), Box> { ssz_gossip_consumer_eng, incoming_rpc_consumer_eng, incoming_engine_resp_producer, + ssz_gossip_consumer_api, + incoming_rpc_consumer_api, ); // Spine diff --git a/crates/columns/src/tile.rs b/crates/columns/src/tile.rs index d1a21fee..77e4d313 100644 --- a/crates/columns/src/tile.rs +++ b/crates/columns/src/tile.rs @@ -12,9 +12,9 @@ use flux_profiler::timed; use silver_beacon_state_data::{B256, BeaconStateReader, SLOTS_PER_EPOCH, SpecConfig}; use silver_common::{ BeaconStateEvent, BlockSource, BlockStage, ColumnSource, DataColumnsEvent, DataKind, - EngineResp, GossipDataColumn, GossipMetadata, GossipTopic, IngestionTime, NewGossipMsg, Origin, - P2pStreamId, PeerEvent, RequestId, RpcInbound, RpcSeverity, SilverSpine, SilverSpineProducers, - StreamProtocol, SyncNeed, SyncUpdate, TCacheRead, TProducer, TRandomAccess, TRead, Wheel, + EngineResp, GossipTopic, IngestionTime, NewGossipMsg, Origin, P2pStreamId, PeerEvent, + RequestId, RpcInbound, RpcSeverity, SilverSpine, SilverSpineProducers, StreamProtocol, + SyncNeed, SyncUpdate, TCacheRead, TProducer, TRandomAccess, TRead, Wheel, column_util::{self as util, KzgScratch}, ssz_view::{NUMBER_OF_COLUMNS, SignedBeaconBlockView, StatusView}, ticker::SlotTicker, @@ -521,11 +521,6 @@ impl DataColumnsTile { } fn resolve_validated(&mut self, mut p: PendingKzg, producers: &mut SilverSpineProducers) { - let column = GossipDataColumn { - slot: p.slot, - block_root: p.block_root, - column_index: p.column_index, - }; match mem::replace(&mut p.relay, RelayMeta::None) { RelayMeta::Gossip { topic, msg_hash, recv_ts, protobuf } => { producers.produce(PeerEvent::SendGossip { @@ -534,7 +529,7 @@ impl DataColumnsTile { msg_hash, recv_ts, protobuf, - metadata: Some(GossipMetadata::DataColumn(column)), + ssz: p.sidecar.read, }); } RelayMeta::Rpc { ssz } if self.sync_state.is_synced() => { @@ -542,7 +537,6 @@ impl DataColumnsTile { originator: p.stream_id, topic: GossipTopic::DataColumnSidecar(p.column_index), ssz, - column, }); } _ => {} @@ -782,6 +776,7 @@ mod tests { use silver_common::{ BlockSource, BlockStage, EngineGetBlobsResp, EngineReq, MessageId, Nanos, P2pStreamId, StreamProtocol, TCache, TCacheProducer, TCacheRead, + column_util::SidecarIdentity, ssz_view::{ DATA_COLUMN_SIDECAR_MIN, DataColumnSidecarFuluView, NUMBER_OF_COLUMNS, SIGNED_BEACON_BLOCK_MIN, @@ -900,8 +895,11 @@ mod tests { self.tile.sync_state.update(ssz); } + /// The protobuf placeholder cannot decode as a sidecar, exposing relays + /// that substitute its handle for SSZ. fn gossip_sidecar(&mut self, index: u64, bytes: &[u8]) { let ssz = tcache_write(&mut self.gossip_p, bytes); + let protobuf = tcache_write(&mut self.gossip_p, b"encoded frame"); let recv_ts = Nanos::now(); let mut id = [0u8; 20]; id.copy_from_slice(&bytes[..20]); @@ -911,8 +909,7 @@ mod tests { msg_hash: MessageId { id }, recv_ts, ssz, - // Fixture bypass: networking is absent, so protobuf reuses the SSZ handle. - protobuf: ssz, + protobuf, }; self.tile.gossip_sidecar(index, gossip, &mut self.conn.producers); } @@ -961,11 +958,20 @@ mod tests { SyncNeed::BackfillPrefill(_) => {} }); self.inj.consume(|_: EngineReq, _| out.engine += 1); - self.inj.consume(|event: PeerEvent, _| match event { - PeerEvent::SendGossip { .. } | PeerEvent::PublishDataColumn { .. } => { - out.publications.push(event) - } - _ => {} + let consumers = &mut self.tile.consumers; + self.inj.consume(|event: PeerEvent, _| { + let (source, topic, sidecar) = match event { + PeerEvent::SendGossip { topic, ssz, .. } => { + (ColumnSource::Gossip, topic, consumers.gossip.acquire(ssz)) + } + PeerEvent::PublishDataColumn { topic, ssz, .. } => { + (ColumnSource::Rpc, topic, consumers.rpc.acquire(ssz)) + } + _ => return, + }; + let (bytes, _) = sidecar.buffer().expect("published bytes readable"); + let column = SidecarIdentity::of(bytes).expect("a published sidecar has a layout"); + out.publications.push((source, topic, column)); }); out } @@ -983,29 +989,12 @@ mod tests { available: usize, custody_complete: usize, receipts: Vec, - publications: Vec, + publications: Vec<(ColumnSource, GossipTopic, SidecarIdentity)>, engine: usize, missing: Vec, } impl Produced { - fn column_publications(&self) -> Vec<(ColumnSource, GossipTopic, GossipDataColumn)> { - self.publications - .iter() - .map(|event| match *event { - PeerEvent::SendGossip { - metadata: Some(GossipMetadata::DataColumn(column)), - topic, - .. - } => (ColumnSource::Gossip, topic, column), - PeerEvent::PublishDataColumn { column, topic, .. } => { - (ColumnSource::Rpc, topic, column) - } - _ => panic!("expected a column publication, got {event:?}"), - }) - .collect() - } - fn persisted(&self, root: BlockRoot, index: u64) -> bool { self.receipts.iter().any(|event| { matches!(event, @@ -1172,10 +1161,10 @@ mod tests { let out = rig.drain(); if parent_first { - assert_eq!(out.column_publications(), [( + assert_eq!(out.publications, [( ColumnSource::Gossip, GossipTopic::DataColumnSidecar(index), - GossipDataColumn { slot, block_root, column_index: index } + SidecarIdentity { slot, block_root, column_index: index } )]); } else { assert!(out.publications.is_empty(), "a buffered copy is not relayed"); diff --git a/crates/columns/src/tile/tests/publication.rs b/crates/columns/src/tile/tests/publication.rs index c378b3ce..8bf30cb4 100644 --- a/crates/columns/src/tile/tests/publication.rs +++ b/crates/columns/src/tile/tests/publication.rs @@ -144,7 +144,7 @@ impl Rig { } #[test] -fn column_publications_carry_metadata_for_gossip_and_following_rpc() { +fn column_publications_name_the_sidecar_for_gossip_and_following_rpc() { const SLOT: u64 = 40; let blob = BlockBlob::counting(); let block = block_around(SLOT, &gloas_body(&blob.commitment)); @@ -166,12 +166,8 @@ fn column_publications_carry_metadata_for_gossip_and_following_rpc() { rig.turn(); let out = rig.drain(); if following { - let column = GossipDataColumn { slot: SLOT, block_root, column_index: index }; - assert_eq!(out.column_publications(), [( - source, - GossipTopic::DataColumnSidecar(index), - column - )]); + let column = SidecarIdentity { slot: SLOT, block_root, column_index: index }; + assert_eq!(out.publications, [(source, GossipTopic::DataColumnSidecar(index), column)]); } else { assert!(out.persisted(block_root, index), "syncing still processes the column"); assert!(out.publications.is_empty(), "syncing RPC columns do not request publication"); @@ -198,8 +194,8 @@ fn fulu_column_publication_requires_a_resolved_proposer() { rig.turn(); let out = rig.drain(); if relay_eligible { - let column = GossipDataColumn { slot, block_root, column_index: 3 }; - assert_eq!(out.column_publications(), [( + let column = SidecarIdentity { slot, block_root, column_index: 3 }; + assert_eq!(out.publications, [( ColumnSource::Gossip, GossipTopic::DataColumnSidecar(3), column @@ -227,8 +223,8 @@ fn held_columns_do_not_request_publication_again() { let sidecar = blob.gloas_sidecar(3, SLOT, &block_root); rig.gossip_sidecar(3, &sidecar); rig.turn(); - let column = GossipDataColumn { slot: SLOT, block_root, column_index: 3 }; - assert_eq!(rig.drain().column_publications(), [( + let column = SidecarIdentity { slot: SLOT, block_root, column_index: 3 }; + assert_eq!(rig.drain().publications, [( ColumnSource::Gossip, GossipTopic::DataColumnSidecar(3), column @@ -255,8 +251,8 @@ fn only_columns_with_valid_kzg_proofs_request_publication() { // Column 7 carries column 6's proofs: structural checks pass, KZG fails. rig.gossip_sidecar(7, &blob.gloas_sidecar_with_proofs(7, SLOT, &block_root, 6)); rig.turn(); - let column = GossipDataColumn { slot: SLOT, block_root, column_index: 3 }; - assert_eq!(rig.drain().column_publications(), [( + let column = SidecarIdentity { slot: SLOT, block_root, column_index: 3 }; + assert_eq!(rig.drain().publications, [( ColumnSource::Gossip, GossipTopic::DataColumnSidecar(3), column diff --git a/crates/common/src/column_util.rs b/crates/common/src/column_util.rs index 7fdebebb..6e793216 100644 --- a/crates/common/src/column_util.rs +++ b/crates/common/src/column_util.rs @@ -12,7 +12,7 @@ use silver_common::{ ssz_view::{ BYTES_PER_CELL, BYTES_PER_KZG_COMMITMENT, BYTES_PER_KZG_PROOF, BeaconBlockBodyGloasView, DATA_COLUMN_SIDECAR_MIN, DataColumnSidecarFuluView, DataColumnSidecarGloasView, - MAX_BLOB_COMMITMENTS_PER_BLOCK, NUMBER_OF_COLUMNS, SignedBeaconBlockView, + MAX_BLOB_COMMITMENTS_PER_BLOCK, NUMBER_OF_COLUMNS, SidecarLayout, SignedBeaconBlockView, }, }; @@ -86,6 +86,32 @@ pub fn block_root_from_sidecar(sidecar: &[u8]) -> B256 { ]) } +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct SidecarIdentity { + pub slot: u64, + pub block_root: B256, + pub column_index: u64, +} + +impl SidecarIdentity { + /// Returns `None` if the length and column offset identify neither Fulu nor + /// Gloas. This does not validate the remaining sidecar fields. + pub fn of(sidecar: &[u8]) -> Option { + Some(match SidecarLayout::of(sidecar)? { + SidecarLayout::Fulu => Self { + slot: DataColumnSidecarFuluView::slot(sidecar), + block_root: block_root_from_sidecar(sidecar), + column_index: DataColumnSidecarFuluView::index(sidecar), + }, + SidecarLayout::Gloas => Self { + slot: DataColumnSidecarGloasView::slot(sidecar), + block_root: *DataColumnSidecarGloasView::beacon_block_root(sidecar), + column_index: DataColumnSidecarGloasView::index(sidecar), + }, + }) + } +} + fn check_sidecar_shape(column: &[u8], commits: &[u8], proofs: &[u8], max_blobs: usize) -> bool { if !column.len().is_multiple_of(BYTES_PER_CELL) || !commits.len().is_multiple_of(BYTES_PER_KZG_COMMITMENT) || diff --git a/crates/common/src/lib.rs b/crates/common/src/lib.rs index efeb73a3..f70c8eb5 100644 --- a/crates/common/src/lib.rs +++ b/crates/common/src/lib.rs @@ -23,9 +23,8 @@ pub use crate::{ EngineGetPayloadBodiesByRangeReq, EngineGetPayloadBodiesResp, EngineGetPayloadReq, EngineGetPayloadResp, EngineHealthEvent, EngineNewPayloadEnvelopeReq, EngineNewPayloadReq, EngineNewPayloadResp, EnginePreparePayloadReq, EngineReq, EngineResp, Error as TCacheError, - GossipBlock, GossipDataColumn, GossipMetadata, GossipMsgIn, GossipMsgOut, IpBytes, - LOCAL_GOSSIP_STREAM_ID, LocalAttestationFailure, LocalAttestationResult, - MAX_BLOBS_PER_BLOCK, MAX_PAYLOAD_BODIES_PER_REQ, MULTISTREAM_V1, + GossipMsgIn, GossipMsgOut, IpBytes, LOCAL_GOSSIP_STREAM_ID, LocalAttestationFailure, + LocalAttestationResult, MAX_BLOBS_PER_BLOCK, MAX_PAYLOAD_BODIES_PER_REQ, MULTISTREAM_V1, MultiProducer as TMultiProducer, NewGossipMsg, P2pConnectionStats, P2pSend, P2pStreamId, PREFILL_SLOTS, PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, PeerStatus, PeerTopicScores, Prefill, Producer as TProducer, REJECT_RESPONSE, diff --git a/crates/common/src/spine.rs b/crates/common/src/spine.rs index fc407147..543084ab 100644 --- a/crates/common/src/spine.rs +++ b/crates/common/src/spine.rs @@ -7,13 +7,13 @@ pub use messages::{ EngineGetBlobsResp, EngineGetPayloadBodiesByHashReq, EngineGetPayloadBodiesByRangeReq, EngineGetPayloadBodiesResp, EngineGetPayloadReq, EngineGetPayloadResp, EngineHealthEvent, EngineNewPayloadEnvelopeReq, EngineNewPayloadReq, EngineNewPayloadResp, - EnginePreparePayloadReq, EngineReq, EngineResp, GossipBlock, GossipDataColumn, GossipMetadata, - GossipMsgIn, GossipMsgOut, IpBytes, LocalAttestationFailure, LocalAttestationResult, - MAX_BLOBS_PER_BLOCK, MAX_PAYLOAD_BODIES_PER_REQ, NewGossipMsg, P2pConnectionStats, P2pSend, - PREFILL_SLOTS, PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, - PeerStatus, PeerTopicScores, Prefill, ReplayBlock, RpcInbound, RpcOutbound, RpcRequest, - RpcRequestInbound, RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, - RpcSeverity, SyncNeed, SyncUpdate, SyncingStrategy, WithdrawalInline, + EnginePreparePayloadReq, EngineReq, EngineResp, GossipMsgIn, GossipMsgOut, IpBytes, + LocalAttestationFailure, LocalAttestationResult, MAX_BLOBS_PER_BLOCK, + MAX_PAYLOAD_BODIES_PER_REQ, NewGossipMsg, P2pConnectionStats, P2pSend, PREFILL_SLOTS, + PayloadValidationStatus, PeerControl, PeerEvent, PeerScores, PeerStats, PeerStatus, + PeerTopicScores, Prefill, ReplayBlock, RpcInbound, RpcOutbound, RpcRequest, RpcRequestInbound, + RpcRequestOutbound, RpcResponse, RpcResponseInbound, RpcResponseOutbound, RpcSeverity, + SyncNeed, SyncUpdate, SyncingStrategy, WithdrawalInline, }; pub use stream_id::{LOCAL_GOSSIP_STREAM_ID, P2pStreamId}; pub use stream_protocol::{ diff --git a/crates/common/src/spine/messages.rs b/crates/common/src/spine/messages.rs index 40a618f8..f5498010 100644 --- a/crates/common/src/spine/messages.rs +++ b/crates/common/src/spine/messages.rs @@ -5,7 +5,7 @@ use std::{ }; use flux::timing::Nanos; -use silver_beacon_state_data::{B256, SLOTS_PER_EPOCH}; +use silver_beacon_state_data::SLOTS_PER_EPOCH; use crate::{ DataKind, Enr, GossipTopic, Identify, MessageId, Origin, P2pStreamId, PeerId, StreamProtocol, @@ -318,28 +318,6 @@ impl RpcOutbound { } } -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -#[repr(C)] -pub struct GossipBlock { - pub slot: u64, - pub block_root: B256, -} - -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -#[repr(C)] -pub struct GossipDataColumn { - pub slot: u64, - pub block_root: B256, - pub column_index: u64, -} - -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -#[repr(C, u8)] -pub enum GossipMetadata { - Block(GossipBlock), - DataColumn(GossipDataColumn), -} - #[derive(Clone, Copy, Debug)] #[repr(C, u8)] #[allow(clippy::large_enum_variant)] @@ -475,7 +453,6 @@ pub enum PeerEvent { originator: P2pStreamId, topic: GossipTopic, ssz: TCacheRead, - column: GossipDataColumn, }, /// Emitted in order to trigger sending of a gossip message. /// Peer manager will generate select peers to send to. @@ -485,7 +462,8 @@ pub enum PeerEvent { msg_hash: MessageId, recv_ts: Nanos, protobuf: TCacheRead, - metadata: Option, + /// Handle to the relayed object's decompressed SSZ bytes. + ssz: TCacheRead, }, /// Misbehaviour observed on the RPC (req/resp) sub-protocol. The peer /// manager translates `severity` into a P5 application-score delta; diff --git a/crates/control/src/tile.rs b/crates/control/src/tile.rs index 38a1bcbf..b531d033 100644 --- a/crates/control/src/tile.rs +++ b/crates/control/src/tile.rs @@ -2,10 +2,9 @@ use std::time::{Duration, Instant}; use flux::{spine::SpineAdapter, tile::Tile}; use silver_common::{ - BeaconStateEvent, GossipMetadata, GossipTopic, Nanos, P2pSend, PeerControl, PeerEvent, - PeerStats, RpcInbound, RpcOutbound, RpcRequest, RpcRequestOutbound, RpcResponse, - RpcResponseInbound, SilverSpine, SilverSpineProducers, SyncNeed, SyncUpdate, TMultiProducer, - TRandomAccess, + BeaconStateEvent, GossipTopic, Nanos, P2pSend, PeerControl, PeerEvent, PeerStats, RpcInbound, + RpcOutbound, RpcRequest, RpcRequestOutbound, RpcResponse, RpcResponseInbound, SilverSpine, + SilverSpineProducers, SyncNeed, SyncUpdate, TMultiProducer, TRandomAccess, ssz_view::{METADATA_SIZE, STATUS_V2_SIZE, StatusView}, }; use silver_gossip::{GossipHandler, GossipHandlerEvent}; @@ -174,7 +173,7 @@ impl Tile for Controller { } adapter.consume(|event: PeerEvent, producers| { - if let PeerEvent::PublishDataColumn { originator, topic, ssz, column } = event { + if let PeerEvent::PublishDataColumn { originator, topic, ssz } = event { let read = self.rpc_ssz_consumer.acquire(ssz); match read.buffer() { Ok((bytes, _)) => { @@ -188,7 +187,7 @@ impl Tile for Controller { msg_hash, recv_ts: Nanos::now(), protobuf, - metadata: Some(GossipMetadata::DataColumn(column)), + ssz, }, now, &mut |evt| { @@ -215,7 +214,7 @@ impl Tile for Controller { msg_hash, recv_ts: _, protobuf, - metadata: _, + ssz: _, } = &event { self.gossip_handler.mcache_insert(*msg_hash, *topic, *protobuf); diff --git a/crates/control/src/tile/tests.rs b/crates/control/src/tile/tests.rs index 641637b2..e7bf17b0 100644 --- a/crates/control/src/tile/tests.rs +++ b/crates/control/src/tile/tests.rs @@ -2,9 +2,8 @@ use std::{io::Write, sync::Arc}; use silver_chain_spec::SpecConfig; use silver_common::{ - GossipBlock, GossipDataColumn, GossipMsgIn, GossipMsgOut, IpBytes, Keypair, MessageId, - P2pStreamId, PeerId, StreamProtocol, TCache, TCacheProducer, TCacheRead, TProducer, - test_util::ShmemDir, + GossipMsgIn, GossipMsgOut, IpBytes, Keypair, MessageId, P2pStreamId, PeerId, StreamProtocol, + TCache, TCacheProducer, TCacheRead, TProducer, test_util::ShmemDir, }; use silver_peer::SyncingConfig; @@ -126,33 +125,21 @@ fn write_bytes(producer: &mut TProducer, bytes: &[u8]) -> TCacheRead { } #[test] -fn relay_metadata_preserves_routing_and_iwant_service() { - // Forwarding and IWANT service treat the payload as opaque bytes. +fn relay_requests_preserve_routing_and_iwant_service() { + // Distinct payloads expose confusion between the encoded and decompressed + // handles. let bytes = b"relay payload"; let hash = MessageId { id: [0xCD; 20] }; - for (topic, metadata) in [ - (GossipTopic::BeaconBlock, None), - ( - GossipTopic::BeaconBlock, - Some(GossipMetadata::Block(GossipBlock { slot: 37, block_root: [0xAB; 32] })), - ), - ( - GossipTopic::DataColumnSidecar(5), - Some(GossipMetadata::DataColumn(GossipDataColumn { - slot: 38, - block_root: [0xCD; 32], - column_index: 5, - })), - ), - ] { + for topic in [GossipTopic::BeaconBlock, GossipTopic::DataColumnSidecar(5)] { let mut capture = GossipPublications::new(topic, bytes); + let ssz = write_bytes(&mut capture.rpc, b"decompressed object"); capture.observer.produce(PeerEvent::SendGossip { originator_stream_id: P2pStreamId::new(1, 0, StreamProtocol::GossipSub, true), topic, msg_hash: hash, recv_ts: Nanos::now(), protobuf: capture.payload, - metadata, + ssz, }); capture.crank(); assert_eq!(capture.sent(), [(2, bytes.to_vec())], "the sender is excluded"); @@ -169,7 +156,6 @@ fn relay_metadata_preserves_routing_and_iwant_service() { #[test] fn column_publication_encodes_and_routes_without_another_spine_request() { let topic = GossipTopic::DataColumnSidecar(5); - let column = GossipDataColumn { slot: 38, block_root: [0xCD; 32], column_index: 5 }; // Transport decoding checks payload size; consensus validation is outside this // fixture. let bytes = vec![0x42; topic.min_uncompressed_size()]; @@ -180,7 +166,6 @@ fn column_publication_encodes_and_routes_without_another_spine_request() { originator: P2pStreamId::new(1, 0, StreamProtocol::DataColumnSidecarsByRoot, true), topic, ssz, - column, }); capture.crank(); let sent = capture.sent(); diff --git a/crates/peer/src/manager/mod.rs b/crates/peer/src/manager/mod.rs index 9ab5f959..2a91bbf2 100644 --- a/crates/peer/src/manager/mod.rs +++ b/crates/peer/src/manager/mod.rs @@ -424,7 +424,7 @@ impl PeerManager { msg_hash, recv_ts: _, protobuf, - metadata: _, + ssz: _, } => { // TODO recv_ts elapsed metric self.on_send_gossip(originator_stream_id.peer(), msg_hash, topic, protobuf, emit); diff --git a/crates/peer/src/manager/promises.rs b/crates/peer/src/manager/promises.rs index 7630d322..1d45456f 100644 --- a/crates/peer/src/manager/promises.rs +++ b/crates/peer/src/manager/promises.rs @@ -342,7 +342,7 @@ impl PeerManager { mod tests { use std::time::Duration; - use silver_common::{GossipBlock, GossipMetadata, PeerEvent, TCacheProducer}; + use silver_common::{PeerEvent, TCacheProducer}; use silver_config::ScoreParams; use super::*; @@ -1026,10 +1026,7 @@ mod tests { msg_hash: hash, recv_ts: silver_common::Nanos::now(), protobuf: mk_tcache_read(), - metadata: Some(GossipMetadata::Block(GossipBlock { - slot: 37, - block_root: [0xAB; 32], - })), + ssz: mk_tcache_read(), }, now, &mut |c| cap.0.push(c), @@ -1083,7 +1080,7 @@ mod tests { msg_hash: silver_common::MessageId { id: [0xCD; 20] }, recv_ts: silver_common::Nanos::now(), protobuf: mk_tcache_read(), - metadata: None, + ssz: mk_tcache_read(), }, now, &mut |event| cap.0.push(event), diff --git a/docs/adr/0004-sync-materialized-api.md b/docs/adr/0004-sync-materialized-api.md index b0aaf9e3..9ee38058 100644 --- a/docs/adr/0004-sync-materialized-api.md +++ b/docs/adr/0004-sync-materialized-api.md @@ -86,9 +86,9 @@ delivery to peers. This deliberately narrows the Beacon API's validation contract: RPC block imports remain silent because they do not request relay. The topic follows silver's relay policy, keeping its promise tied to gossip publication without duplicating an observation on the spine. The boundary -selects block metadata on the existing `SendGossip` request, without reading -its payload; producers own topic consistency. The `block` topic still follows -`Applied` import receipts. These queues establish no shared ordering. +selects `SendGossip` requests on the block topic. It reads the slot and +computes the block root from the relayed block's SSZ bytes. The `block` topic +still follows `Applied` import receipts. These queues establish no shared ordering. Repeated requests for one root are not deduplicated, and late subscribers receive no replay. Subscribers to both topics can reach the existing send cap sooner. @@ -97,14 +97,27 @@ Amended 2026-09-10: `/eth/v1/events` also serves `data_column_sidecar` for column publication requests following silver's gossip checks, including KZG. This deliberately narrows the Beacon API's validation contract: validation without a publication request produces no event. The boundary selects -column metadata on `SendGossip` or `PublishDataColumn`, without reading -payload bytes or filtering by custody. Producers own topic consistency. -Control's converted request stays off the spine, avoiding a second -notification. These events acknowledge requests, including RPC requests -that can fail before encoding; they do not guarantee delivery to peers. +`SendGossip` requests on column topics and every `PublishDataColumn`. It +derives the slot, block root and column index from Fulu or Gloas sidecar +bytes, without filtering by custody. Control's converted request stays off the +spine, avoiding a second notification. These events acknowledge requests, +including RPC requests that can fail before encoding; they do not guarantee +delivery to peers. Buffered copies, RPC columns processed while syncing, held copies, and EL reconstruction remain silent under the existing publication policy. `Persist` and `Available` retain their existing meaning and selection. Repeated requests are not deduplicated, and late subscribers receive no replay. The `beacon_events` and `peer_events` queues establish no shared ordering. Additional subscriptions can reach the existing send cap sooner. + +Amended 2026-09-14: publication requests carry handles to decompressed SSZ +bytes so API consumers can derive event fields without adding API-specific +metadata to those requests. The boundary reads `SendGossip` objects from +the gossip cache and `PublishDataColumn` objects from the RPC cache. +It computes each block root, including the body hash, even when no clients +subscribe to `block_gossip`. For Fulu sidecars, it computes the block root +from the five header fields. Gloas sidecars contain the block root directly. +If the cache has overwritten an object's bytes, the boundary logs a warning +and emits no event for that request. It also skips sidecars whose length +and column offset identify neither supported layout. These failures do not +cancel the publication request.