Discovery options for phase 0

[jannikluhn]

Nodes need a way to find peers. In phase 0, this task is less challenging than in later phases as there are no shards and nodes can choose peers randomly from the whole set of nodes in the network [1]. For this reason, we should be able to just pick an existing protocol and the main consideration is implementation overhead. Here are some options that we could consider:

Bootstrap list

• implemented in libp2p, probably trivial to implement by ourselves
• maybe fine for a testnet, but too centralized as the only mechanism for a real network.

libp2p MulticastDNS

• not an option as it only works in a local network.

libp2p KadDHT

• Implementation status:
  • Go
  • Rust
  • JavaScript
  • Python
  • Nim
  • Java
  • Daemon

devp2p discovery

• Implementation status:
  • Go
  • Rust
  • JavaScript
  • Python
  • Nim
  • Java
• Note that it's also possible to implement devp2p's discovery protocol on top of libp2p primitives.

Do you have other suggestions? What are your preferences?

[1] One caveat has been pointed out here: Signatures still need to be aggregated at the shard level. However, since the bandwidth required to transmit signatures is much less than to transmit shard blocks, I think we can just gossip signatures to everyone, even to nodes that don't participate in the aggregation. This is something we should discuss though.

[raulk]

Thanks for putting this candidate list together, @jannikluhn. It seems fair to unblock the discovery part by picking an existing off-the-shelf solution for the time being.

Regarding libp2p's Kad DHT, there are some algorithmical differences with Ethereum's implementation, particularly the way that buckets behave and the lack of PING/PONGs. This has worked well for IPFS so far, but I'm convinced other projects will want different behaviours, so we can introduce configuration options to customise the DHT mechanics.

Some further reading:

• WIP Kad DHT specification \o/
• Two items on the libp2p roadmap: Service discovery and DHT 2.0.

[raulk]

One particular approach I wanted to call out specifically with libp2p Kad DHT is using "provider records". Each shard has a CID (content identifier) and nodes participating in that shard publish a provider record for themselves in the DHT. Nodes looking for peers in a shard, search for providers of that CID.