Cluster design

[magick93]

NOTE: The following is WIP.

Kubernetes Cluster Design

Component	Recommended Kubernetes Resource Type	Notes	Suggested Node Grouping
blockprint	Deployment (Pod)	Core service, stateless	Group with blockprint-bg, blockgauge
blockprint-bg	Deployment (Pod)	Background service, stateless	Group with blockprint, blockgauge
blockgauge	Deployment (Pod)	Monitoring/metrics service, stateless	Group with blockprint, blockprint-bg
caddy	DaemonSet	Reverse proxy/load balancer, needs to run on all nodes	Run on all nodes
blockprint-ui	Deployment (Pod)	Frontend web interface, stateless	Can run on any node
lighthouse (main)	StatefulSet	Primary consensus client, stateful	Group with EL client
Execution Layer (EL)	StatefulSet	Execution client (Nethermind/Geth), stateful	Group with lighthouse main
eleel	Deployment (Pod)	Middleware connecting CL and EL, stateless	Group with lighthouse and EL
Additional CL clients	StatefulSet (per client)	Secondary consensus clients, stateful	Run on worker nodes
blockdreamer	?	Produce blocks every slot	Run on worker nodes

Node Grouping Recommendations

1. Core Services Group:

   • blockprint
   • blockprint-bg
   • blockgauge
   • blockprint-ui
   • caddy (as DaemonSet)

2. Blockchain Clients Group:

   • lighthouse (main instance)
   • Execution Layer client
   • eleel

3. Worker Nodes:

   • Additional consensus clients (lighthouse instances)
   • (Optional) Additional execution clients if needed
   • blockdreamer

Storage Considerations

• StatefulSets for blockchain clients (lighthouse, EL) should use PersistentVolumes
• Consider storage class with automatic expansion for blockchain data
• Monitor disk usage closely (previous issues with Nethermind running out of disk)

Network Considerations

• Ensure low latency between:
  • blockprint services and main lighthouse instance
  • lighthouse and its paired execution client
• Ingress via caddy for web interfaces

[michaelsproul]

Added blockdreamer as a service that needs to run on the worker nodes.

https://github.com/blockprint-collective/blockdreamer