Interactive Distributed Inference Simulator — chakra.guruswami.ai

[guruswami-ai]

We've been building an interactive simulator to teach distributed inference concepts on Apple Silicon clusters using MLX. It uses real benchmark data from a 5-node M3 Ultra cluster (512GB each, TB5 mesh) to visualise how model configuration choices affect TPS, TTFT, memory, and topology.

Try it: https://chakra.guruswami.ai

What it does

• 5 models: Qwen 32B, Llama 405B, Mixtral 8x7B, DeepSeek V3, Kimi K2.5
• Real benchmark data (284 rows across topologies, quants, context lengths)
• Interactive topology selection: EP, TP2, TP4, PP2-PP5 with real error modes (SHARD_ERR, GPU_TIMEOUT, OOM, RDMA instability)
• KV cache calculated from actual memory measurements (not theoretical)
• Guided tutorial with a voiced character (Guruswami) who walks you through the basics
• 185 original music tracks across 7 genres — each model has its own sonic identity with lyrics that teach architecture concepts. Synthwave for Qwen, hip-hop for Llama, EDM for Mixtral, Chinese-American fusion for DeepSeek, space ambient for Kimi. Yes, there are songs called "Thirty-Seven Billion Awake", "The Topology Trap", and "Q4 On The Mic". We believe this is the world's largest collection of LLM-specific music.

Caveats — it's a work in progress

• Still has bugs and state-machine quirks in the tutorial flow
• Currently optimised for large displays (Apple Studio Display resolution) — mobile responsive is planned
• The character voice sometimes gets stuck or repeats
• Some visual animations are being refined

What it teaches (without ever saying "distributed inference")

• Why some models need tensor parallel and others don't
• Why attention heads must divide evenly across nodes
• The difference between TP and EP for throughput
• Why Pipeline Parallel triggers Metal GPU timeouts on 405B
• How quantisation trades quality for speed and memory
• Why context length is the hidden memory killer

Real benchmarks behind it

All the TPS, TTFT, and memory numbers come from actual MLX distributed runs on our cluster. The scaling factors, error conditions, and topology constraints are measured, not estimated. The simulator lets people explore these tradeoffs interactively without needing the hardware.

Next steps

• Mobile responsive layout
• Game mode with structured challenges (scored on finding optimal configurations with minimal hints)
• More detailed Codex with technical deep-dives

If you have ideas on how to improve the teaching or spot things that are wrong, we'd love to hear it. The goal is to make distributed inference on Apple Silicon accessible to people who don't have a 5-node cluster to experiment with.

Built with React, XState, and real data from mlx-lm benchmarks.

[guruswami-ai]

A bit of context on how this happened — it started because I wanted a cool real-time visualisation of distributed inference running on the cluster, similar to what the Exo team had built for their setup. I thought it would be a quick weekend project.

Several months later, there are 185 music tracks, a talking guru character with 1,100 voice clips, and an interactive tutorial that teaches you distributed inference by deliberately crashing the cluster.

The simulator runs entirely client-side with static benchmark data, but there's also a live mode that connects to the real cluster via MQTT and shows real-time TPS, memory, GPU utilisation, and TB5 mesh status. If anyone running MLX distributed inference wants a real-time dashboard/visualisation for their cluster, I'm happy to make that available as a separate tool. It works with any number of Apple Silicon nodes.