Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 17 additions & 8 deletions docs/running-multiple-mpc-nodes-on-one-host.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ In addition to the standard [hardware requirements](./running-an-mpc-node-in-tdx
- **2× the single-node hardware** ([single-node minimums](./running-an-mpc-node-in-tdx-external-guide.md#hardware-requirements)
apply per CVM): ≥ 128 GB memory, ≥ 16 cores, ≥ 1 TB SSD/NVMe disk.
- **Firewall opened on both IPs** for the [required ports](./running-an-mpc-node-in-tdx-external-guide.md#required-ports)
(80, 8080, 24567).
(80, 8080, 24567, 8079).
- *(Optional)* a DNS A record per IP. Recommended if you want to
re-IP later without rotating operator-side configuration.

Expand All @@ -60,7 +60,7 @@ on the public internet on the canonical port `:24567`.
| ^ ^ |
| | | |
| hostfwd :80, :8080, hostfwd :80, :8080, |
| :24567 -> CVM_M :24567 -> CVM_T |
| :24567, :8079 -> CVM_M :24567, :8079 -> CVM_T |
| | | |
| +-----------+ +-----------+ |
| | qemu CVM | | qemu CVM | |
Expand All @@ -84,7 +84,7 @@ Key properties:
- One `dstack-vmm` process and `vmm.toml` — same as a single-node
deployment.
- Each CVM specifies `host_address` per port mapping at creation
time, so its `:24567` / `:80` / `:8080` forwards land on a
time, so its `:24567` / `:80` / `:8080` / `:8079` forwards land on a
distinct host IP.
- The **SGX sealing-key-provider, PCCS endpoints, and `dstack-vmm`
itself are shared by every CVM on this host** — they're host-level
Expand Down Expand Up @@ -119,8 +119,8 @@ this guide:

| Chain | Host IP | Ports (host & CVM) |
|---|---|---|
| mainnet | `203.0.113.10` (`IP_M`) | `:24567`, `:80`, `:8080` |
| testnet | `203.0.113.11` (`IP_T`) | `:24567`, `:80`, `:8080` |
| mainnet | `203.0.113.10` (`IP_M`) | `:24567`, `:80`, `:8080`, `:8079` |
| testnet | `203.0.113.11` (`IP_T`) | `:24567`, `:80`, `:8080`, `:8079` |

Both CVMs use the same canonical ports — `host_address` is what
disambiguates them on the host side.
Expand All @@ -132,9 +132,18 @@ walkthrough. In the same dstack-vmm web UI (`http://127.0.0.1:10000`),
create the second CVM following
[Configuring and starting the MPC binary in a CVM](./running-an-mpc-node-in-tdx-external-guide.md#configuring-and-starting-the-mpc-binary-in-a-cvm).

For **each port forward** (`:80`, `:8080`, `:24567`), set the **host
address** to that CVM's assigned IP. The dstack-vmm web UI exposes
this field next to host port / VM port at CVM-creation time.
For **every public port forward** (`:80`, `:8080`, `:24567`, **and the
migration port `:8079`**), set the **host address** to that CVM's
assigned IP — never `0.0.0.0`. The dstack-vmm web UI exposes this field
next to host port / VM port at CVM-creation time.
Comment on lines +135 to +138
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 45eaa88 — added :8079 to the Step 1 Plan-IPs table (and the firewall bullet, architecture diagram, and key-properties bullet) so the whole doc is consistent.


> ⚠️ **Do not leave any public port on `0.0.0.0`.** A wildcard
> `0.0.0.0:<port>` bind blocks *every* other CVM from binding that port
> on its own IP, so whichever CVM boots first wins the port and the
> others fail to start with
> `qemu: Could not set up host forwarding rule 'tcp:<ip>:<port>-:<port>'`
> and crash-loop. The migration port `:8079` is the easy one to miss —
> its single-node default is `0.0.0.0:8079`.

### Step 3 — Per-CVM `user-config.toml`

Expand Down
Loading