From 654535ebe0333fc551225144871ca2dc464a0467 Mon Sep 17 00:00:00 2001 From: "mintlify[bot]" <109931778+mintlify[bot]@users.noreply.github.com> Date: Mon, 17 Aug 2026 13:07:33 +0000 Subject: [PATCH 1/2] docs: clarify WebSocket session duration and multi-region field interaction --- deployments/multi-region-deployment.mdx | 27 +++++++++++++++++++++++++ endpoints/websockets.mdx | 11 ++++++++++ 2 files changed, 38 insertions(+) diff --git a/deployments/multi-region-deployment.mdx b/deployments/multi-region-deployment.mdx index 5ed4d31b..798628f3 100644 --- a/deployments/multi-region-deployment.mdx +++ b/deployments/multi-region-deployment.mdx @@ -95,6 +95,33 @@ compute = ["HOPPER_H100", "HOPPER_H200", "AMPERE_A100_80GB"] The set of eligible regions is the intersection of both constraints: any region with several acceptable GPU types is the widest pool; one region with a single GPU type is the narrowest. +### How Region, Provider, and Compute Interact + +Each field constrains placement independently. Pinning any of them shrinks the pool; omitting a field lets the platform choose. + +- `region = "global"` widens the region pool but does not change `compute` or `provider`. A pinned GPU type or provider is still enforced. +- A single `compute` value pins the GPU type. A `compute` list lets the platform pick the first available type from the list in order. +- Pinning `provider` restricts placement to regions on that provider. To let the platform pick the provider, omit `provider` entirely. + +For example, this configuration runs only on `nebius` regions that have `HOPPER_H100` available, and will not fall back to AWS: + +```toml +[cerebrium.hardware] +region = "global" +compute = "HOPPER_H100" +provider = "nebius" +``` + +To allow fallback across GPU types and providers, use a `compute` list and omit `provider`: + +```toml +[cerebrium.hardware] +region = "global" +compute = ["HOPPER_H100", "HOPPER_H200", "AMPERE_A100_80GB"] +``` + +Changes to these fields take effect on the next `cerebrium deploy` and update the existing app; you do not need to delete and recreate it. + ## Endpoints Apps keep a single endpoint no matter how many regions they run in: diff --git a/endpoints/websockets.mdx b/endpoints/websockets.mdx index 0ea371d3..f48af84f 100644 --- a/endpoints/websockets.mdx +++ b/endpoints/websockets.mdx @@ -29,6 +29,17 @@ Fields: - WebSocket URL: Requests must use a `wss://` URL. The client must support secure WebSocket connections. +## Session Duration + +A WebSocket connection is treated as a single long-running request and is bounded by `response_grace_period` in `cerebrium.toml`. The default is 900 seconds (15 minutes). When the grace period elapses, Cerebrium terminates the connection with a GatewayTimeout error. Raise the value to match the longest session your app needs to support: + +```toml +[cerebrium.scaling] +response_grace_period = 3600 # 1 hour, in seconds +``` + +The same value governs how long an instance drains in-flight WebSocket sessions during a shutdown or migration. Custom runtimes must handle `SIGTERM` to close open sockets gracefully before the grace period expires; see [Graceful Termination](/scaling/graceful-termination). + ## Making a request Test the WebSocket endpoint using websocat, a command-line WebSocket client: From 8bbcf556e59b3317075902ce9b325019e468f2f0 Mon Sep 17 00:00:00 2001 From: "mintlify[bot]" <109931778+mintlify[bot]@users.noreply.github.com> Date: Mon, 17 Aug 2026 13:08:35 +0000 Subject: [PATCH 2/2] docs: add SEO descriptions to WebSocket and multi-region pages --- deployments/multi-region-deployment.mdx | 2 +- endpoints/websockets.mdx | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/deployments/multi-region-deployment.mdx b/deployments/multi-region-deployment.mdx index 798628f3..94f9cf99 100644 --- a/deployments/multi-region-deployment.mdx +++ b/deployments/multi-region-deployment.mdx @@ -1,6 +1,6 @@ --- title: Multi-Region Deployment -description: Deploy an app once and run it across multiple regions, or pin it to a specific region for data residency +description: Deploy a Cerebrium app across multiple regions or pin it to one for data residency, and control placement with region, provider, and compute. --- Deploy an app once and run it in multiple regions. The `region` parameter in `cerebrium.toml` controls placement: run globally on whatever capacity is available (recommended), or pin the app to a specific region. The parameter is optional; when omitted, the platform chooses placement automatically based on the app's hardware requirements. diff --git a/endpoints/websockets.mdx b/endpoints/websockets.mdx index f48af84f..3b6a0a84 100644 --- a/endpoints/websockets.mdx +++ b/endpoints/websockets.mdx @@ -1,5 +1,6 @@ --- title: "WebSocket Endpoints" +description: "Configure WebSocket endpoints on Cerebrium with a custom runtime, control session duration via response_grace_period, and handle SIGTERM." --- WebSocket endpoints stream responses to the client, enabling real-time, bidirectional communication.