compute: resolve how GPU work is funded at runtime, and tell the agent honestly

backend/cli/src/agent/prompt/biology.txt

@@ -56,7 +56,8 @@ Before using any external service, verify credentials:
   [ -n "$VAR_NAME" ] && echo "set" || echo "not set"
 
 If not connected, tell the user:
-  "Connect [service] at https://app.syntheticsciences.ai -> Services, then restart openscience."
+  "Connect [service] at https://app.syntheticsciences.ai -> Services (GPU provider keys:
+  Settings ▸ Compute). A newly connected key is picked up on the next message — no restart."
 
 ## Native Database & Analysis Tools
 

backend/cli/src/agent/prompt/ml.txt

@@ -57,7 +57,8 @@ Before using any external service, verify credentials are set:
 ```
 Common: `TINKER_API_KEY`, `MODAL_TOKEN_ID`+`MODAL_TOKEN_SECRET`, `HF_TOKEN`, `WANDB_API_KEY`,
 `PRIME_API_KEY`, `TENSORPOOL_KEY`. If a service is not connected, tell the user to connect it
-in Settings → Credentials and restart, then continue with what you can do.
+in Settings → Credentials (GPU provider keys: Settings → Compute); it is picked up on the
+next message, no restart. Continue with what you can do meanwhile.
 
 ## CRITICAL: Environment Setup
 Before running ANY Python, set up an isolated environment in the working directory:

backend/cli/src/agent/prompt/research.txt

@@ -42,7 +42,8 @@ Before using any external service, verify credentials:
   [ -n "$VAR_NAME" ] && echo "set" || echo "not set"
 
 If not connected, tell the user:
-  "Connect [service] at https://app.syntheticsciences.ai -> Services, then restart openscience."
+  "Connect [service] at https://app.syntheticsciences.ai -> Services (GPU provider keys:
+  Settings ▸ Compute). A newly connected key is picked up on the next message — no restart."
 
 ## CRITICAL: Convergence & Anti-Loop
 Your job is to finish the task IN THIS SESSION, not to hand it off.
@@ -251,12 +252,17 @@ implementation in up-to-date usage rather than potentially stale training data.
 
 ### Stage 5: COMPUTE
 Execute computational work. If `methodology.md` exists, follow the pipeline defined there.
-- Managed compute (Daytona-backed) runs through the bundled `atlas` CLI when your Atlas
-  session is active. Run `atlas doctor --format=json` first; if it reports the CLI is
-  unavailable/unauthenticated, print a one-line note and fall back to the BYOK cloud-compute
-  skills below (Modal, Tinker, TensorPool, Prime Intellect, HF Jobs) — never block on it.
-- Load: `modal-research-gpu` for GPU-accelerated scientific computing
-- Load: `modal` for general serverless GPU (inference, serving)
+- Call the `compute_status` tool before launching any GPU work. It reports how compute is
+  funded right now — `byok`, `managed`, or `none` — which providers are usable, and the rule
+  that applies. This is the only compute-availability signal — CLI auth status is not one.
+- If it returns `byok`, load the cloud-compute skill for one of the providers it lists —
+  when Modal is one of them, `modal-research-gpu` for GPU-accelerated scientific computing
+  or `modal-serverless-gpu` for general serverless GPU (inference, serving). Never load a
+  compute skill for a provider the tool did not list.
+- If it returns `managed`, do not launch GPU work either — OpenScience cannot start a
+  managed lease yet. Follow the rule the tool returns and tell the user what it says.
+- If it returns `none`, do not launch GPU work — tell the user to connect a provider key
+  in Settings ▸ Compute.
 - Load: domain libraries as needed (see Scientific Computing skills)
 - Present cost estimate and get approval before launching jobs
 - Run computations, monitor progress, collect outputs
@@ -350,7 +356,7 @@ Tracking: weights-and-biases, mlflow, tensorboard, langsmith, phoenix
 ### Inference & Deployment — serving, quantization, benchmarking
 Serving: vllm, sglang, tensorrt-llm, llama-cpp, outlines
 Quantization: bitsandbytes, awq, gptq, hqq, gguf
-Platforms: modal, lambda-labs, tensorpool, skypilot, fireworks-ai, groq, together-ai
+Platforms: modal-serverless-gpu, lambda-labs, tensorpool, skypilot, fireworks-ai, groq, together-ai
 Eval: lm-evaluation-harness, bigcode-evaluation-harness, llm-as-judge-evaluation, hugging-face-evaluation
 
 ### Data & Embeddings — DataFrames, datasets, tokenizers, vector stores
@@ -384,7 +390,7 @@ Writing: scientific-writing, ml-paper-writing, research-grants, venue-templates,
 Docs: scientific-slides, paper-2-web, latex-posters, pptx-posters, markitdown, market-research-reports
 
 ### Cloud Compute — GPU provisioning, serverless, distributed
-GPU: modal-research-gpu, modal, lambda-labs, tensorpool, prime-intellect-lab, skypilot
+GPU: modal-research-gpu, modal-serverless-gpu, lambda-labs, tensorpool, prime-intellect-lab, skypilot
 Distributed: ray-train, ray-data
 Managed: tinker, hugging-face-jobs
 Utility: get-available-resources

backend/cli/src/compute/mode.ts

@@ -0,0 +1,257 @@
+/**
+ * Runtime resolution of how GPU compute is funded.
+ *
+ * `billing.compute` used to answer this from config alone, which meant a
+ * brand-new user with zero provider keys resolved to "byok" — claiming BYOK
+ * with nothing to BYOK with. This module answers it from the environment
+ * instead, and can say "none", which is the state we previously handled worst.
+ *
+ * Resolution deliberately happens ON DEMAND and never at startup. Provider keys
+ * reach process.env from three places — the user's shell, the Credentials panel
+ * (`applyCredentialEnv`, src/index.ts:102) and the Compute panel
+ * (`applyComputeEnv`, src/index.ts:106) — and the latter two are wrapped in
+ * `.catch(() => {})`. Detecting at boot would report "none" for a user whose
+ * keys are configured through the UI. Both call sites (SkillTool.init and the
+ * compute_status tool) run per request, long after those injections, so the
+ * ordering constraint cannot be violated and cannot silently regress if someone
+ * reorders src/index.ts later.
+ */
+import { Config } from "@/config/config"
+import { API_BASE, OpenScience } from "@/openscience"
+
+export namespace ComputeMode {
+  export type Source = "byok" | "managed" | "none"
+
+  /**
+   * `env` is a list of ALTERNATIVE groups; a group is satisfied when every var in
+   * it is set and non-empty. Modal is the only pair — its single pasted key
+   * splits into a token id + secret, and a half-pasted one maps to nothing
+   * (mirroring `mapProviderEnv`, server/routes/settings/compute.ts:181).
+   *
+   * `skills` are frontmatter `name` values, NOT directory names and NOT
+   * category-prefixed. Only these names are subject to mode filtering; the other
+   * cloud-compute skills (tinker, skypilot, fireworks, together) are inference
+   * APIs and orchestrators keyed by their own credentials, not GPU leases this
+   * mode governs, and are never hidden.
+   */
+  export const PROVIDERS: Record<string, { env: string[][]; skills: string[] }> = {
+    modal: {
+      env: [["MODAL_TOKEN_ID", "MODAL_TOKEN_SECRET"]],
+      skills: ["modal-serverless-gpu", "modal-ml-training", "modal-research-gpu"],
+    },
+    lambda: {
+      env: [["LAMBDA_API_KEY"], ["LAMBDA_LABS_API_KEY"]],
+      skills: ["lambda-labs-gpu-cloud"],
+    },
+    tensorpool: {
+      env: [["TENSORPOOL_KEY"], ["TENSORPOOL_API_KEY"]],
+      skills: ["tensorpool-gpu-cloud"],
+    },
+    prime: {
+      env: [["PRIME_API_KEY"], ["PRIME_INTELLECT_API_KEY"]],
+      skills: ["prime-intellect-lab"],
+    },
+    runpod: {
+      env: [["RUNPOD_API_KEY"]],
+      skills: [],
+    },
+    vast: {
+      env: [["VAST_API_KEY"]],
+      skills: [],
+    },
+  }
+
+  /** Every provider skill name — the exact set the catalog filter operates on. */
+  export const SKILLS = new Set(Object.values(PROVIDERS).flatMap((p) => p.skills))
+
+  /** Read process.env directly rather than Env.get: applyComputeEnv writes to
+   *  process.env first and mirrors to Env only when instance state exists, so
+   *  process.env is the one source that is always populated. */
+  function keyed(groups: string[][]): boolean {
+    return groups.some((group) => group.every((name) => !!process.env[name]))
+  }
+
+  /**
+   * The credentialed GPU providers, in declaration order.
+   *
+   * A credential is the whole test. An earlier revision also required a
+   * matching skill, on the theory that a provider with no skill gives the agent
+   * nothing to act on — but a capable agent drives a documented cloud API from a
+   * key, so that conjunction only produced a false "no compute available" for
+   * users holding a perfectly workable key. A skill, where one exists, is a
+   * quality boost; the catalog filter still offers a provider's skills only when
+   * that provider is credentialed.
+   */
+  export function usable(): string[] {
+    return Object.keys(PROVIDERS).filter((id) => keyed(PROVIDERS[id].env))
+  }
+
+  /**
+   * WHY `mode` is what it is — the one thing a caller cannot reconstruct from
+   * `mode` alone, and the thing that decides what advice is even actionable.
+   * "managed" reached from the environment means the user holds no provider
+   * credential, so connecting one flips the next call to byok. "managed"
+   * reached from `billing.compute` means the setting pins it: connecting a
+   * credential changes neither `resolve()` (`funded()` never consults
+   * `providers`) nor `offered()` (empty under a managed override), so telling
+   * the user to connect one is advice the setting itself defeats. The override
+   * value is carried, not just a forced/not-forced bit, because "none" is
+   * reachable from BOTH overrides and they need opposite advice.
+   */
+  export type Origin = "environment" | "config:byok" | "config:managed"
+
+  export interface Resolution {
+    mode: Source
+    /** Credentialed BYOK providers, in PROVIDERS declaration order. */
+    providers: string[]
+    /**
+     * Whether managed compute is available — `undefined` when nothing measured
+     * it. The byok arms return before `available()` runs (that skip is the
+     * point: a byok user never pays for the round trip), so they have no
+     * verdict to report. Absence means "not checked", never "no": a hardcoded
+     * `false` there would be a fact the code never established, which is the
+     * exact defect this module exists to remove.
+     */
+    managed?: boolean
+    /** Wallet balance in USD. Present only when mode === "managed". */
+    balance?: number
+    origin: Origin
+  }
+
+  /** Hard ceiling on how long resolution may block an agent turn. Atlas's own
+   *  60s default is far too long to sit in front of a tool call; a slow or
+   *  hanging backend must degrade to "none", not stall the turn. */
+  const TIMEOUT = 3_000
+
+  /** Short in-process TTL, enough to stop a chatty agent hammering the endpoint
+   *  inside one turn and no longer. The whole reason this is a tool rather than
+   *  a prompt injection is that the answer changes mid-session, so a long cache
+   *  would reintroduce exactly the staleness the tool exists to avoid. */
+  const TTL = 5_000
+
+  let cache: { at: number; value: { managed: boolean; balance?: number } } | undefined
+
+  /** Drop the availability cache. Called by tests; also safe after a connect. */
+  export function invalidate() {
+    cache = undefined
+  }
+
+  /**
+   * One authenticated call to /api/compute/options, which already annotates each
+   * provider with `funding` — "managed" when reselling is on and an operator key
+   * exists, else "unavailable". A failed, unauthenticated or timed-out call is
+   * treated as UNAVAILABLE: failing toward "none" produces an honest "connect a
+   * key" message, whereas failing toward "managed" would reproduce the bug this
+   * design exists to fix, promising a capability we never confirmed.
+   */
+  async function available() {
+    if (cache && Date.now() - cache.at < TTL) return cache.value
+    const value = await probe()
+    cache = { at: Date.now(), value }
+    return value
+  }
+
+  async function probe(): Promise<{ managed: boolean; balance?: number }> {
+    const session = await OpenScience.getSession().catch(() => null)
+    if (!session) return { managed: false }
+    try {
+      const res = await fetch(`${API_BASE}/api/compute/options`, {
+        headers: { Authorization: `Bearer ${session.api_key}` },
+        signal: AbortSignal.timeout(TIMEOUT),
+      })
+      if (!res.ok) return { managed: false }
+      const data = await res.json()
+      const providers = Array.isArray(data?.providers) ? data.providers : []
+      const managed = providers.some((entry: { funding?: string }) => entry?.funding === "managed")
+      if (!managed) return { managed: false }
+      const cents = data?.cli_effective_balance_cents
+      return { managed: true, balance: typeof cents === "number" ? cents / 100 : undefined }
+    } catch {
+      return { managed: false }
+    }
+  }
+
+  /** Build a Resolution purely from an availability probe's verdict — never
+   *  from `providers`, which is passed through only for display. Shared by the
+   *  managed-override arm and the no-override/no-provider fallback: both trust
+   *  `available()` completely and must never fall back to "byok" just because
+   *  a credential happens to be present (that would silently defeat the
+   *  managed override — see the "managed with a usable provider" test). */
+  function funded(providers: string[], state: { managed: boolean; balance?: number }, origin: Origin): Resolution {
+    return {
+      mode: state.managed ? "managed" : "none",
+      providers,
+      managed: state.managed,
+      balance: state.managed ? state.balance : undefined,
+      origin,
+    }
+  }
+
+  /**
+   * The single shared entry point. `billing.compute` is an OVERRIDE, not the
+   * source of truth: it may narrow the outcome to "none", but it may never
+   * manufacture a capability that isn't there.
+   */
+  export async function resolve(): Promise<Resolution> {
+    const providers = usable()
+    const override = (await Config.get()).billing?.compute
+
+    if (override === "byok") {
+      return { mode: providers.length ? "byok" : "none", providers, origin: "config:byok" }
+    }
+
+    if (override === "managed") {
+      return funded(providers, await available(), "config:managed")
+    }
+
+    // BYOK wins when a credentialed provider is present: it is free to the user,
+    // it works today, and it needs nothing from Atlas. This is also why a BYOK
+    // user never pays for the availability call — and why `managed` is left
+    // unset here rather than false (see Resolution.managed).
+    if (providers.length) return { mode: "byok", providers, origin: "environment" }
+
+    return funded(providers, await available(), "environment")
+  }
+
+  /**
+   * The skill names the catalog filter should offer. The invariant is two
+   * one-way implications, not a single "iff" on non-emptiness:
+   *   - `offered()` non-empty IMPLIES `resolve()`'s mode is "byok" — only the
+   *     funded path ever offers anything.
+   *   - mode "byok" IMPLIES `offered()` equals the credentialed providers'
+   *     skills exactly, which is the EMPTY set when those providers carry no
+   *     catalogued skill (`runpod`, `vast`: `skills: []`). A byok user with
+   *     only a RunPod key correctly sees nothing offered here — RunPod has no
+   *     skill to offer, though the agent can still drive its API directly.
+   * Empty in every other mode too, including "managed" with a real credential
+   * sitting unused (see `Resolution.providers`'s doc comment) — offering it
+   * there would dangle the user's own uncapped provider account in front of
+   * an agent that has just been told not to touch it.
+   *
+   * This mirrors `resolve()`'s byok arms exactly but never reaches the
+   * availability probe, because it doesn't need to: whether the mode is
+   * "byok" is fully decided by `usable()` (env-only) and the override alone.
+   *
+   *   - override "byok"    -> byok iff a credential exists; no network either way.
+   *   - override "managed" -> mode is "managed" or "none"; either way offered is empty.
+   *   - unset              -> byok iff a credential exists; the network arm only
+   *                           runs when there are no credentials, and then offered
+   *                           is empty regardless of what it returns.
+   *
+   * `available()`/`probe()` — the per-LLM-step Atlas round trip this function
+   * exists to eliminate — is never reached here. The only I/O is
+   * `Config.get()`, which `Instance.state` memoizes per project instance
+   * after its first read within that instance; that first read can itself
+   * issue a fetch when a `wellknown` auth entry is configured
+   * (config.ts:82-105), but that cost belongs to `Config.get()` and is paid
+   * at most once per instance, not once per step — it is not a cost this
+   * function adds.
+   */
+  export async function offered(): Promise<Set<string>> {
+    const providers = usable()
+    const override = (await Config.get()).billing?.compute
+    if (override === "managed") return new Set()
+    if (!providers.length) return new Set()
+    return new Set(providers.flatMap((id) => PROVIDERS[id].skills))
+  }
+}

backend/cli/src/config/config.ts

@@ -1057,9 +1057,10 @@ export namespace Config {
             ),
           compute: z
             .enum(["managed", "byok"])
+            .nullable()
             .optional()
             .describe(
-              "How GPU/compute is paid for. 'managed' runs on Atlas-provisioned compute billed to your wallet (via the bundled atlas CLI); 'byok' uses your own connected GPU providers (Modal, Tinker, TensorPool, …). Unset = byok.",
+              "How GPU/compute is paid for. 'managed' runs on Atlas-provisioned compute billed to your wallet; 'byok' uses your own connected GPU providers (Modal, Lambda Labs, TensorPool, Prime Intellect, RunPod, Vast.ai). Unset or null = auto-detect from your connected providers. Setting this can only narrow the result — if the mode you pick isn't actually available, compute resolves to none rather than pretending.",
             ),
         })
         .optional()

backend/cli/src/sandbox/sandbox.ts

@@ -259,6 +259,16 @@ export namespace Sandbox {
     // Whole fs read-only, a fresh /dev and /proc, and a throwaway writable /tmp;
     // then re-mount the bits that must be writable on top.
     const args = ["--ro-bind", "/", "/", "--dev", "/dev", "--proc", "/proc", "--tmpfs", "/tmp"]
+    // $HOME is deliberately refused as a writable root (tooBroadToConfine), which
+    // leaves the whole-fs --ro-bind covering the XDG cache dir too. Startup tools
+    // that write there unconditionally (zsh's compdump/history lock, pip/npm/uv
+    // caches, ...) then fail with "Read-only file system". A tmpfs — not a bind —
+    // is the fix: writes succeed so those tools stop erroring, but nothing here
+    // persists to the real home, so the containment tooBroadToConfine enforces
+    // stays intact. Mounted before the writable binds below, so an explicitly
+    // writable path under the cache dir still wins.
+    const cache = process.env.XDG_CACHE_HOME || path.join(os.homedir(), ".cache")
+    args.push("--tmpfs", cache)
     for (const p of dedupe(policy.writable)) {
       // Skip only the /tmp mount root itself — it is provided as a fresh tmpfs and
       // re-binding host /tmp would defeat it. A workspace that lives *under* /tmp