feat(providers): add Phala confidential TDX CVM SSH-lease provider

[anagnorisis2peripeteia]

Summary

Adds phala as a Linux-only ProviderKindSSHLease provider that leases a Phala Cloud /
dstack confidential Intel TDX CVM, and — before trusting it — verifies a genuine
hardware TDX attestation that binds the box to the exact code crabbox deployed. Every
other instance provider leases a plain VM; this one leases a hardware enclave and proves
it. Adapted structurally from internal/providers/namespaceinstance, then corrected
against the real phala/dstack contract discovered by running it end-to-end on live TDX
hardware.

Attestation is the gate (not deferred)

After the CVM is reachable, Acquire fetches the dstack guest-agent attestation over SSH
(/var/run/tappd.sock → Tappd.Info → app_cert + tcb_info) and verifies, against the
app id of the CVM it just deployed:

1. RTMR replay — recompute RTMR0..3 as the SHA-384 fold of the tcb_info event log and
   require they equal the hardware registers. The measurement is genuine and unforgeable; a
   single tampered event breaks it.
2. Quote ↔ measurement — extract the Intel TDX quote embedded in app_cert (X.509 ext
   OID 1.3.6.1.4.1.62397.1.1) and require its TD-report MRTD/RTMRs equal tcb_info.
3. DCAP signature — verify.TdxQuote (go-tdx-guest) chains the quote to the Intel
   SGX/TDX Root CA: genuine Intel silicon, not an emulator.
4. Identity binding — the RTMR3 event log's app-id must equal the deployed CVM's app
   id, so the attested enclave is our deployment.

On failure the just-created CVM is destroyed (never leaked) and the lease is refused. On
success the verified app-id/compose-hash/rtmr3 are surfaced in lease labels
(attested=true). Gate defaults on (--phala-skip-attestation to opt out); untrusted
repo config may only tighten it. Verification is pure-Go (go-tdx-guest); the DCAP step
reaches Intel PCS at runtime (documented host dependency).

Live proof — real Intel TDX hardware (funded account)

End-to-end crabbox run --provider phala (no --keep → auto-release):

provisioning provider=phala lease=cbx_bbfab33501f7 slug=attest-… instance_type=tdx.small
waiting for …:22 phala cvm ssh ssh-auth… (TLS gateway)
WARN: Using embedded Intel certificate for TDX attestation root of trust   (DCAP verify)
attested phala_cvm=ea6874fd… app_id=ea6874fd… compose_hash=fe3df84a… rtmr3=72da6a69…
provisioned … state=ready            ← only AFTER attestation passed
sync complete in 14.8s ; ATTESTED_WORKLOAD_RAN ; exit=0
releasing … ✓ CVM crabbox-cbx-bbfab33501f7 deleted successfully   (final CVM count = 0)

The same path is proven on a larger class too — a tdx.medium run attested and ran
end-to-end (instance_type=tdx.medium … attested … TDX_MEDIUM_ATTESTED_OK … exit 0 … deleted). The measurement is bound dynamically: the verified app_id/compose_hash are
this deployment's, not a constant, and rtmr3 differs across runs because it folds the
per-CVM instance id — the gate matched each correctly.

The committed unit tests run against a real attestation captured off live TDX silicon
(testdata/real_attestation_info.json, real_tdx_quote.bin): RTMR replay match +
mutation-breaks, quote↔tcb_info MRTD/RTMR equality, app-id binding (+ wrong-id rejection),
and an opt-in network DCAP test (CRABBOX_TDX_DCAP_NETWORK_TEST=1) that chains the real
quote to Intel.

Test quality (mutation tested): a gremlins mutation run over the provider package
killed 267/267 runnable mutants — 100% efficacy, zero survivors (the 33 not-covered
lines are the SSH-gated guest-agent fetch / acquire orchestration, exercised live above).

Real-contract corrections (found by running it live, not by inspection)

The first live runs failed in five distinct ways the adapted-from-namespace code never
anticipated; each is fixed and pinned by a test using the real CLI shapes:

• deploy output prints a leading Provisioning CVM … line before the JSON; ids span
  snake_case (app_id, deploy/get) and camelCase (appId, list).
• cvms list omits labels and puts the name under cvmName — ownership is anchored on
  the local lease claim by cloud-id, with cvms get (which has the name) used to
  corroborate the name/lease before any destructive op.
• dev-os guest is an immutable appliance (read-only squashfs root, no package manager,
  no egress) that already ships rsync/tar/python3 but not git → bootstrap requires only the
  rsync-sync essentials; work root defaults to the writable /var/volatile/crabbox (the old
  /work sat on the read-only root and failed at sync).
• SSH is via the dstack TLS gateway (<appId>-22.<gateway-domain>:443, tunneled with
  openssl s_client), and the gateway host is cached in the claim so connections skip a
  per-connection cvms get.

Security hardening

• TLS server auth (MITM fix): the gateway tunnel now passes -verify_return_error +
  -verify_hostname (SSH host-key checking is necessarily off for a fresh per-lease CVM, so
  TLS is the only server authentication; previously it enforced nothing).
• Destructive-op safety: both ReleaseLease and Cleanup require a matching local claim
  and cvms get name corroboration, so a foreign crabbox-*-named CVM (or an app-id
  reused after teardown) is never deleted; a transient/false "not found" no longer drops the
  claim+key and orphans a live billing CVM.
• No API key is ever passed as a flag (stored phala auth only).

Review

Ran a multi-dimension AI adversarial review (ownership-safety, dstack-integration,
go-quality, security, test-quality, integration-consistency), each finding independently
verify-checked. 13 confirmed findings, all addressed — the two P1s (Cleanup name
corroboration; the TLS MITM above), the orphan-on-false-not-found P2s, the docs/marker P3s,
and the missing-coverage gaps (ReleaseLease positive-destroy is now mutation-checked; Acquire
rollback-destroy tested).

Known limitation / honest gaps

• status --wait: the generic 4-second SSH readiness probe is too tight for a cold
  TLS-gateway SSH connect, so crabbox status --wait can time out even though the box is up
  (the lease→run→release path is unaffected and proven above). The right fix is a provider
  Status method that reports readiness from CVM state rather than a live probe — a small,
  bounded follow-up I'd rather land separately than rush here.
• Adds go-tdx-guest (+ protobuf/logger transitive deps) and a runtime Intel-PCS dependency
  for the DCAP step. Justified for a confidential provider; the full network DCAP test is
  opt-in to keep CI deterministic.
• Direct-only (no coordinator/broker); Linux-only. tdx.small and tdx.medium exercised
  live (attested); the larger tdx.large/tdx.xlarge class mappings are not yet shown live.

AI assistance

Built and reviewed with AI assistance (Claude). The verification recipe (RTMR replay, quote
extraction, DCAP) was derived by pulling and verifying a real quote off live hardware before
writing the Go, and the live proofs above were captured on a funded Phala account.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed June 15, 2026, 8:51 AM ET / 12:51 UTC.

Summary
This PR adds a Linux-only Phala Cloud/dstack Intel TDX CVM SSH-lease provider with config, flags, docs, tests, live-smoke scripts, TLS-gateway SSH, cleanup, and default-on TDX attestation.

Reproducibility: not applicable. this is a new provider PR rather than a current-main bug report. The relevant verification is contributor live output plus source and fixture review, not a failing reproduction path.

Review metrics: 3 noteworthy metrics.

• Diff surface: 26 files changed; 6,087 additions, 1 deletion. This is a broad provider addition touching runtime code, docs, tests, scripts, and dependency metadata.
• New runtime dependency: 1 direct module added: github.com/google/go-tdx-guest v0.3.1. The attestation gate depends on new cryptographic verification code and its transitive supply chain.
• Live proof scope: 2 instance classes claimed: tdx.small and tdx.medium. The provided live proof covers two classes while larger class mappings remain maintainer-acceptance scope.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• none.

Risk before merge

• [P1] This creates a new confidential-compute trust boundary: users will rely on TDX quote parsing, DCAP verification, RTMR replay, app-id binding, and TLS gateway authentication before trusting a leased box.
• [P1] The provider owns live billing CVM lifecycle and destructive cleanup through local claims plus phala cvms get/delete; maintainers should accept the ownership model before merge.
• [P1] The default attestation path adds a runtime Intel PCS network dependency and a new go-tdx-guest dependency, which unit tests and green CI do not fully settle.
• [P1] The PR body still documents a cold TLS-gateway status --wait readiness caveat and larger tdx.large/tdx.xlarge mappings without live proof.

Maintainer options:

1. Accept the confidential provider boundary after review (recommended)
   Maintainers can merge after provider/security review accepts the default-on DCAP attestation path, TLS gateway server auth, Intel PCS runtime dependency, and cleanup model.
2. Require broader live class proof
   Maintainers can ask for tdx.large and tdx.xlarge live proof, or remove those class mappings until they are demonstrated on real Phala hardware.
3. Track readiness as a follow-up
   Maintainers can accept the proven lease-run-release path now and track provider-specific status readiness separately if that caveat is acceptable for first merge.

Next step before merge

• No automated repair is indicated; the remaining action is human maintainer review of the new provider scope, trust boundary, dependencies, and final merge readiness.

Security
Cleared: No concrete patch-introduced security or supply-chain defect was found, but the new confidential-compute trust boundary remains a maintainer merge-risk decision.

Review details

Best possible solution:

Merge only after maintainers explicitly accept the confidential-compute trust model and availability caveats, while keeping provider-specific lifecycle and attestation logic inside the Phala adapter.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a new provider PR rather than a current-main bug report. The relevant verification is contributor live output plus source and fixture review, not a failing reproduction path.

Is this the best way to solve the issue?

Yes, the implementation shape fits the repository boundary by keeping Phala-specific lifecycle, ownership, gateway, and attestation behavior in the provider adapter with minimal generic hooks. The remaining question is maintainer acceptance of the trust and availability model, not a narrow mechanical repair.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 57440dad819d.

Label changes

Label justifications:

• P2: This is a normal-priority provider feature with meaningful review surface but no emergency regression signal.
• merge-risk: 🚨 security-boundary: Merging makes Crabbox responsible for a new confidential-compute trust boundary involving attestation, DCAP, and TLS gateway authentication.
• merge-risk: 🚨 availability: Merging adds a live billing CVM lifecycle path with cleanup responsibilities and documented readiness/class-coverage caveats.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• feature: ✨ showcase: ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. A default-attested confidential Intel TDX CVM provider is an unusually substantive provider capability for hardware-backed isolated execution.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body and follow-up comments include after-change live lease, attestation, run, and release output on funded Phala Intel TDX hardware, including tdx.small and tdx.medium.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR body and follow-up comments include after-change live lease, attestation, run, and release output on funded Phala Intel TDX hardware, including tdx.small and tdx.medium.

Evidence reviewed

What I checked:

• Repository policy read and applied: AGENTS.md was read fully; its provider-neutral boundary is relevant because this PR keeps Phala-specific reconciliation, gateway, attestation, lifecycle, and cleanup behavior behind a provider adapter with only small generic CLI/config hooks. (AGENTS.md:13, 57440dad819d)
• Current main does not already implement Phala: A current-main search found no phala, dstack, TDX, or go-tdx-guest provider implementation, so the central provider addition is not obsolete on main. (57440dad819d)
• PR diff surface: The PR diff against its base changes 26 files with 6,087 additions and 1 deletion across provider code, CLI config/proxy hooks, docs, tests, dependency metadata, and live-smoke scripts. (1a06ea0b7185)
• Provider registration is adapter-scoped: The new provider registers as a Linux SSH-lease provider with SSH, crabbox-sync, cleanup, and coordinator disabled, matching the repository's provider-adapter model. (internal/providers/phala/provider.go:20, 1a06ea0b7185)
• Attestation gates acquire: Acquire waits for SSH, fetches dstack attestation, verifies it, records attestation labels only after success, and rolls back the created CVM on fetch or verification failure. (internal/providers/phala/backend.go:380, 1a06ea0b7185)
• Attestation verification checks measurement and identity: The attestation verifier replays RTMRs, checks quote measurements, performs optional DCAP verification, and binds the RTMR3 app-id event to the created CVM app id. (internal/providers/phala/attestation.go:255, 1a06ea0b7185)

Likely related people:

• steipete: Current-main blame points to shared config/claim surfaces, and GitHub commit metadata shows recent Namespace Compute provider work adjacent to this PR's stated structural starting point. (role: recent core/provider infrastructure contributor; confidence: medium; commits: f6b4a9765285, 5627fce63188; files: internal/cli/config.go, internal/cli/claim.go, internal/providers/namespaceinstance)
• coygeek: Merged DigitalOcean and other direct lease provider work touches the same registration, config, claim, docs, and live-smoke patterns used by this provider. (role: adjacent direct SSH provider contributor; confidence: medium; commits: aff04c9f19a3, d6be66ec65cf, baa8562f139b; files: internal/providers/digitalocean, internal/cli/config.go, internal/cli/claim.go)
• anagnorisis2peripeteia: Beyond this PR, current-main history shows prior merged provider work on Hyper-V and Tart lifecycle/cleanup surfaces, so they have relevant domain context for this provider branch. (role: adjacent provider contributor; confidence: medium; commits: f09c059f011f, b3c0c4c3cb6f; files: internal/providers/hyperv, internal/providers/tart)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[anagnorisis2peripeteia]

Thanks for the precise review — both P1s are addressed in f641dc5e, using the Phala CLI contract you cited:

P1 — compose required for non-interactive deploy (backend.go): the provider now always supplies --compose. When none is configured it writes a bundled minimal default (default-compose.yml — a long-lived debian:stable-slim box) into the per-lease dir; an explicit --phala-compose still overrides. TestCreateAlwaysSuppliesComposeFlag asserts --compose is present on both the default and configured paths.

P1 — TLS gateway SSH contract (phala_proxy.go): rewritten off raw TCP / flat fields. It now parses the cvms get --json gateway object (gateway_domain / nested base_domain) plus the app id, derives <appId>-22.<gateway-domain>, and tunnels via openssl s_client -connect <host>:443 -servername <host> (openssl documented as a host dependency). Tests cover the host derivation and the openssl tunnel argv. The earlier TODO(phala) is removed now that the contract is known.

Local gates green: go build ./..., go vet ./internal/providers/phala/... ./internal/cli/..., and go test ./internal/providers/phala/... ./internal/cli/... -run Phala.

The remaining status: needs proof item is the funded live smoke. The account's $20 trial credit is gated behind a first card-bind (Phala's "$20 with your first card bind or $1+ top-up"), which the contributor needs to complete; once funded I'll attach a redacted lease→run→release transcript and lift the draft status.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27531466350
• Updated: 2026-06-15T07:56:13.061Z

[anagnorisis2peripeteia]

Major update since the last review — the provider is now live-proven on real Intel TDX hardware and gained an attestation gate:

• Attestation (new, the point of a confidential provider): Acquire now fetches the dstack guest-agent quote over SSH and verifies RTMR replay + quote↔tcb_info consistency + the Intel DCAP signature chain (go-tdx-guest → Intel SGX/TDX Root CA) + app-id binding to the deployed CVM, before trusting the box; it destroys + refuses a non-attesting CVM. Default-on. Tested against a real quote captured off live silicon (testdata/).
• Live proof: full lease → attest → run → release exercised end-to-end on a funded account (transcript in the PR body); the earlier environment_blocked smoke is now real.
• 5 real-contract fixes found by running it live (deploy prefix, cvms list name under cvmName + claim-anchored ownership, immutable dev-os guest/work-root, TLS-gateway SSH + cached host).
• Security: TLS server-auth MITM fix (-verify_return_error -verify_hostname); Cleanup now name-corroborates like ReleaseLease; no orphaning a live CVM on a transient not-found.
• A 6-dimension AI adversarial review found 13 findings, all addressed (incl. the 2 P1s above).

Known gap, stated honestly: crabbox status --wait's generic 4s SSH readiness probe is too tight for a cold TLS-gateway connect — a bounded follow-up (needs a core change), the lease→run→release path is unaffected.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Superseded
• Detail: A newer re-review for this item started before this run finished, so GitHub cancelled this older run. Check the latest ClawSweeper run for the current result.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27544658289
• Updated: 2026-06-15T12:04:53.652Z

[anagnorisis2peripeteia]

Both findings from the last review are addressed in eb47bdcd:

• [P1] Persist non-repo acquire claims — Acquire's success path no longer calls the repo-only claim writer unconditionally. Extracted claimAcquiredLease() which falls back to ClaimLeaseTargetForConfig when req.Repo.Root is empty, so a non-repo acquire (warmup, or run outside a repo) still writes the local ownership claim that List/stop/Cleanup depend on. TestClaimAcquiredLeasePersistsNonRepoClaim pins that an empty (and whitespace) repo root writes a resolvable claim with the right cloud-id and slug.
• [P3] Attestation docs — docs/providers/phala.md now documents the default-on Acquire gate (RTMR replay + quote↔tcb_info consistency + DCAP-to-Intel-root + app-id binding) and the --phala-skip-attestation opt-out; added attest to the config example, corrected the provider-metadata.json caveat, and regenerated the provider matrix (check-provider-matrix passes).

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27545023102
• Updated: 2026-06-15T12:13:27.131Z

[anagnorisis2peripeteia]

Strengthened since the last pass — addressing the remaining maintainer-acceptance items:

• Larger-class live proof: tdx.medium now attested + run + released end-to-end (instance_type=tdx.medium … attested … TDX_MEDIUM_ATTESTED_OK … exit 0 … deleted), so the live matrix is no longer tdx.small-only (added to the PR body).
• Mutation tested: a gremlins run over the provider package kills 267/267 runnable mutants — 100% efficacy, zero survivors. The only uncovered lines are the SSH-gated guest-agent fetch / acquire orchestration (exercised live, not unit-reachable without an SSH seam); parseDstackInfo was split out of fetchAttestation (commit 1a06ea0b) to make the attestation-response parsing deterministically covered.

Known follow-ups, stated honestly: status --wait cold-TLS-gateway readiness (needs a small core cli change, kept out to stay focused) and the tdx.large/xlarge class mappings (not yet shown live).

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27546585877
• Updated: 2026-06-15T12:41:17.804Z