From 3d094bcde5e9cac73518399869f74e47b83f31bd Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Sun, 30 Aug 2026 11:35:12 -0400 Subject: [PATCH] docs(#6689): two setup routes and per-repository trust in the OpenAI WIF guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guide read as if managing the organization's identity provider were a prerequisite. In most companies an IT administrator owns Organization Settings and the GitHub Actions provider, and a team owns (or requests) a project and a service account. The setup is now an explicit fork: shared claims check first, then route A (you manage providers: add or reuse one, add the mapping) or route B (send the administrator a request — template included — and record the identifiers you get back); step 4 onwards is common to both. The audience is described as whatever the provider has (any string, used verbatim) instead of a fullsend:// value. The per-repository trust rule is stated for both routes: one mapping (or allowlist entry) per company-owned repository, never a pattern over the organization, since an organization can contain repositories the company does not own. Also drops the org-wide installation note (that mode is deprecated), re-points the troubleshooting rows, and fixes the pi runtime doc's "three repository variables" wording. Assisted-by: Claude Signed-off-by: Wayne Sun --- .../openai-workload-identity.md | 199 ++++++++++++------ docs/runtimes/pi.md | 3 +- 2 files changed, 139 insertions(+), 63 deletions(-) diff --git a/docs/guides/infrastructure/openai-workload-identity.md b/docs/guides/infrastructure/openai-workload-identity.md index 0d978da96..d3c1589c0 100644 --- a/docs/guides/infrastructure/openai-workload-identity.md +++ b/docs/guides/infrastructure/openai-workload-identity.md @@ -6,43 +6,62 @@ short-lived token (minutes — it never outlives the GitHub token it came from, fullsend renews it for as long as the run lasts, and the agent sandbox never sees any of it — it only ever holds a placeholder that the OpenShell gateway swaps for the real token on the way out. -You set this up once per OpenAI organization and once per repository. It takes one visit to the -OpenAI console and three repository variables. No key is created, downloaded or rotated. +Setting it up is one visit to the OpenAI console — yours, or your IT administrator's — and one +`fullsend github setup` run per repository. No key is created, downloaded or rotated. > **GitHub Actions only.** The exchange needs the job's OIDC endpoint. For GitLab CI and for runs > on your own machine, use an API key in the runner environment — see [Run it locally](#run-it-locally). -## Before you start - -- [ ] You can manage **Workload Identity Providers** in your OpenAI organization - (Organization Settings → Security). This is an organization-level permission; being an owner - of a project is not enough. -- [ ] You have an OpenAI **project** for the runs, ideally a dedicated one with a budget alert. -- [ ] The repository is enrolled per-repo on a fullsend release that includes this feature - (the release notes for #6689 name it). - -## 1. Add the identity provider (once per organization) - -In the OpenAI console go to **Organization Settings → Security → Workload Identity Provider** and -add a provider: - -| Field | Enter | -|---|---| -| OIDC issuer URL | `https://token.actions.githubusercontent.com` | -| Audience | `fullsend://` (or any string you choose — you will use it verbatim below) | -| Use uploaded JWKS for token verification | **Off** | - -Copy the **identity provider ID** — you need it in step 4. - -The audience is not issued by anyone: whatever you type here is what your runs will request, -character for character. Pick it once and reuse it for every repository. - -## 2. See what your repository actually claims (once) - -OpenAI decides whether to trust a run by comparing claims in the GitHub token with the mapping you -write in step 3. A wrong audience and a wrong claim fail the same way, so look at the real values -before writing the mapping. Add this temporary workflow to the repository, run it, and copy the -printed claims: +## What you end up with + +Three identifiers, which you hand to fullsend in [step 4](#4-tell-fullsend-the-three-identifiers). +They are not secrets: on their own they grant nothing. + +| Identifier | What it is | Where it comes from | +|---|---|---| +| **Audience** | The string your runs ask GitHub to put in the token's `aud` claim. It must equal what the identity provider was created with, character for character. | Whoever created the provider — you (route A) or your IT admin (route B). It is not issued by anyone and has no required format. | +| **Identity provider ID** | The OpenAI Workload Identity Provider for GitHub Actions in your organization. | The provider's page in the OpenAI console (route A), or your IT admin (route B). | +| **Service account ID** | The service account the mapping grants to your repository, in the project the runs are billed to. | The mapping (route A), or your IT admin (route B). | + +You also need an OpenAI **project** to bill the runs to (ideally a dedicated one with a budget +alert), and the repository must be enrolled per-repo on a fullsend release that includes this +feature (one that includes fullsend PR #6695; check the release notes). + +## Which route are you on? + +Workload Identity Providers and their service-account mappings are an **organization-level** +security setting in OpenAI (Organization Settings → Security). Who can edit them decides your route: + +- **Route A — you can manage providers in your OpenAI organization.** Being an owner of a project + is not enough; you need the organization-level permission. You create (or reuse) the provider and + add one mapping per repository yourself. Do [step 1](#1-see-what-your-repository-actually-claims-both-routes), + then [A2](#a2-add-or-reuse-the-identity-provider-route-a) and [A3](#a3-map-the-repository-to-a-service-account-route-a). +- **Route B — the provider is managed centrally.** This is the common shape in a company: an IT + administrator owns Organization Settings and the GitHub Actions provider; you own (or request) a + project. A service account is a non-human API principal inside a project; the administrator can + create it while adding the mapping, so all you need to know is the project's name or ID + (Organization → Projects). You send one request per repository and receive the three + identifiers back. Do [step 1](#1-see-what-your-repository-actually-claims-both-routes), then + [B2](#b2-send-the-request-route-b) and [B3](#b3-record-what-you-get-back-route-b). + +Either way, steps 4 and onwards are the same. + +> **The rule that holds in both routes: trust is per repository.** A mapping asserts +> `repository == /` (and `ref == refs/heads/main`) for each company-owned +> repository you enrol — one mapping per repository (a mapping has no list form: its assertions are +> exact values, all of which must match). Never a pattern over the organization +> (`repository_owner`, a name prefix, a derived attribute): a GitHub organization often contains +> repositories the company does not own, and a pattern would let every one of them obtain your +> token. Adding a repository later means adding its assertion; that is the gate, by design. + +## 1. See what your repository actually claims (both routes) + +OpenAI decides whether to trust a run by comparing claims in the GitHub token with the mapping's +assertions. A wrong audience and a wrong claim fail the same way, so look at the real values before +writing the mapping (route A) or before asking for it (route B). Add this temporary workflow to +the repository, run it, and copy the printed claims. Put the provider's audience in `AUD` if you +already know it; if you do not yet, any string works for this check — `repository` and `ref` are +what you are after. ```yaml name: openai-wif-check @@ -55,7 +74,7 @@ jobs: runs-on: ubuntu-latest steps: - env: - AUD: fullsend:// + AUD: run: | TOK=$(curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=$(jq -rn --arg a "$AUD" '$a|@uri')" | jq -r .value) @@ -70,7 +89,7 @@ You will see something like: ```json { "iss": "https://token.actions.githubusercontent.com", - "aud": "fullsend://", + "aud": "", "repository": "/", "ref": "refs/heads/main", "workflow_ref": "//.github/workflows/openai-wif-check.yml@refs/heads/main", @@ -81,36 +100,44 @@ You will see something like: For the real agent runs, `repository` and `ref` are the same as above; `workflow_ref` names whichever of the fullsend workflows installed in your repository started the run (`code.yml`, `triage.yml`, `review.yml`, `fix.yml`, `prioritize.yml`, `retro.yml` or `dispatch.yml` under `.github/workflows/`), -and `job_workflow_ref` the pinned fullsend reusable workflow it calls. Delete the check workflow when -you are done. +and `job_workflow_ref` the pinned fullsend reusable workflow it calls — which is why a mapping asserts +`repository` and `ref`, not `workflow_ref`. Delete the check workflow when you are done. -## 3. Map the repository to a service account (once per repository) +## A2. Add or reuse the identity provider (route A) -Open the provider you created and add a **service account mapping**: +If your organization already has a provider for GitHub Actions, reuse it: open it, note its +**audience** and copy its **identity provider ID**, and go to A3. Otherwise, in **Organization +Settings → Security → Workload Identity Provider**, add one: | Field | Enter | |---|---| -| Claim assertions | `iss` = `https://token.actions.githubusercontent.com` · `aud` = your audience · `repository` = `/` · `ref` = `refs/heads/main` | +| OIDC issuer URL | `https://token.actions.githubusercontent.com` | +| Audience | Any string you choose, for example `fullsend://` — your runs will request it verbatim | +| Use uploaded JWKS for token verification | **Off** | + +Copy the **identity provider ID**. One provider serves every repository; OpenAI allows 50 providers +per organization and 50 mappings per provider. + +## A3. Map the repository to a service account (route A) + +Open the provider and add a **service account mapping**: + +| Field | Enter | +|---|---| +| Claim assertions | `iss` = `https://token.actions.githubusercontent.com` · `aud` = the provider's audience · `repository` = `/` · `ref` = `refs/heads/main` | | Project | the project the runs should be billed to | | Service account | create a new one, for example `fullsend--ci` | | Permissions | `api.model.request` (fullsend also accepts `api.model.read`; anything broader is refused at run time) | -Every assertion must match what step 2 printed character for character; a one-letter difference -fails every exchange. Do not assert `workflow_ref`: fullsend runs agents from seven workflow files -(see step 2), so a single `workflow_ref` value would exclude all but one of them. If you want that -narrowing anyway, create one mapping per workflow file. - -Copy the **service account ID** — you need it in step 4. +Every assertion must match what step 1 printed character for character; a one-letter difference +fails every exchange. Copy the **service account ID**. Two things not to do: - **Do not create an API key** for the service account. The mapping is the credential; a key would put a long-lived secret back into the picture. -- **Do not assert `repository_owner` instead of `repository`.** That would trust every repository - in your GitHub organization, including ones anybody with repository-create rights can add. - -OpenAI allows 50 mappings per provider and 50 providers per organization, and mappings are created -in the console only. Treat enrolment as a deliberate step per repository. +- **Do not assert `repository_owner`, a prefix or any pattern instead of `repository`** — see the + per-repository rule above. **Which runs this mapping trusts.** `ref` = `refs/heads/main` covers every fullsend agent workflow as installed on `main`, and GitHub mints the token for the repository the job runs in — so the @@ -120,9 +147,56 @@ dispatch authorization (only events from people with write access trigger agents If you want the mapping itself to exclude some triggers, add a claim the untrusted runs cannot carry — for example a GitHub `environment` that the agent jobs reference and that protects itself with required reviewers — and assert `environment == ""` here. A `workflow_dispatch` of a -fullsend workflow on a feature branch will not match `refs/heads/main`; that is expected. In the -org-wide installation mode the runs happen in the organization's central `.fullsend` repository, -so assert that repository instead. +fullsend workflow on a feature branch will not match `refs/heads/main`; that is expected. + +## B2. Send the request (route B) + +You cannot see the provider, so ask the administrator who owns it. Send one request per +repository (or one listing several — the administrator still creates one mapping per repository), +and include the claims from step 1 so the assertions are copied, not retyped. A template: + +```text +Subject: OpenAI Workload Identity mapping for GitHub repository / + +Please add a service-account mapping on the organization's GitHub Actions identity provider +(issuer https://token.actions.githubusercontent.com) with these claim assertions, exactly: + + iss = https://token.actions.githubusercontent.com + aud = + repository = / + ref = refs/heads/main + +Target project: (the project these runs are billed to) +Service account: create a new one named fullsend--ci in that project + (or map the existing service account ) +Permissions on the mapping: api.model.request only — nothing broader, and please do not +create an API key for the service account; the mapping is the credential. + +Please do not add a workflow_ref or sub assertion: these runs start from seven different +workflow files in the repository, so a single value would exclude the others. + +One mapping per repository, please (a mapping matches exact values only) — not a wildcard +or a pattern over the organization, since not every repository in it is ours. + +Please send back: the identity provider ID, the provider's audience string, and the +service account ID. +``` + +If the administrator's standard mapping grants more than model access, ask for it to be narrowed: +fullsend refuses a token whose permissions exceed `api.model.request`/`api.model.read`, and only +warns when the mapping does not narrow at all. If you want the mapping to exclude some triggers +(a GitHub `environment` with required reviewers, for example), the reasoning is under +[A3](#a3-map-the-repository-to-a-service-account-route-a) — ask for the extra assertion in the +same request. + +## B3. Record what you get back (route B) + +The reply gives you the three identifiers from [What you end up with](#what-you-end-up-with). +Re-run the step-1 workflow with the real audience in `AUD` and confirm `aud` and `repository` print +as expected — that is the whole verification you can do from your side before the first run. If the +first run's exchange still returns 4xx, the assertions and the claims differ somewhere (compare them +character for character with the administrator); a `repository` that is not in any mapping is the +usual cause when a second repository is enrolled. ## 4. Tell fullsend the three identifiers @@ -130,9 +204,9 @@ Re-run the setup command you enrolled the repository with, adding the three valu ```bash fullsend github setup / \ - --openai-audience "fullsend://" \ - --openai-identity-provider-id "" \ - --openai-service-account-id "" + --openai-audience "" \ + --openai-identity-provider-id "" \ + --openai-service-account-id "" ``` It writes them into the repository's `.fullsend/config.yaml`, the same way it records the Vertex @@ -141,18 +215,19 @@ project and provider: ```yaml inference: openai: - audience: fullsend:// + audience: identity_provider_id: service_account_id: ``` Commit that change (setup opens a pull request for it unless you pass `--direct`). A base configuration (`config.base.yaml`, or a vendor preset) can carry the block for many repositories, -and a repository can restate any one of the three. +and a repository can restate any one of the three — with a centrally managed provider, the audience +and the provider ID are typically the same for every repository and only the service account differs. **Is it safe to commit these?** Yes. They are identifiers, not secrets: on their own they grant nothing. OpenAI issues a token only to a caller presenting a GitHub OIDC token whose claims match -the mapping from step 3, and only your repository's `main` workflow can obtain one. fullsend reads +a mapping, and only your repository's `main` workflow can obtain one. fullsend reads `.fullsend/config.yaml` from the base branch for pull-request events, so a pull request cannot change them for the run that reviews it. The worst thing someone with write access could do is point the block at their own OpenAI organization — and pay for your runs. fullsend prints the three @@ -256,9 +331,9 @@ agent starts and names the rule. | `no OpenAI credential: set FULLSEND_OPENAI_AUDIENCE, …` | Run step 4 (or add the three variables), or bump the workflow pin to a release that includes this feature. | | `OpenAI WIF is partially configured: missing …` / `inference.openai in config.yaml is partially configured` | One value is empty in the place you chose (variables, or `config.yaml`). Fill it in — fullsend will not silently fall back to an API key, and it does not mix the two sources. | | `… the job has no GitHub OIDC endpoint` | This is not a GitHub Actions job, or `permissions: id-token: write` is missing from the workflow. On GitLab CI or locally, use an API key. | -| `OpenAI WIF exchange failed: … token endpoint returned 4xx` | The mapping does not match this run. Check the audience first (one character off is enough), then compare the claims from step 2 with the mapping's assertions. Works on `main` but not on a pull request → the `ref`/`workflow_ref` differ. Works in one repository but not another → that repository has no mapping yet. | +| `OpenAI WIF exchange failed: … token endpoint returned 4xx` | The mapping does not match this run. Check the audience first (one character off is enough), then compare the claims from step 1 with the mapping's assertions (route B: with the administrator). Works in one repository but not another → that repository has no mapping yet. | | Exchange succeeded, the model call was refused | The mapping's permissions do not cover the call, or the project cannot use that model. The `scope` in the exchange response shows what was granted. | -| `OpenAI WIF token refused: the service-account mapping grants …` | The mapping grants more than model access. Narrow its permissions to `api.model.request` (step 3); fullsend will not run an agent with a broader token. | +| `OpenAI WIF token refused: the service-account mapping grants …` | The mapping grants more than model access. Narrow its permissions to `api.model.request` (route A: edit the mapping in A3; route B: ask your administrator); fullsend will not run an agent with a broader token. | | `the service-account mapping does not narrow permissions` (warning) | The mapping has no permission restriction, so the token holds whatever the service account holds. Add `api.model.request` on the mapping. | | `OPENAI_API_KEY in the sandbox is not a gateway placeholder` | A real key reached the sandbox environment by some other route (an env file copied into the sandbox, for example). Remove it; the provider is the only supported way in. | | `pi config dir has models.json or an auth.json that is not the runner-seeded openai placeholder` | Something wrote into pi's config directory between iterations. Re-run with `--keep-sandbox` and inspect it. | diff --git a/docs/runtimes/pi.md b/docs/runtimes/pi.md index 3108bcd86..34de38046 100644 --- a/docs/runtimes/pi.md +++ b/docs/runtimes/pi.md @@ -32,7 +32,8 @@ through fullsend's table, and a bare id gets the provider from `FULLSEND_PI_PROV > canonical spec. > **GPT via OpenAI** needs no API key in CI: the runner exchanges the job's GitHub identity for a -> short-lived OpenAI token (set three repository variables — see [OpenAI Workload +> short-lived OpenAI token (give fullsend three identifiers with `fullsend github setup --openai-*` +> or repository variables — see [OpenAI Workload > Identity](../guides/infrastructure/openai-workload-identity.md); GitHub Actions only) and keeps it > in a provider that belongs to this run, refreshed before it expires and removed when the run > ends. Locally, put `OPENAI_API_KEY` in an env file for the runner ([Running agents