Skip to content

docs(#6689): two setup routes and per-repository trust in the OpenAI WIF guide #6777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
waynesun09 wants to merge 1 commit into from
Closed

docs(#6689): two setup routes and per-repository trust in the OpenAI WIF guide #6777

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
199 changes: 137 additions & 62 deletions docs/guides/infrastructure/openai-workload-identity.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,43 +6,62 @@ short-lived token (minutes — it never outlives the GitHub token it came from,
fullsend renews it for as long as the run lasts, and the agent sandbox never sees any of it — it only
ever holds a placeholder that the OpenShell gateway swaps for the real token on the way out.

You set this up once per OpenAI organization and once per repository. It takes one visit to the
OpenAI console and three repository variables. No key is created, downloaded or rotated.
Setting it up is one visit to the OpenAI console — yours, or your IT administrator's — and one
`fullsend github setup` run per repository. No key is created, downloaded or rotated.

> **GitHub Actions only.** The exchange needs the job's OIDC endpoint. For GitLab CI and for runs
> on your own machine, use an API key in the runner environment — see [Run it locally](#run-it-locally).

## Before you start

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Prerequisites heading was removed 📜 Skill insight ✧ Quality

The guide replaces its Before you start section with What you end up with, leaving prerequisite
information unlabeled before step 1. Readers can begin the procedure without a clearly identified
prerequisites section.
Agent Prompt 
## Issue description
The modified guide no longer has a clearly labeled prerequisites section before its procedure.

## Issue Context
The OpenAI project, repository enrollment, and route-specific access requirements should be grouped under `## Prerequisites` before step 1.

## Fix Focus Areas
- docs/guides/infrastructure/openai-workload-identity.md[15-30]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


- [ ] You can manage **Workload Identity Providers** in your OpenAI organization
(Organization Settings → Security). This is an organization-level permission; being an owner
of a project is not enough.
- [ ] You have an OpenAI **project** for the runs, ideally a dedicated one with a budget alert.
- [ ] The repository is enrolled per-repo on a fullsend release that includes this feature
(the release notes for #6689 name it).

## 1. Add the identity provider (once per organization)

In the OpenAI console go to **Organization Settings → Security → Workload Identity Provider** and
add a provider:

| Field | Enter |
|---|---|
| OIDC issuer URL | `https://token.actions.githubusercontent.com` |
| Audience | `fullsend://<your-github-org>` (or any string you choose — you will use it verbatim below) |
| Use uploaded JWKS for token verification | **Off** |

Copy the **identity provider ID** — you need it in step 4.

The audience is not issued by anyone: whatever you type here is what your runs will request,
character for character. Pick it once and reuse it for every repository.

## 2. See what your repository actually claims (once)

OpenAI decides whether to trust a run by comparing claims in the GitHub token with the mapping you
write in step 3. A wrong audience and a wrong claim fail the same way, so look at the real values
before writing the mapping. Add this temporary workflow to the repository, run it, and copy the
printed claims:
## What you end up with

Three identifiers, which you hand to fullsend in [step 4](#4-tell-fullsend-the-three-identifiers).
They are not secrets: on their own they grant nothing.

| Identifier | What it is | Where it comes from |
|---|---|---|
| **Audience** | The string your runs ask GitHub to put in the token's `aud` claim. It must equal what the identity provider was created with, character for character. | Whoever created the provider — you (route A) or your IT admin (route B). It is not issued by anyone and has no required format. |
| **Identity provider ID** | The OpenAI Workload Identity Provider for GitHub Actions in your organization. | The provider's page in the OpenAI console (route A), or your IT admin (route B). |
| **Service account ID** | The service account the mapping grants to your repository, in the project the runs are billed to. | The mapping (route A), or your IT admin (route B). |

You also need an OpenAI **project** to bill the runs to (ideally a dedicated one with a budget
alert), and the repository must be enrolled per-repo on a fullsend release that includes this
feature (one that includes fullsend PR #6695; check the release notes).

## Which route are you on?

Workload Identity Providers and their service-account mappings are an **organization-level**
security setting in OpenAI (Organization Settings → Security). Who can edit them decides your route:

- **Route A — you can manage providers in your OpenAI organization.** Being an owner of a project
is not enough; you need the organization-level permission. You create (or reuse) the provider and
add one mapping per repository yourself. Do [step 1](#1-see-what-your-repository-actually-claims-both-routes),
then [A2](#a2-add-or-reuse-the-identity-provider-route-a) and [A3](#a3-map-the-repository-to-a-service-account-route-a).
- **Route B — the provider is managed centrally.** This is the common shape in a company: an IT
administrator owns Organization Settings and the GitHub Actions provider; you own (or request) a
project. A service account is a non-human API principal inside a project; the administrator can
create it while adding the mapping, so all you need to know is the project's name or ID
(Organization → Projects). You send one request per repository and receive the three
identifiers back. Do [step 1](#1-see-what-your-repository-actually-claims-both-routes), then
[B2](#b2-send-the-request-route-b) and [B3](#b3-record-what-you-get-back-route-b).

Either way, steps 4 and onwards are the same.

> **The rule that holds in both routes: trust is per repository.** A mapping asserts
> `repository == <your-github-org>/<repo>` (and `ref == refs/heads/main`) for each company-owned
> repository you enrol — one mapping per repository (a mapping has no list form: its assertions are
> exact values, all of which must match). Never a pattern over the organization
> (`repository_owner`, a name prefix, a derived attribute): a GitHub organization often contains
> repositories the company does not own, and a pattern would let every one of them obtain your
> token. Adding a repository later means adding its assertion; that is the gate, by design.

## 1. See what your repository actually claims (both routes)

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

6. Troubleshooting uses obsolete steps 🐞 Bug ⚙ Maintainability

The route split renames the claims check from step 2 to step 1 and the mapping step from step 3 to
A3/B2, but troubleshooting still tells readers to compare claims from “step 2” and narrow
permissions in “step 3.” Those instructions now point to nonexistent sections and can send users to
the wrong setup action.
Agent Prompt 
## Issue description
The new route-specific numbering leaves two troubleshooting entries referring to the removed step 2 and step 3 labels.

## Issue Context
Point the claims comparison to step 1. Point permission narrowing to A3 for route A or to the administrator request in B2 for route B; links are preferable to bare step numbers.

## Fix Focus Areas
- docs/guides/infrastructure/openai-workload-identity.md[319-329]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


OpenAI decides whether to trust a run by comparing claims in the GitHub token with the mapping's
assertions. A wrong audience and a wrong claim fail the same way, so look at the real values before
writing the mapping (route A) or before asking for it (route B). Add this temporary workflow to
the repository, run it, and copy the printed claims. Put the provider's audience in `AUD` if you
Comment on lines +57 to +60

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

3. Step 1 explains before acting 📜 Skill insight ✧ Quality

Step 1 begins with diagnostic explanation before telling the reader to add and run the temporary
workflow. The required action should lead the step, with the explanation following it.
Agent Prompt 
## Issue description
Step 1 explains claim matching before stating the action the reader must perform.

## Issue Context
Start the step with the action to add and run the temporary workflow, then explain why inspecting the claims is necessary and how to set `AUD`.

## Fix Focus Areas
- docs/guides/infrastructure/openai-workload-identity.md[55-62]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

already know it; if you do not yet, any string works for this check — `repository` and `ref` are
what you are after.

```yaml
name: openai-wif-check
Expand All @@ -55,7 +74,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- env:
AUD: fullsend://<your-github-org>
AUD: <the provider's audience>
run: |
TOK=$(curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=$(jq -rn --arg a "$AUD" '$a|@uri')" | jq -r .value)
Expand All @@ -70,7 +89,7 @@ You will see something like:
```json
{
"iss": "https://token.actions.githubusercontent.com",
"aud": "fullsend://<your-github-org>",
"aud": "<the provider's audience>",
"repository": "<your-github-org>/<repo>",
"ref": "refs/heads/main",
"workflow_ref": "<your-github-org>/<repo>/.github/workflows/openai-wif-check.yml@refs/heads/main",
Expand All @@ -81,36 +100,44 @@ You will see something like:
For the real agent runs, `repository` and `ref` are the same as above; `workflow_ref` names whichever
of the fullsend workflows installed in your repository started the run (`code.yml`, `triage.yml`,
`review.yml`, `fix.yml`, `prioritize.yml`, `retro.yml` or `dispatch.yml` under `.github/workflows/`),
and `job_workflow_ref` the pinned fullsend reusable workflow it calls. Delete the check workflow when
you are done.
and `job_workflow_ref` the pinned fullsend reusable workflow it calls — which is why a mapping asserts
`repository` and `ref`, not `workflow_ref`. Delete the check workflow when you are done.

## 3. Map the repository to a service account (once per repository)
## A2. Add or reuse the identity provider (route A)

Open the provider you created and add a **service account mapping**:
If your organization already has a provider for GitHub Actions, reuse it: open it, note its
**audience** and copy its **identity provider ID**, and go to A3. Otherwise, in **Organization
Settings → Security → Workload Identity Provider**, add one:
Comment on lines +106 to +108

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Route procedures use prose 📜 Skill insight ✧ Quality

The new route-specific procedures describe sequences of actions in prose paragraphs rather than
ordered lists. This makes the operational sequence less explicit and violates the required guide
procedure format.
Agent Prompt 
## Issue description
The new A2/A3 and B2/B3 procedures express multiple actions as prose instead of numbered steps.

## Issue Context
Preserve the route headings, but format each action sequence as an ordered list, including provider reuse/creation, mapping creation, administrator request, and verification.

## Fix Focus Areas
- docs/guides/infrastructure/openai-workload-identity.md[104-138]
- docs/guides/infrastructure/openai-workload-identity.md[150-191]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


| Field | Enter |
|---|---|
| Claim assertions | `iss` = `https://token.actions.githubusercontent.com` · `aud` = your audience · `repository` = `<your-github-org>/<repo>` · `ref` = `refs/heads/main` |
| OIDC issuer URL | `https://token.actions.githubusercontent.com` |
| Audience | Any string you choose, for example `fullsend://<your-github-org>` — your runs will request it verbatim |
| Use uploaded JWKS for token verification | **Off** |

Copy the **identity provider ID**. One provider serves every repository; OpenAI allows 50 providers
per organization and 50 mappings per provider.

@fullsend-ai-review fullsend-ai-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] removed guidance

Section A3 (route A) no longer explicitly warns against asserting workflow_ref in the 'Two things not to do' list. The old section 3 had a clear prohibition explaining that fullsend runs agents from seven workflow files, so a single workflow_ref value would exclude the others. In the new structure, this reasoning appears in step 1's explanation and in the B2 request template, but route A users creating mappings themselves lose the direct warning at the point where they fill in assertions.

Suggested fix: Add a brief 'Do not assert workflow_ref' note to the A3 section's 'Two things not to do' list, expanding it to three items.


## A3. Map the repository to a service account (route A)

Open the provider and add a **service account mapping**:

| Field | Enter |
|---|---|
| Claim assertions | `iss` = `https://token.actions.githubusercontent.com` · `aud` = the provider's audience · `repository` = `<your-github-org>/<repo>` · `ref` = `refs/heads/main` |
| Project | the project the runs should be billed to |
| Service account | create a new one, for example `fullsend-<repo>-ci` |
| Permissions | `api.model.request` (fullsend also accepts `api.model.read`; anything broader is refused at run time) |

Every assertion must match what step 2 printed character for character; a one-letter difference
fails every exchange. Do not assert `workflow_ref`: fullsend runs agents from seven workflow files
(see step 2), so a single `workflow_ref` value would exclude all but one of them. If you want that
narrowing anyway, create one mapping per workflow file.

Copy the **service account ID** — you need it in step 4.
Every assertion must match what step 1 printed character for character; a one-letter difference
fails every exchange. Copy the **service account ID**.

Two things not to do:

- **Do not create an API key** for the service account. The mapping is the credential; a key would
put a long-lived secret back into the picture.
- **Do not assert `repository_owner` instead of `repository`.** That would trust every repository
in your GitHub organization, including ones anybody with repository-create rights can add.

OpenAI allows 50 mappings per provider and 50 providers per organization, and mappings are created
in the console only. Treat enrolment as a deliberate step per repository.
- **Do not assert `repository_owner`, a prefix or any pattern instead of `repository`** — see the
per-repository rule above.

**Which runs this mapping trusts.** `ref` = `refs/heads/main` covers every fullsend agent workflow
as installed on `main`, and GitHub mints the token for the repository the job runs in — so the
Expand All @@ -120,19 +147,66 @@ dispatch authorization (only events from people with write access trigger agents
If you want the mapping itself to exclude some triggers, add a claim the untrusted runs cannot
carry — for example a GitHub `environment` that the agent jobs reference and that protects itself
with required reviewers — and assert `environment == "<name>"` here. A `workflow_dispatch` of a
fullsend workflow on a feature branch will not match `refs/heads/main`; that is expected. In the
org-wide installation mode the runs happen in the organization's central `.fullsend` repository,
so assert that repository instead.
Comment on lines -123 to -125

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

4. Adr 0044 reference omitted 📘 Rule violation § Compliance

The PR removes documentation for deprecated org-wide installation mode, but the PR description does
not reference ADR 0044. The checklist explicitly requires that reference even for removals of
per-org mode content.

fullsend workflow on a feature branch will not match `refs/heads/main`; that is expected.

## B2. Send the request (route B)

You cannot see the provider, so ask the administrator who owns it. Send one request per
repository (or one listing several — the administrator still creates one mapping per repository),
and include the claims from step 1 so the assertions are copied, not retyped. A template:

```text
Subject: OpenAI Workload Identity mapping for GitHub repository <org>/<repo>

Please add a service-account mapping on the organization's GitHub Actions identity provider
(issuer https://token.actions.githubusercontent.com) with these claim assertions, exactly:

iss = https://token.actions.githubusercontent.com
aud = <the provider's audience>
repository = <org>/<repo>
ref = refs/heads/main

Target project: <project name / id> (the project these runs are billed to)
Service account: create a new one named fullsend-<repo>-ci in that project
(or map the existing service account <id>)
Permissions on the mapping: api.model.request only — nothing broader, and please do not
create an API key for the service account; the mapping is the credential.

Please do not add a workflow_ref or sub assertion: these runs start from seven different
workflow files in the repository, so a single value would exclude the others.

One mapping per repository, please (a mapping matches exact values only) — not a wildcard
or a pattern over the organization, since not every repository in it is ours.

Please send back: the identity provider ID, the provider's audience string, and the
service account ID.
```

If the administrator's standard mapping grants more than model access, ask for it to be narrowed:
fullsend refuses a token whose permissions exceed `api.model.request`/`api.model.read`, and only
warns when the mapping does not narrow at all. If you want the mapping to exclude some triggers
(a GitHub `environment` with required reviewers, for example), the reasoning is under
[A3](#a3-map-the-repository-to-a-service-account-route-a) — ask for the extra assertion in the
same request.

## B3. Record what you get back (route B)

The reply gives you the three identifiers from [What you end up with](#what-you-end-up-with).
Re-run the step-1 workflow with the real audience in `AUD` and confirm `aud` and `repository` print
as expected — that is the whole verification you can do from your side before the first run. If the
first run's exchange still returns 4xx, the assertions and the claims differ somewhere (compare them
character for character with the administrator); a `repository` that is not in any mapping is the
usual cause when a second repository is enrolled.

## 4. Tell fullsend the three identifiers

Re-run the setup command you enrolled the repository with, adding the three values:

```bash
fullsend github setup <your-github-org>/<repo> \
--openai-audience "fullsend://<your-github-org>" \
--openai-identity-provider-id "<identity provider ID from step 1>" \
--openai-service-account-id "<service account ID from step 3>"
--openai-audience "<the provider's audience>" \
--openai-identity-provider-id "<identity provider ID>" \
--openai-service-account-id "<service account ID>"
```

It writes them into the repository's `.fullsend/config.yaml`, the same way it records the Vertex
Expand All @@ -141,18 +215,19 @@ project and provider:
```yaml
inference:
openai:
audience: fullsend://<your-github-org>
audience: <the provider's audience>
identity_provider_id: <identity provider ID>
service_account_id: <service account ID>
```

Commit that change (setup opens a pull request for it unless you pass `--direct`). A base
configuration (`config.base.yaml`, or a vendor preset) can carry the block for many repositories,
and a repository can restate any one of the three.
and a repository can restate any one of the three — with a centrally managed provider, the audience
and the provider ID are typically the same for every repository and only the service account differs.

**Is it safe to commit these?** Yes. They are identifiers, not secrets: on their own they grant
nothing. OpenAI issues a token only to a caller presenting a GitHub OIDC token whose claims match
the mapping from step 3, and only your repository's `main` workflow can obtain one. fullsend reads
a mapping, and only your repository's `main` workflow can obtain one. fullsend reads

@qodo-code-review qodo-code-review Bot Aug 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

5. Mapping trusts every main workflow 🐞 Bug ⛨ Security

The recommended mapping constrains only repository and ref, so any job in that repository
running from main with id-token: write can obtain a matching GitHub token—not only the implied
fullsend workflow. This overstates the mapping's security boundary and can cause administrators to
overlook other OIDC-enabled workflows in the repository.
Agent Prompt 
## Issue description
The guide says only the repository's `main` workflow can obtain a token, but the documented mapping does not assert a workflow identity. Any workflow job in that repository on `main` with `id-token: write` can request matching OIDC claims.

## Issue Context
Clarify that the mapping establishes repository-and-ref isolation, while workflow-level restriction requires an additional claim or another control. Keep the explanation consistent with the earlier discussion of `workflow_ref` and dispatch authorization.

## Fix Focus Areas
- docs/guides/infrastructure/openai-workload-identity.md[220-225]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

`.fullsend/config.yaml` from the base branch for pull-request events, so a pull request cannot
change them for the run that reviews it. The worst thing someone with write access could do is
point the block at their own OpenAI organization — and pay for your runs. fullsend prints the three
Expand Down Expand Up @@ -256,9 +331,9 @@ agent starts and names the rule.
| `no OpenAI credential: set FULLSEND_OPENAI_AUDIENCE, …` | Run step 4 (or add the three variables), or bump the workflow pin to a release that includes this feature. |
| `OpenAI WIF is partially configured: missing …` / `inference.openai in config.yaml is partially configured` | One value is empty in the place you chose (variables, or `config.yaml`). Fill it in — fullsend will not silently fall back to an API key, and it does not mix the two sources. |
| `… the job has no GitHub OIDC endpoint` | This is not a GitHub Actions job, or `permissions: id-token: write` is missing from the workflow. On GitLab CI or locally, use an API key. |
| `OpenAI WIF exchange failed: … token endpoint returned 4xx` | The mapping does not match this run. Check the audience first (one character off is enough), then compare the claims from step 2 with the mapping's assertions. Works on `main` but not on a pull request → the `ref`/`workflow_ref` differ. Works in one repository but not another → that repository has no mapping yet. |
| `OpenAI WIF exchange failed: … token endpoint returned 4xx` | The mapping does not match this run. Check the audience first (one character off is enough), then compare the claims from step 1 with the mapping's assertions (route B: with the administrator). Works in one repository but not another → that repository has no mapping yet. |
| Exchange succeeded, the model call was refused | The mapping's permissions do not cover the call, or the project cannot use that model. The `scope` in the exchange response shows what was granted. |
| `OpenAI WIF token refused: the service-account mapping grants …` | The mapping grants more than model access. Narrow its permissions to `api.model.request` (step 3); fullsend will not run an agent with a broader token. |
| `OpenAI WIF token refused: the service-account mapping grants …` | The mapping grants more than model access. Narrow its permissions to `api.model.request` (route A: edit the mapping in A3; route B: ask your administrator); fullsend will not run an agent with a broader token. |
| `the service-account mapping does not narrow permissions` (warning) | The mapping has no permission restriction, so the token holds whatever the service account holds. Add `api.model.request` on the mapping. |
| `OPENAI_API_KEY in the sandbox is not a gateway placeholder` | A real key reached the sandbox environment by some other route (an env file copied into the sandbox, for example). Remove it; the provider is the only supported way in. |
| `pi config dir has models.json or an auth.json that is not the runner-seeded openai placeholder` | Something wrote into pi's config directory between iterations. Re-run with `--keep-sandbox` and inspect it. |
Expand Down
3 changes: 2 additions & 1 deletion docs/runtimes/pi.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ through fullsend's table, and a bare id gets the provider from `FULLSEND_PI_PROV
> canonical spec.

> **GPT via OpenAI** needs no API key in CI: the runner exchanges the job's GitHub identity for a
> short-lived OpenAI token (set three repository variables — see [OpenAI Workload
> short-lived OpenAI token (give fullsend three identifiers with `fullsend github setup --openai-*`
> or repository variables — see [OpenAI Workload
> Identity](../guides/infrastructure/openai-workload-identity.md); GitHub Actions only) and keeps it
> in a provider that belongs to this run, refreshed before it expires and removed when the run
> ends. Locally, put `OPENAI_API_KEY` in an env file for the runner ([Running agents
Expand Down
Loading