diff --git a/.agents/skills/fleet-onboarding/SKILL.md b/.agents/skills/fleet-onboarding/SKILL.md index b3317c0..e6930d3 100644 --- a/.agents/skills/fleet-onboarding/SKILL.md +++ b/.agents/skills/fleet-onboarding/SKILL.md @@ -37,10 +37,17 @@ registry) is worse than one that never started. | `git` | `command -v git` | install git | | `gh` | `command -v gh` | install the GitHub CLI: | | `gh` authenticated | `gh auth status` | `gh auth login` | +| `glab`, only if this fleet works on GitLab | `command -v glab` | install the GitLab CLI: | +| `glab` authenticated | `glab auth status` | `glab auth login` (`GITLAB_HOST` for a self-hosted instance) | | `jq` | `command -v jq` | install jq (`brew install jq`, `apt install jq`, …) | | `thurbox-cli` | `command -v thurbox-cli` | install thurbox: | | thurbox ≥ floor | compare against `min_thurbox_version` in `extension.toml.in` | `thurbox-cli` is too old; upgrade to the floor or newer | +`gh` is required even on a fleet whose work is entirely on GitLab: it is what +builds the repo map from `registry/owners.txt`, which is a list of GITHUB +owners. `glab` is what fleet asks about a GitLab merge request, and nothing +here needs it until a task's repository lives there. + **Read the version floor from the manifest, never from memory.** It is one number with one owner, and `extension.toml.in` records why it sits there: diff --git a/.agents/skills/fleet-pane/SKILL.md b/.agents/skills/fleet-pane/SKILL.md index 8c55943..0e05b37 100644 --- a/.agents/skills/fleet-pane/SKILL.md +++ b/.agents/skills/fleet-pane/SKILL.md @@ -83,7 +83,7 @@ row** — it replaces it rather than sitting above it. It says what the task was told to produce (`publish.method` — `no-mistakes`, `pr` or `push`), which pull request or commit that turned out to be, what fleet last saw when it looked at it, and how long ago it looked. The whole row is the link: Ctrl+Click it and the -terminal opens the pull request. Every word on it comes off `task.yaml`'s +terminal opens the change request. Every word on it comes off `task.yaml`'s `publish` block, written by `collect`, `shepherd` and `reap` — the commands that do the looking — so the pane calls no `gh` and says nothing `queue.sh show` would not print in the same word. Colour carries the verdict, and **`green` is diff --git a/.agents/skills/fleet-queue/SKILL.md b/.agents/skills/fleet-queue/SKILL.md index f705c83..0ee1363 100644 --- a/.agents/skills/fleet-queue/SKILL.md +++ b/.agents/skills/fleet-queue/SKILL.md @@ -117,9 +117,10 @@ rule to remember: | POSIX hosts only | `add` | a host with a non-`tmux` `multiplexer` is how `hosts.toml` spells a Windows host, and is refused by name. Every remote command fleet runs is POSIX shell | | session sharing must be on | `add` | `share_sessions = false` switches off the delegation that lets `session capture` see that pane, so the trust dialog could not be answered and the worker would stall unread | -**Credentials are never moved.** The host needs its OWN GitHub credentials to -clone, fetch and push; yours are not inherited and nothing sends them. Probe 2 -below asks whether the host has any and refuses the dispatch when it does not. +**Credentials are never moved.** The host needs its OWN credentials for the forge +that repository lives on — GitHub or GitLab — to clone, fetch and push; yours are +not inherited and nothing sends them. The `forge` probe below asks whether the +host has any and refuses the dispatch when it does not. Forwarding your SSH agent also fixes it and forwards every key that agent holds — your call to make on that machine, not something a dispatch makes for you. @@ -316,10 +317,16 @@ the host and re-running `dispatch` sends it: ```text reachable it answers ssh, and answers as a POSIX shell - forge it has GitHub credentials of its own — an ssh key, or a gh login repo --repo is a git checkout at that path ON THAT MACHINE + forge it has credentials of its own for the forge THAT repo's `origin` + names — an ssh key, or a `gh` / `glab` login ``` +The repo is asked about before its forge because which forge to prove a +credential against is a fact about that checkout's `origin`: a GitLab repository +needs a GitLab credential, and a probe that named github.com flatly would pass a +host that then fails at its first `git push`. + The report names the probe that failed. A remote worker that starts and then fails at its first `git` call looks exactly like an agent bug and is not one. @@ -474,13 +481,13 @@ Three answers, and the third is not the second: | it is not, or not from this branch | **leaves the task OPEN** and says so, loudly | | could not run | closes the task, and says the check could not run | -"Could not run" is the forge CLI absent, no network, a pull request it cannot read, or a +"Could not run" is the forge CLI absent, no network, a change request it cannot read, or a base branch this machine cannot see. That must never read as a pass or a fail — CI and an offline laptop both still have to collect. `queue.sh show ` prints the method and the verdict, so both survive the scrollback. **The head-branch check is the one a worker cannot write for itself.** Whatever -the body says, "this pull request comes from this task's branch" is a fact of +the body says, "this change request comes from this task's branch" is a fact of the forge — which closes the hole that reading prose never could: a worker pasting somebody else's good pull request. @@ -491,7 +498,7 @@ as it stands, `collect --allow-unverified` closes it and records that you did. ### 5b. `reap` — a session lives until its work lands, and not one turn longer **The gate is the merge, not the conclusion.** For the two methods that end in a -pull request, `outcome: shipped` only means one is OPEN, and the session that +change request, `outcome: shipped` only means one is OPEN, and the session that opened it is the cheap way to fix what review finds — reaping at collect time makes that fix cost a re-spawn: a new worktree, a cold agent, the brief read from nothing. A `push` task has no such gap — `collect` refuses to conclude it @@ -504,16 +511,16 @@ So a task gets a state AFTER `done`: | state | means | its session | |---|---|---| -| `done` | the worker concluded; its pull request is open, or its already-confirmed `push` commit is about to be promoted by this same `collect` run | **kept** — the cheap way to fix what review finds | -| `landed` | the pull request merged, the pushed commit reached the base branch, or there was never an artifact | released | -| `abandoned` | the pull request was closed unmerged | released; the work is NOT on main | +| `done` | the worker concluded; its change request is open, or its already-confirmed `push` commit is about to be promoted by this same `collect` run | **kept** — the cheap way to fix what review finds | +| `landed` | the change request merged, the pushed commit reached the base branch, or there was never an artifact | released | +| `abandoned` | the change request was closed unmerged | released; the work is NOT on main | | `stuck` / `failed` | the worker gave up | **kept** — that session is the evidence, and you decide | `landed` comes from asking the forge, never from a worker claiming it, so it works long after the session is gone. **Blockers clear on `landed`**, not on `done` — a dependent task waits for the code to actually be on `main`, which is the same bug in its other form: a task collected `shipped` once released its -dependents while its pull request sat unreviewed. +dependents while its change request sat unreviewed. ```text topic/01-drop-idle-default landed https://…/pull/999 is merged diff --git a/.agents/skills/thurbox-session/SKILL.md b/.agents/skills/thurbox-session/SKILL.md index 9805a14..6b2c5f1 100644 --- a/.agents/skills/thurbox-session/SKILL.md +++ b/.agents/skills/thurbox-session/SKILL.md @@ -121,18 +121,23 @@ window, and the git worktrees. Only the TUI is local. Three consequences: - **The `BRIEF.md` trick needs the file on the remote.** `Write` puts it on your machine. Copy it over (`scp` / `ssh 'cat >'`) into the remote worktree, or the worker reads nothing. -- **The remote needs its own GitHub credentials** to clone, fetch, and push. - Yours are not inherited. Forwarding your SSH agent fixes it, but forwards - every key the agent holds — decide that before reaching for it. +- **The remote needs its own credentials for that repository's forge** — GitHub + or GitLab — to clone, fetch, and push. Yours are not inherited. Forwarding + your SSH agent fixes it, but forwards every key the agent holds — decide that + before reaching for it. Before spawning remotely, check all three, in this order: ```bash ssh true # reachable? -ssh 'ssh -T git@github.com' # can it reach GitHub? ssh 'ls -d ' # does the repo exist there? +ssh 'ssh -T git@$(...origin's host...)' # can it reach THAT forge? ``` +The third one asks the host named by that checkout's `origin`, not github.com: +a GitLab repository needs a GitLab credential, and GitLab's welcome banner reads +`Welcome to GitLab, @you!` where GitHub's says `successfully authenticated`. + Until all three pass, **spawn locally**. A remote worker will start and then fail at its first `git` call, which looks like an agent bug and is not one. diff --git a/AGENTS.md b/AGENTS.md index a92150f..53600fc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -50,14 +50,20 @@ names every path and the reason for each. - `scripts/lib/forge.py` — the FORGE seam. Everything fleet knows about a change request — a pull request on GitHub, a merge request on GitLab — it asks this module for; `scripts/lib/queue.py` runs no forge CLI itself and - builds no forge URL. GitHub, through `gh`, is the one implementation shipped, - and it is a CONFIGURATION and not an assumption. The file's own header owns - the interface and how to add another. Two things follow: a repository is - identified by HOST plus path (`github.com/Thurbeen/fleet`), because a bare - `owner/repo` names two different repositories once two forges exist; and - `queue-selftest.sh` drives `collect`, the landing check and `shepherd` - through a second, fake forge with no network and no `gh` behind it, which is - what keeps the seam honest rather than merely asserted. + builds no forge URL. TWO implementations ship — GitHub through `gh`, GitLab + through `glab` — and each is a CONFIGURATION and not an assumption: which + hosts one owns comes from that CLI's own variable (`GH_HOST`, `GITLAB_HOST`), + so a self-hosted instance is the ordinary case and not a special one. The + file's own header owns the interface and how to add a third. Two things + follow: a repository is identified by HOST plus path + (`github.com/Thurbeen/fleet`), because a bare `owner/repo` names two + different repositories once two forges exist; and `queue-selftest.sh` drives + `collect`, the landing check and `shepherd` through a second forge with no + network behind it — §13 through a fake one, §14 through the real GitLab + adapter over recorded `glab` output in `scripts/fixtures/glab/` (whose README + says which files are recorded and which are constructed). `gh` on those + sections' PATH is a tripwire, which is what keeps the seam honest rather than + merely asserted. - `orchestration/reconcile/` — the reconciler's runtime state: its supervisor's pid, the heartbeat proving its loop is ticking, its log, the advisory `nudge` flag and the `down` flag. Written by `./scripts/reconcile.sh` and created on @@ -120,9 +126,11 @@ The loop, driven by `./scripts/queue.sh`: (`./scripts/session-trust.sh`), because sending one into that dialog is how every fleet-spawned worker used to break. A task may name a `--host` from thurbox's `hosts.toml` and run on that machine instead; `--repo` is then a - path THERE, three probes run before anything is spawned, and the brief and - the result travel by ssh so that completion stays one model. No host means - no change. + path THERE, three probes run before anything is spawned — reachable, the + repo is there, and it has its own credentials for the forge THAT repo's + `origin` names, which is why a GitLab checkout is not probed against + github.com — and the brief and the result travel by ssh so that completion + stays one model. No host means no change. 5. **Completion is two things you read, never something that interrupts you.** `queue.sh watch` folds `thurbox-cli watch`'s event stream into each task's record and closes nothing; `queue.sh collect` reads the `result.md` the @@ -132,11 +140,11 @@ The loop, driven by `./scripts/queue.sh`: is the half no record can hold: the goal in your words, the decisions, what went wrong, the outcome. Write those into it while you still know them. 6. **Release is a third thing, and it is not manual.** `outcome: shipped` - means a pull request is OPEN, or, for a task whose declared publish method is - `push`, a commit already on the base branch — and that session is kept as the - cheap way to fix what review finds. A task moves to `landed` only when the - FORGE says its artifact merged (immediately, for `push`, since there is no - pull request to wait on), and `queue.sh reap` — which `collect` runs itself — + means a change request is OPEN, or, for a task whose declared publish method + is `push`, a commit already on the base branch — and that session is kept as + the cheap way to fix what review finds. A task moves to `landed` only when + the FORGE says its artifact merged (immediately, for `push`, since there is + nothing open to wait on), and `queue.sh reap` — which `collect` runs itself — deletes the session and its worktree then. It never touches one thurbox says is working or blocked, nor one a worker gave up in: that session is the evidence. `reap --dry-run` says what it would do. Blockers clear on `landed` @@ -146,17 +154,20 @@ The loop, driven by `./scripts/queue.sh`: which still prints how many it is hiding. `stuck` and `failed` are not terminal for that, `list --archived` and `show ` still reach it, and `add` un-archives. -7. **The pull request outlives the task, so `queue.sh shepherd` is a fourth +7. **The change request outlives the task, so `queue.sh shepherd` is a fourth thing, run as reflexively as `collect`** — which names it whenever it closed - a task that left a PR open. It asks the FORGE for every open PR on the repos - the queue's tasks name, not the tasks' recorded artifacts. A PR is linked - back by artifact or head branch; an unlinked one is + a task that left one open. It asks the FORGE for every open change request + on the repos the queue's tasks name, not the tasks' recorded artifacts. One + is linked back by artifact or head branch; an unlinked one is still classified and merged, it just has no session to fix it. It merges only in the repos `AUTO_MERGE_REPOS` names in `scripts/lib/queue.py` — each entry host-qualified, and one that names no forge refused rather than - matched — and only for a PR whose head branch is in that repo, opened by + matched — and only for one whose head branch is in that repo, opened by someone who can push there, carrying a `no-mistakes` attestation for its **current** head. + Squash is the only method fleet merges by, and a forge or a project that + forbids squash — a GitLab project can — is a refusal fleet RECORDS rather + than a merge by some other method. That attestation gate is the one thing the declared publish method moves: a task that was declared `no-mistakes` and carries none gets a fixer, one that was never asked for one is recorded `green` and handed back diff --git a/FLEET.md b/FLEET.md index 0caf236..1c63cf5 100644 --- a/FLEET.md +++ b/FLEET.md @@ -1,8 +1,10 @@ # FLEET.md — standing context for the control-plane session You are the **Mission Control** session: the long-lived control plane for its -owner's work across GitHub — whichever accounts and orgs are listed in -`registry/owners.txt`. +owner's work across GitHub and GitLab. `registry/owners.txt` is the map you +cover — GitHub accounts and orgs, because repository DISCOVERY still runs on +`gh`; which forge a task's change request lives on is a separate question, +answered per repository by `scripts/lib/forge.py`. The SESSION is called Mission Control, and it wears a mark in front of that: thurbox has no per-session icon field, so the glyph the TUI shows can only live @@ -84,20 +86,21 @@ YAML by hand. Nothing to push — the map is gitignored. and `shepherd` keep its facts current as you run them. Write the goal, decisions and outcome into it in your own words — that half never comes from a record. It is gitignored and not backed up by the repo. -6. **`shepherd`, as reflexively as `collect`.** The pull request outlives the - task, and `collect` names `shepherd` whenever it closed one that left a PR - open. It asks the forge for every open PR on the queue's repos, not just - recorded artifacts, dispatches a fixer for one that conflicts, fails a - check, was reviewed with changes requested, or was declared `no-mistakes` +6. **`shepherd`, as reflexively as `collect`.** The change request outlives the + task, and `collect` names `shepherd` whenever it closed one that left one + open. It asks the forge for every open change request on the queue's repos, + not just recorded artifacts, dispatches a fixer for one that conflicts, fails + a check, was reviewed with changes requested, or was declared `no-mistakes` and carries no attestation for its current head, and squash-merges one that clears every gate in the repos `AUTO_MERGE_REPOS` allows — entries there - name their forge (`github.com/Thurbeen/fleet`), because a bare `owner/repo` + name their forge (`github.com/Thurbeen/fleet`, + `gitlab.example.com/acme/group/widgets`), because a bare `owner/repo` is two different repositories once two forges are configured. It writes down what it saw either way, so a task's record says `checks-running` or `unattested` and not just `shipped`. -7. Review the PRs; the operator merges every one `shepherd` did not. Sessions +7. Review them; the operator merges every one `shepherd` did not. Sessions release themselves once their artifact lands on the base branch — a merged - pull request, or, for a task that published by pushing directly, the + change request, or, for a task that published by pushing directly, the commit itself — `collect` reaps them, `queue.sh reap --dry-run` shows what it would do — see `AGENTS.md`. 8. **`refuel` a worker that hit its agent's token limit and never reported — diff --git a/README.md b/README.md index 59a33a8..9a657cd 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,10 @@ # fleet -A **control plane** for your work across GitHub. You hand it a goal; it splits -the goal into tasks, runs an AI agent session on each one in a real repo, and -gives you back pull requests to review. +A **control plane** for your work across GitHub and GitLab. You hand it a goal; +it splits the goal into tasks, runs an AI agent session on each one in a real +repo, and gives you back change requests — pull requests, merge requests — to +review. It is one repo holding two things: a **map** of your projects, and the **orchestration** of the agent sessions run against them, using @@ -63,7 +64,9 @@ before it writes a thing. Run it twice and it converges. It hands you a guarded block to add yourself: the pane's slot in your thurbox `layout.lua`. -Requires `gh` (authenticated), `jq`, and `thurbox-cli` **2.19.0 or newer**. +Requires `jq` and `thurbox-cli` **2.19.0 or newer**, plus the CLI for each forge +you work on: `gh` for GitHub, `glab` for GitLab, each authenticated. `gh` is not +optional even on a GitLab-only fleet — it is what builds the repo map. That done, open the Mission Control session in thurbox and give it a goal. @@ -75,7 +78,7 @@ it. Onboarding installs it and `F3` opens and closes it. It **displays and does not control** — `./scripts/queue.sh` stays the only thing that writes. It reads the same four files per task and no fifth: the plan, the progress, the -outcome, and the pull request. A column is narrow, so it draws only what you +outcome, and the change request. A column is narrow, so it draws only what you would act on from a glance and leaves the rest to `./scripts/queue.sh show`. ![The queue pane in a thurbox column beside the session list: the account's fuel diff --git a/extension.toml.in b/extension.toml.in index 929b0d1..a81bea4 100644 --- a/extension.toml.in +++ b/extension.toml.in @@ -244,7 +244,7 @@ # nothing resolves those names, that half cost none of the argument above. # Existing workers keep the name they were spawned with: there is no rename # verb for them either, and none of the sequences above is worth running on a -# session that will be reaped when its pull request merges. +# session that will be reaped when its change request merges. name = "fleet" description = "Control-plane session: the repo map and thurbox orchestration" diff --git a/interface/fleet_queue.lua b/interface/fleet_queue.lua index 07e2448..f2a548e 100644 --- a/interface/fleet_queue.lua +++ b/interface/fleet_queue.lua @@ -1174,12 +1174,18 @@ end --- The artifact, in the fewest columns that still identify it. --- ---- This is what the row absorbed the artifact ROW to say. A pull request is its ---- number and a `push` task's commit is a short sha — both are what a reader ---- would have read off the end of the URL anyway — and anything else keeps the ---- URL with its scheme off, because a shape this does not recognise is one it ---- must not pretend to summarise. The whole row carries the link either way, so ---- what is drawn here is a label for a click target rather than the target. +--- This is what the row absorbed the artifact ROW to say. A change request is +--- its number and a `push` task's commit is a short sha — both are what a +--- reader would have read off the end of the URL anyway — and anything else +--- keeps the URL with its scheme off, because a shape this does not recognise +--- is one it must not pretend to summarise. The whole row carries the link +--- either way, so what is drawn here is a label for a click target rather than +--- the target. +--- +--- `#47` and `!52` are each their own forge's notation, kept rather than +--- flattened: `#` is a pull request on GitHub and `!` is a merge request on +--- GitLab, and an operator reading a queue that spans both wants to know which +--- one a row points at without opening it. local function artifact_ref(artifact) if artifact == "" then return nil @@ -1188,6 +1194,10 @@ local function artifact_ref(artifact) if number then return "#" .. number end + number = artifact:match("/%-/merge_requests/(%d+)") + if number then + return "!" .. number + end local sha = artifact:match("/commit/(%x%x%x%x%x%x%x+)") if sha then return sha:sub(1, 7) diff --git a/orchestration/queue/POLICY.md b/orchestration/queue/POLICY.md index 4883fca..ba6be93 100644 --- a/orchestration/queue/POLICY.md +++ b/orchestration/queue/POLICY.md @@ -55,11 +55,14 @@ Your brief's **Publish** line names one of three methods, what it must leave behind, and what proves it. It is the authority: where it names a tool, use that tool. Do not switch methods. -`queue.sh collect` then goes and looks for that artifact: the forge for a pull -request, git for a commit on the base branch. A task whose artifact is not -there, or is not from your branch, **is not closed** — the lead sees it at -collect time and sends you back. So verify your own artifact before you report -done. For a `no-mistakes` task that is one command: +`queue.sh collect` then goes and looks for that artifact: the forge for a change +request — a pull request on GitHub, a merge request on GitLab — git for a commit +on the base branch. A task whose artifact is not there, or is not from your +branch, **is not closed** — the lead sees it at collect time and sends you back. +So verify your own artifact before you report done. For a `no-mistakes` task +that is one command, in the CLI your forge has. + +On GitHub: ```sh gh pr view --json headRefOid,body -q \ @@ -69,13 +72,25 @@ gh pr view --json headRefOid,body -q \ + ($attested | if . then .[0:8] else "no attestation found in the body" end)' ``` +On GitLab — the same question, in GitLab's own field names (`sha` for the head, +`description` for the body), and `-R` takes the project's full URL so that a +self-hosted instance is asked and not gitlab.com: + +```sh +glab mr view -R https://// -F json --jq \ + '.sha[0:8] as $head + | ([.description | capture("head_sha\"\\s*:\\s*\"(?[0-9a-f]+)").s] | .[0]) as $attested + | $head + " is the head; the attestation names " + + ($attested | if . then .[0:8] else "no attestation found in the description" end)' +``` + **Those two must be the same commit.** An attestation is a verdict about the code the pipeline saw, so one naming any other commit proves nothing about what would merge, and `collect` holds your task open exactly as it does for a body with no attestation at all. They come apart on their own. The pipeline writes the attestation while it -opens the pull request and can then push its own `no-mistakes: apply CI fixes` +opens the change request and can then push its own `no-mistakes: apply CI fixes` commit on top, which leaves the head one commit ahead of what was attested — this is what happened to #38, #40 and #48. **Run `/no-mistakes --yes` again** and it re-attests the new head; then run the command above once more before @@ -86,13 +101,16 @@ The default for every task here is the frontmatter at the top of this file. ## Do not merge -- **Squash merge only.** It is the only method the remote allows, so the pull - request title becomes the commit on `main`. Write the title accordingly. +- **Squash merge only.** It is the only method fleet merges by, and on this + control plane's own remote it is the only one allowed, so the change + request's title becomes the commit on `main`. Write the title accordingly. + A repo that forbids squash — a GitLab project can, with + `squash_option: never` — is one fleet reports and leaves for you. - **You do not merge.** Opening it is where your work ends. `queue.sh shepherd` may later merge it for you in the repos its `AUTO_MERGE_REPOS` - allowlist names (host-qualified, as in `github.com/owner/repo`), but only - once your pull request clears its gates — never merge it yourself in the - meantime. + allowlist names (host-qualified, as in `github.com/owner/repo` or + `gitlab.example.com/group/project`), but only once it clears its gates — + never merge it yourself in the meantime. ## Reporting back — write a file, do not send mail @@ -110,8 +128,8 @@ A short paragraph: what you actually did, and anything the lead must know. ``` `outcome` is one of those four words and nothing else. `artifact` is whatever -your brief's Publish line says it is — a pull request URL for two of the three -methods, a commit URL for `push`; `not-applicable` and `stuck` usually have +your brief's Publish line says it is — a change request URL for two of the +three methods, a commit URL for `push`; `not-applicable` and `stuck` usually have none, and that is fine. `shipped` is a claim that the artifact exists, so report it without one, or with something of the wrong shape, and the lead's `collect` holds your task open rather than trusting the word alone. diff --git a/orchestration/queue/README.md b/orchestration/queue/README.md index 245c5c7..8da2f94 100644 --- a/orchestration/queue/README.md +++ b/orchestration/queue/README.md @@ -87,7 +87,7 @@ closes anything. **`collect` checks the artifact it is handed.** Each task declares a publish METHOD — `no-mistakes`, `pr` or `push` — naming what it must produce, and -`collect` goes and looks: the forge for a pull request from that task's own +`collect` goes and looks: the forge for a change request from that task's own branch (carrying a `no-mistakes` attestation for its head, for that method), git for a commit that reached the base branch. A task whose artifact is not there is reported and left OPEN, because "use the pipeline" is an instruction @@ -100,8 +100,8 @@ that came of it; the tool itself is `publish.how`, free text fleet renders into the brief and never parses. **`done` is not the end of the record.** `queue.sh reap` — which `collect` -runs for you — asks the forge whether a `done` task's pull request merged and -moves it to `landed` (or, if the pull request closed unmerged, `abandoned`); a -task with no PR artifact goes straight to `landed`. Landing releases the +runs for you — asks the forge whether a `done` task's change request merged and +moves it to `landed` (or, if it closed unmerged, `abandoned`); a task with no +change-request artifact goes straight to `landed`. Landing releases the task's session and worktree and stamps `task.yaml` with a `reaped: {session, how, at}` receipt, because the id it names no longer resolves to anything. diff --git a/scripts/fixtures/glab/README.md b/scripts/fixtures/glab/README.md new file mode 100644 index 0000000..8675c76 --- /dev/null +++ b/scripts/fixtures/glab/README.md @@ -0,0 +1,58 @@ +# Recorded `glab` output + +What `scripts/queue-selftest.sh` §14 drives the GitLab adapter with, so that +the adapter's parsing is tested against **what `glab` actually prints** rather +than against what this repo assumed it prints. Nothing here reaches the +network when the selftest runs; a fake `glab` on `PATH` replays these files. + +Every file below was recorded on **2026-09-10** with **`glab` 1.117.0 +(44790937b)** against **gitlab.com**, unauthenticated, from the public project +`gitlab-org/cli`. The commands are exact and repeatable. + +| File | Command | +| --- | --- | +| `mr-view.json` | `glab mr view 3877 -R https://gitlab.com/gitlab-org/cli -F json` | +| `mr-commits.json` | `glab api "projects/gitlab-org%2Fcli/merge_requests/3875/commits?per_page=100" --hostname gitlab.com` | +| `mr-view-missing.json` | `glab mr view 999999 -R https://gitlab.com/gitlab-org/cli -F json` — stdout | +| `mr-view-missing.stderr` | the same call's stderr | + +`glab mr list -F json` answers with the same objects minus `head_pipeline`, so +it is not recorded separately — the adapter reads only `iid` out of a listing +and asks `mr view` for each one, which is exactly because the pipeline is +missing from the list. + +Each file was kept for a specific reason, and the reason is what makes it worth +its bytes in a control-plane repo: + +- **`mr-view.json` is a FORK's merge request.** `source_project_id` is + 86287219 and `target_project_id` is 34675721, so it is the one shape whose + misreading merges a stranger's code — recorded rather than imagined. + Its `head_pipeline.status` is `failed`, and its `detailed_merge_status` is + `title_regex`, a word absent from the documented set the adapter was written + against. That is why the adapter reads only `mergeable` and `conflict` out + of that field and calls everything else "the forge has not said". +- **`mr-commits.json` is NEWEST FIRST.** Fleet's `Commit` list is oldest + first, so the adapter reverses; this file is the evidence that it must. +- **`mr-view-missing.*` is the error shape**, and it is two streams: the + reason is the JSON on *stdout*, while stderr carries a blank-line-padded + `ERROR` box whose first line is decoration. An adapter that read stderr + first would report the box. + +## What is NOT here + +Two answers the adapter needs are behind authentication, and this recording +had none. They are **constructed inside the selftest and labelled there**, from +the field names in GitLab's REST API documentation — not recorded, and not +presented as recorded: + +- `GET /projects/:id` → `squash_option`, which is what can forbid the only + merge method fleet uses. Unauthenticated it comes back `null`. +- `GET /projects/:id/members/all` → `access_level`. Unauthenticated it is + `401 Unauthorized`. + +## Refreshing + +Re-run the commands above. `3877` and `3875` are ordinary merge requests on a +public project and will eventually be closed; when they are, pick any open +merge request with a fork source and any with several commits, and update the +numbers in this table. diff --git a/scripts/fixtures/glab/mr-commits.json b/scripts/fixtures/glab/mr-commits.json new file mode 100644 index 0000000..cd299d6 --- /dev/null +++ b/scripts/fixtures/glab/mr-commits.json @@ -0,0 +1 @@ +[{"id":"f738684fe382b370101b6b6d40fa32070518e0af","short_id":"f738684f","created_at":"2026-09-09T11:57:36.000-05:00","parent_ids":["3f93b6d8e59791f46a6e893d66de0ce941003296"],"title":"refactor: fix gocritic findings and delete comments that restate the code","message":"refactor: fix gocritic findings and delete comments that restate the code\n\nClears the 171 gocritic findings that enabling the linter surfaced, and the\n45 comments the overlap gate rejects, so both gates pass on a clean tree.\n\nReal defects fixed, as opposed to style:\n\n- MCP tools could execute the wrong command. iterCommands yielded a path\n slice that registerTools captures in a long-lived handler closure, while\n sibling recursion appended into the same backing array. Past depth two the\n slice has spare capacity, so a sibling overwrote a registered tool's path.\n- attestation/verify leaked a file descriptor on both error paths, because\n the close was deferred after them, and dereferenced a nil file when\n CreateTemp failed.\n- job/artifact held every extracted file's handles open until the whole\n archive finished; the entry write is now its own function.\n- artifactregistry/login could panic on a credentials line that matched with\n fewer submatches than it indexed.\n- gen-docs, git.RunClone, alias/expand and two config paths appended to a\n slice they did not own.\n\nBehaviour is unchanged otherwise. Verified by reverting every test file and\nrunning the suite against the production changes alone: 5080 tests pass\neither way, so no test was adjusted to accommodate a change.\n","author_name":"Kai Armstrong","author_email":"karmstrong@gitlab.com","authored_date":"2026-09-09T11:57:36.000-05:00","committer_name":"Kai Armstrong","committer_email":"karmstrong@gitlab.com","committed_date":"2026-09-09T11:57:36.000-05:00","trailers":{},"extended_trailers":{},"web_url":"https://gitlab.com/gitlab-org/cli/-/commit/f738684fe382b370101b6b6d40fa32070518e0af"},{"id":"3f93b6d8e59791f46a6e893d66de0ce941003296","short_id":"3f93b6d8","created_at":"2026-09-09T10:11:02.000-05:00","parent_ids":["5d235db58cc86cab543be0dbe66203d8b5f617fd"],"title":"ci: run the comment lint scripts in CI","message":"ci: run the comment lint scripts in CI\n\nLefthook already runs both, but it can be skipped with LEFTHOOK=0, so CI is\nthe enforcement point. The overlap check gates at full token coverage; the\nratio report is advisory and cannot fail the job.\n","author_name":"Kai Armstrong","author_email":"karmstrong@gitlab.com","authored_date":"2026-09-09T10:11:02.000-05:00","committer_name":"Kai Armstrong","committer_email":"karmstrong@gitlab.com","committed_date":"2026-09-09T10:11:02.000-05:00","trailers":{},"extended_trailers":{},"web_url":"https://gitlab.com/gitlab-org/cli/-/commit/3f93b6d8e59791f46a6e893d66de0ce941003296"},{"id":"5d235db58cc86cab543be0dbe66203d8b5f617fd","short_id":"5d235db5","created_at":"2026-09-09T10:06:00.000-05:00","parent_ids":["30dc654fd9c333a2803899b3d039fcf2c404b20f"],"title":"chore(lint): enable gocritic diagnostic and style checks","message":"chore(lint): enable gocritic diagnostic and style checks\n\nAdds 171 findings on the current tree, cleaned up in the following commit.\nFour style checks stay off because they conflict with decisions already\nencoded here: two contradict nonamedreturns and the disabled govet shadow\ncheck, one duplicates nolintlint, and one is cosmetic parameter grouping.\n","author_name":"Kai Armstrong","author_email":"karmstrong@gitlab.com","authored_date":"2026-09-09T10:06:00.000-05:00","committer_name":"Kai Armstrong","committer_email":"karmstrong@gitlab.com","committed_date":"2026-09-09T10:06:00.000-05:00","trailers":{},"extended_trailers":{},"web_url":"https://gitlab.com/gitlab-org/cli/-/commit/5d235db58cc86cab543be0dbe66203d8b5f617fd"},{"id":"30dc654fd9c333a2803899b3d039fcf2c404b20f","short_id":"30dc654f","created_at":"2026-09-09T10:03:21.000-05:00","parent_ids":["48313091a20e89a0f56ca88f7d17ed4a483568b8"],"title":"chore(lint): add comment volume and overlap scripts","message":"chore(lint): add comment volume and overlap scripts\n\nTwo dependency-free Go tools for keeping comment noise down, wired into\nlefthook.\n\ncomment-overlap flags comments whose words are already carried by the\nadjacent code, a mechanical proxy for \"document only what is non-obvious\nfrom the code\". Doc comments are excluded by position through go/ast Doc\nfields, so idiomatic godoc is never flagged for repeating its own symbol\nname. Gated in pre-commit at full token coverage, where the comment\ndemonstrably adds nothing; lower thresholds need human judgement and are\nfor reporting only.\n\ncomment-ratio reports how much of a diff is comment rather than code,\nsplit into doc and inline. It judges no single comment, so unlike a\nclassifier it cannot be defeated by rewording. Advisory in pre-push,\nbecause the measured repo distribution has no natural cutoff.\n","author_name":"Kai Armstrong","author_email":"karmstrong@gitlab.com","authored_date":"2026-09-09T10:03:21.000-05:00","committer_name":"Kai Armstrong","committer_email":"karmstrong@gitlab.com","committed_date":"2026-09-09T10:03:21.000-05:00","trailers":{},"extended_trailers":{},"web_url":"https://gitlab.com/gitlab-org/cli/-/commit/30dc654fd9c333a2803899b3d039fcf2c404b20f"}] \ No newline at end of file diff --git a/scripts/fixtures/glab/mr-view-missing.json b/scripts/fixtures/glab/mr-view-missing.json new file mode 100644 index 0000000..6504a9d --- /dev/null +++ b/scripts/fixtures/glab/mr-view-missing.json @@ -0,0 +1 @@ +{"error":{"message":"failed to get merge request 999999: 404 Not Found"}} diff --git a/scripts/fixtures/glab/mr-view-missing.stderr b/scripts/fixtures/glab/mr-view-missing.stderr new file mode 100644 index 0000000..59561ec --- /dev/null +++ b/scripts/fixtures/glab/mr-view-missing.stderr @@ -0,0 +1,5 @@ + + ERROR + + Failed to get merge request 999999: 404 Not Found. + diff --git a/scripts/fixtures/glab/mr-view.json b/scripts/fixtures/glab/mr-view.json new file mode 100644 index 0000000..76024d5 --- /dev/null +++ b/scripts/fixtures/glab/mr-view.json @@ -0,0 +1 @@ +{"id":530209448,"iid":3877,"target_branch":"main","source_branch":"patch-1","project_id":34675721,"title":"Fix setup.py for setuptools 82+ (removed setuptools.extern)","state":"opened","imported":false,"imported_from":"none","created_at":"2026-09-10T00:48:30.252Z","updated_at":"2026-09-10T00:48:36.499Z","upvotes":0,"downvotes":0,"author":{"id":40777488,"username":"yenrique","name":"Yvonne Locke","state":"active","created_at":null,"avatar_url":"https://secure.gravatar.com/avatar/30323efe48705a6dfab05263db5626e676d3395e4b60916c4be3a71e8583dcaf?s=80\u0026d=identicon","web_url":"https://gitlab.com/yenrique"},"assignee":{"id":40777488,"username":"yenrique","name":"Yvonne Locke","state":"active","created_at":null,"avatar_url":"https://secure.gravatar.com/avatar/30323efe48705a6dfab05263db5626e676d3395e4b60916c4be3a71e8583dcaf?s=80\u0026d=identicon","web_url":"https://gitlab.com/yenrique"},"assignees":[{"id":40777488,"username":"yenrique","name":"Yvonne Locke","state":"active","created_at":null,"avatar_url":"https://secure.gravatar.com/avatar/30323efe48705a6dfab05263db5626e676d3395e4b60916c4be3a71e8583dcaf?s=80\u0026d=identicon","web_url":"https://gitlab.com/yenrique"}],"reviewers":[],"source_project_id":86287219,"target_project_id":34675721,"labels":["1st contribution","Community contribution","workflow::in dev"],"label_details":null,"description":"## Drive\\\\Cursor Work\\\\mdf-toolkit-mr\\\\MR_DESCRIPTION.md\n\nThe YOPP observation siteMIP (model inter-comparison program) was convened to facilitate observational super sites and NWP modelers making their high resolution in-space-and-time data available in a cross-compatible standard formatted way, specifically for process evaluation. The project has grown in somewhat in scope to be a more generic data formatting toolkit. It is a python project lead by Michael Gallagher (CIRES/NOAA Research Scientst), co-maintained with significant contributions by Jareth Holt (MISU), designed to simplify processing data into this standardized format, for scientists working with site observationsand model output files. The idea being that the toolkit provides an interface between your data and the details of the netcdf output format, saving you time, energy, and effort. As a user of the toolkit writing a simple basic well-formatted fully documented netcdf data file could be as simple as seven lines of code:\n\n`from mdftoolkit.MDF_toolkit import MDF_toolkit`\\\n\\\n`import pandas as pd`\\\n`MDF = MDF_toolkit(supersite_name='DEMO', multi_site=False)`\\\n`header = ['albs','rsu','rsd','ts','hur','ta','tdp','ps','rlu']`\\\n`ts_data = pd.read_csv('my_data.csv', sep=',', names=header)`\\\n\\\n`MDF.add_data_timeseries(ts_data, cadence=\"time01\")`\\\n`MDF.write_files(output_dir='./out/')`\\\n\\\nThe MDF toolkit is a python library/API that provides the containers and functions that make writing merged data files easier for data maintainers for complex sites and model files. This includes codified definitions and attributes of the M(O/M)DF specification as well as methods and functions to write your data to disk in this specification. The essential code can be found in `src/mdftoolkit/MDF_toolkit.py` and the data definitions provided by the venerable H-K data documentation table can be found in `src/mdftoolkit/HKTable/`\n\n`src/mdftoolkit/examples/create_example_MDF.py` provides a skeleton example on which you can base your site data processing code. It provides a (semi) detailed example of how to utilize the functionality provided by the MDF_toolkit. This code generates some random fake data files and inserts these data into an MDF using the MDF_toolkit.\n\n## **Installing and getting started:**\n\nThe long term plan is to submit this package to the PyPI repository so using it is as simple as \"pip install mdftoolkit\". For now, please read the more detailed instructions in INSTALL.md\n\n### **Where to find help and collaborate:**\n\nA slack channel was created for general discussion of the toolkit as a place to collaborate together. This is the ideal place to work on the nitty gritty details. You can join the slack channel at:\n\nhttp://mdfmakers.slack.com/\n\n## **Code description:**\n\nA general description of the steps to use the toolkit:\n\n1. ingest \"raw\" data sources from observatory via pandas/xarray\n2. QC/curate your data as appropriate, ideally inside MDF toolkit for documentation/provenance purposes\n3. label the data appropriately via provided toolkit functions so it knows how to read/write your data\n4. ask the toolkit to write these data objects to disk !\n\nThe difficulty of these steps will highly depend on the format of your data. It could be anything from a few lines of code per data source to a few 100 lines of code per data source, largely depending on how much you need to clean and format your data.\n\n### **Required software:**\n\nIf you would like to create an MDF, the following python packages are required:\n\n`python ≥ 3.6\nnetCDF4 ≥ 1.3.0\nnumpy ≥ 1.13.0\npandas ≥ 0.20\n`\n\n## **Linux setup:**\n\n### **Method 1:**\n\nFor Linux, setup is as simple as using your package manager to install the necessary packages\n\nUsing Ubuntu as an example you would run the following commands:\n\n`sudo apt-get install python-numpy python-pandas python-netcdf \n`\n\nOr on Arch:\n\n`pacman -S python python-numpy python-scipy python-netcdf4 \n`\n\n### **Method 2:**\n\nIf you are running on a server and you don't have the permissions to install python packages but you have Pip, you can install these Python packages locally in your home directory. Double check to make sure they aren't installed to root already but if you don't have them, run:\n\n`pip install --user numpy pandas netCDF4\n`\n\nAnd then add the following to your \\~/.bashrc file:\n\n`export PY_USER_BIN=$(python -c 'import site; print(site.USER_BASE + \"/bin\")')\nexport PATH=$PY_USER_BIN:$PATH\n`\n\n## **Windows/OSX setup:**\n\nI'm a \\*NIX guy myself but you most likely want to use [Anaconda](https://docs.anaconda.com/anaconda/user-guide/getting-started/). Using Anaconda, the necessary packages can be installed with the following command:\n\n`conda install -c pandas netcdf4 numpy xarray\n`\n\n## **Want to contribute?**\n\nGreat, I would love your contributions! Check this repository out, make your modification, and send me a pull request here. Or, worst case scenario feel free to open up an \"issue\" here on gitlab with your comments.\n\nIf git is annoying and you would like to contact me instead by e-mail, I'm happy to work something out. Don't hesitate to contact me, [Michael Gallagher](mailto:michael.r.gallagher@noaa.gov).\n\nTo use this code, modify it, and give that mdfication back to me, follow this basic recipe:\n\n`git clone https://gitlab.com/mdf-makers/mdf-toolkit.git # \u003c!-- checkout this repo via https OR --\u003e\ngit clone git@gitlab.com:mdf-makers/mdf-toolkit.git # \u003c!-- checkout this repo via ssh --\u003e\ngit checkout -b supersitename-mdf # \u003c!-- make a new branch for your site --\u003e\n`\n\nThen, make changes inside of your branch and commit them as necessary upstream to the MDF toolkit repo!\n\n## **supersite contacts:**\n\nMDFs are going to be created by teams for each supersite observatory. Here we've collected contact point names for each observatory as a convenient starting point for site specific discussions/questions:\n\n* MOSAiC — michael.r.gallagher@noaa.gov\n\n## **Acknowledgments**\n\nCode contributors:\n\n* Michael Gallagher — chief toolkit architect/maintainer — michael.r.gallagher@noaa.gov\n* Jareth Holt — contributed code for table processing — jareth.holt@misu.su.se\n\nData-specification wranglers:\n\n* Leslie Hartten — leslie.m.hartten@noaa.gov\n* Siri Jodha Khalsa — sjsk@nsidc.org\n\nYOPP siteMIP steerers/evangelists:\n\n* Taneil Uttal — MDF evangelist — taneil.uttal@noaa.gov\n* Gunilla Svensson — siteMIP evangelist — gunilla@misu.su.se\n\nAnd the many more who have put time and energy into the YOPP siteMIP project","draft":false,"milestone":null,"merge_when_pipeline_succeeds":false,"detailed_merge_status":"title_regex","merge_user":null,"merged_at":null,"merge_after":null,"prepared_at":"2026-09-10T00:48:33.677Z","closed_by":null,"closed_at":null,"sha":"c152195ba6b110064690fca331b186c55a674fdf","merge_commit_sha":"","squash_commit_sha":"","user_notes_count":2,"should_remove_source_branch":false,"force_remove_source_branch":true,"allow_collaboration":true,"allow_maintainer_to_push":true,"web_url":"https://gitlab.com/gitlab-org/cli/-/merge_requests/3877","references":{"short":"!3877","relative":"!3877","full":"gitlab-org/cli!3877"},"discussion_locked":false,"time_stats":{"human_time_estimate":"","human_total_time_spent":"","time_estimate":0,"total_time_spent":0},"squash":true,"squash_on_merge":true,"task_completion_status":{"count":0,"completed_count":0},"has_conflicts":false,"blocking_discussions_resolved":true,"merged_by":null,"merge_error":"","subscribed":false,"changes_count":"1","user":{"can_merge":false},"latest_build_started_at":null,"latest_build_finished_at":null,"first_deployed_to_production_at":null,"pipeline":null,"head_pipeline":{"id":2835335288,"iid":2,"project_id":86287219,"status":"failed","source":"merge_request_event","ref":"patch-1","name":"","sha":"c152195ba6b110064690fca331b186c55a674fdf","before_sha":"c152195ba6b110064690fca331b186c55a674fdf","tag":false,"yaml_errors":"","user":{"id":40777488,"username":"yenrique","name":"Yvonne Locke","state":"active","created_at":null,"avatar_url":"https://secure.gravatar.com/avatar/30323efe48705a6dfab05263db5626e676d3395e4b60916c4be3a71e8583dcaf?s=80\u0026d=identicon","web_url":"https://gitlab.com/yenrique"},"updated_at":"2026-09-10T00:48:35.705Z","created_at":"2026-09-10T00:48:35.705Z","started_at":null,"finished_at":"2026-09-10T00:48:35.704Z","committed_at":null,"duration":0,"queued_duration":0,"coverage":"","web_url":"https://gitlab.com/yenrique/cli/-/pipelines/2835335288","detailed_status":{"icon":"status_failed","text":"Failed","label":"failed","group":"failed","tooltip":"failed","has_details":true,"details_path":"/yenrique/cli/-/pipelines/2835335288","illustration":{"image":""},"favicon":"/assets/ci_favicons/favicon_status_failed-41304d7f7e3828808b0c26771f0309e55296819a9beea3ea9fbf6689d9857c12.png"}},"diff_refs":{"base_sha":"b90b860210302907591750fb1a7068509c804b7b","head_sha":"c152195ba6b110064690fca331b186c55a674fdf","start_sha":"b90b860210302907591750fb1a7068509c804b7b"},"rebase_in_progress":false,"diverged_commits_count":0,"first_contribution":true,"work_in_progress":false} diff --git a/scripts/lib/forge.py b/scripts/lib/forge.py index 91b4f08..7cc8166 100644 --- a/scripts/lib/forge.py +++ b/scripts/lib/forge.py @@ -50,6 +50,11 @@ collapse that string into a verdict. A timeout must never be able to manufacture a merge. +WHAT SHIPS. Two adapters: GitHub through `gh`, GitLab through `glab`. Both are +CONFIGURATION — which hosts each one owns comes from that CLI's own variable +(`GH_HOST`, `GITLAB_HOST`), because a self-hosted instance is the normal case +for everything that is not github.com or gitlab.com. + ADDING A FORGE. Write a class with the methods below and register it: either in `BUILTIN` here, or — for a test, or a forge that is not fleet's business to ship — through `FLEET_FORGE_PLUGINS`, a colon-separated list of Python files @@ -67,6 +72,7 @@ import shutil import subprocess import sys +import urllib.parse from dataclasses import dataclass, field # --- identity ---------------------------------------------------------------- @@ -582,10 +588,548 @@ def merge(self, cr: ChangeRequest, method: str, delete_branch: bool) -> tuple: return True, f"{method}-merged" + (", branch deleted" if delete_branch else "") +# --- GitLab, the second implementation ---------------------------------------- + + +# GitLab pages at 100 and no higher, so "every open merge request" is a loop +# rather than one request. The cap is the same promise `GH_LIST_LIMIT` makes: +# reaching it means the project genuinely has that many open merge requests, +# which the caller then treats as unreadable rather than as a short list. +GL_PAGE = 100 +GL_LIST_LIMIT = 1000 + +# What `fleet-status.sh` reads out of a checkout. Lower than the shepherd's cap +# because it decides nothing and one line per merge request is all it prints. +GL_CHECKOUT_LIMIT = 50 + +# GitLab's own state words. `locked` is a real fourth state and is NOT one of +# fleet's three, so it falls out of this map and is reported as a sentence. +GL_STATES = {"opened": "open", "merged": "merged", "closed": "closed"} + +# `head_pipeline.status`. `manual` and `scheduled` are pipelines waiting for +# somebody, which is pending and not passing; `canceled` (GitLab spells it with +# one `l`) is its own verdict for the reason `Check` gives. +GL_PIPELINE_PASSED = {"success", "skipped"} +GL_PIPELINE_FAILED = {"failed"} +GL_PIPELINE_CANCELLED = {"canceled", "cancelling", "canceling"} + +# `detailed_merge_status`, of which GitLab has a long and growing list — the +# capture this adapter was written against answered `title_regex`, which is in +# no version of the documented set this code was checked against. So only the +# two words that mean something definite are read, and everything else is "the +# forge has not said", which `classify` treats as ask-again-shortly. Reading an +# unknown word as mergeable is how fleet would merge something GitLab is still +# thinking about. +GL_MERGEABLE = "mergeable" +GL_CONFLICT = "conflict" + +# A reviewer pressed "request changes". The one review verdict fleet acts on. +GL_CHANGES_REQUESTED = "requested_changes" + +# Developer. GitLab's ladder is 0 none / 5 minimal / 10 guest / 20 reporter / +# 30 developer / 40 maintainer / 50 owner, and developer is the first rung that +# may push. +GL_PUSH_ACCESS_LEVEL = 30 + +# `squash_option: never` is the setting that forbids fleet's merge method, and +# it is a PROJECT setting rather than a forge one — see `merge` below. +GL_SQUASH_FORBIDDEN = "never" + +# A merge request URL: `https://host/group/sub/project/-/merge_requests/12`. +# The host group keeps a `:port`, because a self-hosted instance on one is +# ordinary and `RepoId` carries the port as part of the host. +GL_URL_RE = re.compile( + r"^https?://([^/\s]+)/(.+?)/-/merge_requests/(\d+)(?:[/?#].*)?$" +) +# `https://host/group/proj.git`, `ssh://git@host:2222/group/proj.git`. +GL_REMOTE_URL_RE = re.compile( + r"^(?:https?|ssh|git)://(?:[^@/\s]+@)?([^/\s]+)/(.+?)(?:\.git)?/?$" +) +# `git@host:group/proj.git` — scp syntax, which carries no port. +GL_REMOTE_SCP_RE = re.compile(r"^(?:[^@/\s]+@)?([^:/\s]+):(.+?)(?:\.git)?/?$") + +# GitLab reserves these username prefixes for project and group access tokens, +# so they are the one thing in a merge request author that says "not a person". +GL_BOT_RE = re.compile(r"^(?:project|group)_\d+_bot") + +# glab prints its own errors as a decorated block on stderr. These are the +# decoration, not the message. +GL_NOISE = {"", "error", "warning"} + + +class GitLabForge(Forge): + """GitLab, through the `glab` CLI. Every `glab` invocation fleet makes is here. + + `glab` for the same reason the GitHub adapter uses `gh`: it already holds + whatever credential the operator gave this machine, and a fleet that needed + its own token would need one per machine a worker runs on. + + WHICH HOSTS ARE GITLAB. `gitlab.com`, plus `GITLAB_HOST` — glab's own + variable for a self-hosted instance, the way `GH_HOST` is gh's. A + self-hosted instance is the normal case here, so every call names its + repository by FULL URL (`-R https://host/group/project`) rather than by + slug: that is what makes `gitlab.example.com/group/proj` reach + gitlab.example.com and not gitlab.com. + + WHAT IT COSTS. GitLab does not put a merge request's pipeline in the list + endpoint, so listing open change requests is one call for the list plus one + per merge request. The GitHub adapter gets its whole answer in one call; + this one cannot, and paying the difference is better than reporting `checks` + empty, which every caller reads as "no check has reported yet". + """ + + name = "gitlab" + # GitLab's `squash` is not a merge method: it is a flag ON the merge, and + # the merge method (`merge` / `rebase_merge` / `ff`) is a separate project + # setting. So all three of fleet's words are things this forge can do, and + # the thing that can forbid a squash is per-PROJECT — `merge` asks. + merge_methods = ("squash", "merge", "rebase") + + def __init__(self, hosts=None): + extra = [self._host(h) for h in (hosts or [])] + extra.append(self._host(os.environ.get("GITLAB_HOST", ""))) + self.hosts = tuple(dict.fromkeys( + ["gitlab.com", "www.gitlab.com"] + [h for h in extra if h] + )) + # One answer per (repo, login), and one per repo for the squash + # setting: neither changes inside a run, and every open merge request + # would otherwise ask again. + self._push: dict = {} + self._squash: dict = {} + + @staticmethod + def _host(text: str) -> str: + """`https://gitlab.example.com/` as glab accepts it, down to a bare host.""" + text = str(text or "").strip().lower() + text = re.sub(r"^[a-z][a-z0-9+.-]*://", "", text) + return text.strip("/").split("/")[0] + + # --- naming --- + + def parse_change_url(self, url: str) -> ChangeRef | None: + m = GL_URL_RE.match((url or "").strip()) + if not m or not self.owns_host(m.group(1)): + return None + host, path = m.group(1).lower(), m.group(2) + # `group/project` at the very least: GitLab has no top-level projects, + # so a single segment is not a project path and not ours. + if "/" not in path: + return None + return ChangeRef( + RepoId(host, path), + int(m.group(3)), + f"https://{host}/{path}/-/merge_requests/{m.group(3)}", + ) + + def repo_from_remote(self, remote_url: str) -> RepoId | None: + text = (remote_url or "").strip() + m = GL_REMOTE_URL_RE.match(text) or GL_REMOTE_SCP_RE.match(text) + if not m: + return None + host, path = m.group(1).lower(), m.group(2).strip("/") + if not self.owns_host(host) or "/" not in path: + return None + return RepoId(host, path) + + # --- running glab --- + + def _repo_arg(self, repo: RepoId) -> str: + """How glab is told WHICH host, on every single call. + + `-R` takes a full URL as readily as a slug, and the URL is the only + form that carries the host — so this is what keeps a self-hosted + instance from being asked of gitlab.com. https because a GitLab + instance reachable only over plain http cannot be named this way; that + is the one shape of self-hosted install this adapter cannot address. + """ + return f"https://{repo.host}/{repo.path}" + + def _run(self, argv: list, cwd: str | None = None, timeout: int = 60) -> tuple: + """(stdout, why-not). A non-empty second value is never a verdict.""" + if not shutil.which("glab"): + return None, "glab not found on PATH" + try: + out = subprocess.run( + ["glab"] + argv, capture_output=True, text=True, cwd=cwd, timeout=timeout + ) + except (OSError, subprocess.SubprocessError) as exc: + return None, f"glab could not be run: {exc}" + if out.returncode != 0: + return None, self._why(out.stdout, out.stderr, out.returncode) + return out.stdout, "" + + @staticmethod + def _why(stdout: str, stderr: str, code: int) -> str: + """The sentence glab actually said, out of the two places it says it. + + With `-F json` glab puts `{"error":{"message":...}}` on STDOUT and a + boxed, blank-line-padded `ERROR` block on stderr, so taking the first + line of stderr yields the box and not the reason. `glab api` puts the + API's own `{"message":...}` on stdout instead. Both are read before + stderr is fallen back to. + """ + try: + doc = json.loads(stdout or "") + except ValueError: + doc = None + if isinstance(doc, dict): + err = doc.get("error") + said = err.get("message") if isinstance(err, dict) else doc.get("message") + if isinstance(said, str) and said.strip(): + return said.strip() + for line in (stderr or "").splitlines(): + line = re.sub(r"\x1b\[[0-9;]*m", "", line).strip() + if line.lower().rstrip(":") not in GL_NOISE: + return line + return f"glab exited {code}" + + def _json(self, argv: list, cwd: str | None = None, timeout: int = 60) -> tuple: + out, why = self._run(argv, cwd=cwd, timeout=timeout) + if why: + return None, why + try: + return json.loads(out), "" + except ValueError: + return None, "glab returned output that is not JSON" + + def _api(self, repo: RepoId, path: str, timeout: int = 60) -> tuple: + """`glab api` against ONE host, for the questions `glab mr` has no verb for.""" + return self._json(["api", path, "--hostname", repo.host], timeout=timeout) + + @staticmethod + def _project(repo: RepoId) -> str: + """`group%2Fsub%2Fproject` — how a project path goes into an API path.""" + return urllib.parse.quote(repo.path, safe="") + + def _view(self, repo: RepoId, number: int) -> tuple: + out, why = self._json( + ["mr", "view", str(number), "-R", self._repo_arg(repo), "-F", "json"], + timeout=30, + ) + if why: + return None, f"glab mr view failed: {why}" + if not isinstance(out, dict): + return None, "glab mr view did not answer with an object" + return out, "" + + # --- the questions --- + + def get(self, ref: ChangeRef) -> tuple: + doc, why = self._view(ref.repo, ref.number) + if why: + return None, why + return self._change_request(doc, ref.repo, self._commits(ref)), "" + + def _commits(self, ref: ChangeRef) -> list: + """Oldest first, which is the opposite of the order GitLab answers in. + + Read for one thing — telling the pipeline's own follow-up push apart + from somebody pushing over it — so a call that fails answers with an + empty list rather than a guess, exactly as `Commit` says it must. + """ + docs, _why = self._api( + ref.repo, + f"projects/{self._project(ref.repo)}/merge_requests/{ref.number}" + f"/commits?per_page={GL_PAGE}", + timeout=30, + ) + if not isinstance(docs, list): + return [] + out = [ + Commit(str(c.get("id") or ""), str(c.get("title") or "")) + for c in docs + if isinstance(c, dict) + ] + out.reverse() + return out + + def state(self, ref: ChangeRef) -> tuple: + out, why = self._run( + ["mr", "view", str(ref.number), "-R", self._repo_arg(ref.repo), + "-F", "json", "--jq", ".state"], + timeout=30, + ) + if why: + return None, f"glab mr view could not read the state: {why}" + said = (out or "").strip().strip('"').lower() + state = GL_STATES.get(said) + if not state: + return None, f"glab answered an unrecognised merge request state: {said!r}" + return state, "" + + def open_change_requests(self, repo: RepoId) -> tuple: + docs, why = self._page(repo, GL_LIST_LIMIT) + if why: + return [], why + return self._enriched(docs, repo) + + def open_change_requests_in_checkout(self, path: str) -> tuple: + docs, why = self._json( + ["mr", "list", "-F", "json", "--per-page", str(GL_CHECKOUT_LIMIT)], + cwd=path, + timeout=20, + ) + if why: + return [], why + if not isinstance(docs, list): + return [], "glab returned something that is not a list of merge requests" + docs = [d for d in docs if isinstance(d, dict)] + if len(docs) >= GL_CHECKOUT_LIMIT: + return [], self._truncated(GL_CHECKOUT_LIMIT) + # A checkout names no repository, so take the one every merge request + # already carries: its own web_url. A directory whose merge requests + # are on a host this adapter does not own is not ours to answer for. + repo = None + for d in docs: + ref = self.parse_change_url(str(d.get("web_url") or "")) + if ref is None: + return [], ( + "glab answered with a merge request whose web_url is on no " + "host this adapter owns" + ) + repo = ref.repo + if repo is None: + return [], "" if self._is_ours(path) else "not a checkout of a GitLab project" + return self._enriched(docs, repo) + + def _is_ours(self, path: str) -> bool: + """Does this checkout's `origin` name a host we own? Only asked when it + has no open merge request to answer with, since an empty list has to be + "none are open" and not "this is a GitHub repository".""" + return self.repo_from_remote(_git_remote(path)) is not None + + def _page(self, repo: RepoId, limit: int) -> tuple: + """Every open merge request, one page of 100 at a time.""" + out: list = [] + page = 1 + while len(out) < limit: + docs, why = self._json( + ["mr", "list", "-R", self._repo_arg(repo), "-F", "json", + "--per-page", str(GL_PAGE), "--page", str(page)] + ) + if why: + return [], why + if not isinstance(docs, list): + return [], "glab returned something that is not a list of merge requests" + docs = [d for d in docs if isinstance(d, dict)] + out.extend(docs) + if len(docs) < GL_PAGE: + return out, "" + page += 1 + return [], self._truncated(limit) + + @staticmethod + def _truncated(limit: int) -> str: + return ( + f"the project has at least {limit} open merge requests; glab's result " + "may be truncated, so treating it as unreadable rather than silently " + "dropping some" + ) + + def _enriched(self, docs: list, repo: RepoId) -> tuple: + """The list, with the pipeline GitLab leaves out of it. + + One failure fails the WHOLE list. A short list reads as "these are all + the open merge requests", and dropping the conflicting one from it is + how a shepherd would decide it had nothing to report. + """ + out = [] + for d in docs: + number = d.get("iid") + if not number: + continue + full, why = self._view(repo, int(number)) + if why: + return [], f"could not read merge request !{number} on {repo}: {why}" + out.append(self._change_request(full, repo)) + return out, "" + + def _change_request(self, d: dict, repo: RepoId, commits=None) -> ChangeRequest: + number = int(d.get("iid") or 0) + ref = self.parse_change_url(str(d.get("web_url") or "")) or ChangeRef( + repo, number, f"https://{repo.host}/{repo.path}/-/merge_requests/{number}" + ) + + # WHOSE BRANCH. On GitLab a fork is a project of its own, so this is + # two integers and not a name — and when either is missing the answer + # is `None`, which `classify` reads as undetermined rather than as a + # stranger or as one of ours. + source, target = d.get("source_project_id"), d.get("target_project_id") + if not isinstance(source, int) or not isinstance(target, int): + ours, where = None, "" + elif source == target: + ours, where = True, f"{ref.repo.path} itself" + else: + # The merge request says which project the branch is in by id and + # never by name, and resolving the id would be another call for a + # sentence nobody acts on. + ours, where = False, f"another project on {ref.repo.host} (id {source})" + + said = str(d.get("detailed_merge_status") or "").lower() + if d.get("has_conflicts") is True or said == GL_CONFLICT: + mergeable = "conflicting" + elif said == GL_MERGEABLE: + mergeable = "mergeable" + else: + mergeable = "" + + author = d.get("author") or {} + login = str(author.get("username") or "") + return ChangeRequest( + ref=ref, + title=str(d.get("title") or ""), + state=GL_STATES.get(str(d.get("state") or "").lower(), ""), + draft=bool(d.get("draft")), + body=str(d.get("description") or ""), + head_branch=str(d.get("source_branch") or ""), + base_branch=str(d.get("target_branch") or ""), + head_sha=str(d.get("sha") or ""), + author=login, + # The author object carries no `bot` flag, so the only thing that + # says "not a person" is the username shape GitLab reserves for + # project and group access tokens. + author_is_bot=bool(GL_BOT_RE.match(login)), + mergeable=mergeable, + review_decision=( + "changes-requested" if said == GL_CHANGES_REQUESTED else "" + ), + checks=self._checks(d), + commits=list(commits or []), + head_is_ours=ours, + head_location=where, + ) + + @staticmethod + def _checks(d: dict) -> list: + """The head pipeline, as one check — and NOTHING when it is not the head's. + + GitLab keeps the previous commit's pipeline in `head_pipeline` until the + new one is created, so a pipeline whose `sha` is not the merge request's + is a green light for code nobody ran. An empty list is `classify`'s "no + check has reported yet", which is the correct answer there. + """ + p = d.get("head_pipeline") + if not isinstance(p, dict): + return [] + ran, head = str(p.get("sha") or ""), str(d.get("sha") or "") + if ran and head and ran.lower() != head.lower(): + return [] + name = str(p.get("name") or "") or f"pipeline #{p.get('id') or 'unnumbered'}" + said = str(p.get("status") or "").lower() + if said in GL_PIPELINE_FAILED: + return [Check(name, "failed")] + if said in GL_PIPELINE_PASSED: + return [Check(name, "passed")] + if said in GL_PIPELINE_CANCELLED: + return [Check(name, "cancelled")] + return [Check(name, "pending")] + + def can_push(self, repo: RepoId, login: str) -> tuple: + """May this account push here — asked of the members list, by username. + + `members/all` rather than `members`: it includes membership inherited + from the group, which is how almost everybody who can push to a GitLab + project has it. + """ + key = (repo.qualified, login) + if key in self._push: + return self._push[key] + docs, why = self._api( + repo, + f"projects/{self._project(repo)}/members/all" + f"?query={urllib.parse.quote(login)}&per_page={GL_PAGE}", + ) + if why or not isinstance(docs, list): + answer = ( + False, + f"could not check whether {login} can push to {repo}: " + f"{why or 'unexpected output'}", + ) + else: + level = None + for m in docs: + if not isinstance(m, dict): + continue + if str(m.get("username") or "").lower() == login.lower(): + level = m.get("access_level") + break + if not isinstance(level, int): + answer = (False, f"{login} is not a member of {repo}") + else: + answer = ( + level >= GL_PUSH_ACCESS_LEVEL, + f"{login} has access level {level} on {repo}, and " + f"{GL_PUSH_ACCESS_LEVEL} (developer) is the first that may push", + ) + self._push[key] = answer + return answer + + def _squash_allowed(self, repo: RepoId) -> tuple: + """(True / False / None, why). `None` is "the project did not say". + + A project can be configured `squash_option: never`, and squash is the + only method fleet merges by. Asking first turns that into a refusal + fleet records, rather than an API error after the fact — and `None` + must not block, because "I could not read the setting" is not "the + setting forbids it". + """ + if repo.qualified in self._squash: + return self._squash[repo.qualified] + doc, why = self._api(repo, f"projects/{self._project(repo)}") + if why or not isinstance(doc, dict): + answer = (None, why or "glab did not answer with a project") + elif str(doc.get("squash_option") or "").lower() == GL_SQUASH_FORBIDDEN: + answer = (False, ( + f"{repo} is configured `squash_option: never`, and squash is the " + "only method fleet merges by" + )) + else: + answer = (True, "") + self._squash[repo.qualified] = answer + return answer + + def describe_merge(self, method: str, delete_branch: bool) -> str: + argv = ["glab mr merge --yes --auto-merge=false"] + if method == "squash": + argv.append("--squash") + elif method == "rebase": + argv.append("--rebase") + if delete_branch: + argv.append("--remove-source-branch") + return " ".join(argv) + + def merge(self, cr: ChangeRequest, method: str, delete_branch: bool) -> tuple: + if method not in self.merge_methods: + return False, f"gitlab cannot merge by {method}" + if method == "squash": + allowed, why = self._squash_allowed(cr.repo) + if allowed is False: + return False, why + argv = ["mr", "merge", str(cr.number), "-R", self._repo_arg(cr.repo), + "--yes", "--auto-merge=false"] + if method == "squash": + argv.append("--squash") + elif method == "rebase": + argv.append("--rebase") + if delete_branch: + argv.append("--remove-source-branch") + if cr.head_sha: + # Merge THIS commit or nothing. glab's own flag for it, and the + # only thing standing between "fleet checked the head" and a push + # that lands between the check and the merge. + argv += ["--sha", cr.head_sha] + _, why = self._run(argv, timeout=120) + if why: + return False, why + return True, f"{method}-merged" + ( + ", source branch removed" if delete_branch else "" + ) + + # --- the registry ------------------------------------------------------------ -BUILTIN = (GitHubForge,) +BUILTIN = (GitHubForge, GitLabForge) # A colon-separated list of Python files, each exporting `forges()`. This is # how the selftest drives the whole queue through a forge that has no network @@ -692,13 +1236,17 @@ def open_change_requests_in_checkout(path: str) -> tuple: forge = f break candidates = [forge] if forge else list(forges()) - why = "no forge is configured" + reasons = [] for f in candidates: crs, err = f.open_change_requests_in_checkout(path) if not err: return crs, "" - why = err - return [], why + reasons.append(err) + # EVERY reason, not the last one. Once two forges are configured, the + # commonest failure here is that neither CLI is installed, and reporting + # only whichever was tried second names one missing tool and hides the + # other — which reads as "install glab" on a machine that talks to GitHub. + return [], "; ".join(dict.fromkeys(reasons)) or "no forge is configured" def _git_remote(path: str) -> str: diff --git a/scripts/lib/pane_harness.lua b/scripts/lib/pane_harness.lua index ee0f695..334dcff 100644 --- a/scripts/lib/pane_harness.lua +++ b/scripts/lib/pane_harness.lua @@ -281,7 +281,9 @@ local TOPICS = { }, { id = "03-drop-the-webui-selftest", state = "dispatched", title = "Drop the monitor's selftest and its CI job", - artifact = "https://github.com/Thurbeen/fleet/pull/52", + -- A GitLab merge request, so the pane is rendered against both + -- forges fleet ships an adapter for rather than only one. + artifact = "https://gitlab.example.com/acme/group/widgets/-/merge_requests/52", brief = 1, events = 2, result = 1, moved = ago(3), publish = { "no-mistakes", "open", ago(3) }, }, diff --git a/scripts/lib/queue.py b/scripts/lib/queue.py index 284641d..f9b9447 100644 --- a/scripts/lib/queue.py +++ b/scripts/lib/queue.py @@ -115,9 +115,10 @@ def _load_forge(): # WHICH FORGE. Everything fleet knows about a change request — a pull request # on GitHub, a merge request on GitLab — it asks this module for. Nothing in -# this file runs `gh` or builds a github.com URL; the one exception is -# FORGE_PROBE's ssh check below, which is a different coupling (git hosting, -# not the forge API) and says so where it lives. +# this file runs a forge CLI or builds a forge URL; the one exception is +# FORGE_PROBE_TEMPLATE's ssh check below, which is a different coupling (git +# hosting, not the forge API) and reads the repository's own `origin` rather +# than naming a forge. forge = _load_forge() # The four conditions that justify making one task wait for another. They are @@ -1936,31 +1937,74 @@ def ssh_run(entry: dict, script: str, stdin: str | None = None): ) -# What probe 2 accepts, and why it is two questions and not one. §1a names -# `ssh -T git@github.com`, which proves an SSH key. A host that clones over -# HTTPS with a `gh` token has no such key and is perfectly able to push, so -# testing only the key would refuse a working host. Either credential passes; -# neither is read, moved, or reported beyond the word that says which was found. +# GIT HOSTING, NOT THE FORGE API. `scripts/lib/forge.py` answers "what is this +# change request"; this answers "can this machine clone, fetch and push". They +# are different couplings and this is the one that lives here. +# +# IT ASKS THE REPOSITORY'S OWN HOST, and that is the whole point. §1a used to +# name `ssh -T git@github.com` flatly, which proves nothing about a checkout +# whose `origin` is a GitLab instance — a remote GitLab task would pass the +# probe and then fail at its first `git push`. So the script reads the repo's +# `origin` on the host and probes THAT, port and all. # -# STILL GITHUB-SHAPED, and knowingly. This is GIT HOSTING — can this machine -# clone, fetch and push — and not the forge API that `scripts/lib/forge.py` -# covers. They are different couplings, and this one is left whole for its own -# task rather than half-done here. -FORGE_PROBE = """\ -if ssh -o BatchMode=yes -T git@github.com 2>&1 | grep -q 'successfully authenticated'; then - printf 'an ssh key' +# TWO QUESTIONS AND NOT ONE. The ssh probe proves a key. A host that clones +# over HTTPS with a CLI token has no key and is perfectly able to push, so +# testing only the key would refuse a working host. Either credential passes; +# neither is read, moved, or reported beyond the word that says which was +# found. The banners are GitHub's and GitLab's own — the two forges fleet +# ships adapters for — because `ssh -T` exits non-zero on a successful GitHub +# authentication, so the exit status cannot be the test. +FORGE_PROBE_TEMPLATE = """\ +url=$(git -C __REPO__ remote get-url origin 2>/dev/null || printf '') +case "$url" in +*://*) rest=${url#*://}; rest=${rest#*@}; hostport=${rest%%/*} ;; +*@*:*) rest=${url#*@}; hostport=${rest%%:*} ;; +*) hostport='' ;; +esac +host=${hostport%%:*} +port='' +case "$hostport" in *:*) port=${hostport#*:} ;; esac +if [ -z "$host" ]; then + printf 'no forge: %s has no readable `origin`, so there is no host to prove a credential against' __REPO__ + exit 1 +fi +if [ -n "$port" ]; then + banner=$(ssh -o BatchMode=yes -p "$port" -T "git@$host" 2>&1) +else + banner=$(ssh -o BatchMode=yes -T "git@$host" 2>&1) +fi +case "$banner" in +*'successfully authenticated'*|*'Welcome to GitLab'*) + printf '%s with an ssh key' "$host" + exit 0 + ;; +esac +if command -v gh >/dev/null 2>&1 && gh auth status --hostname "$host" >/dev/null 2>&1; then + printf '%s with a gh token' "$host" exit 0 fi -if command -v gh >/dev/null 2>&1 && gh auth status >/dev/null 2>&1; then - printf 'a gh token' +if command -v glab >/dev/null 2>&1 && glab auth status --hostname "$host" >/dev/null 2>&1; then + printf '%s with a glab token' "$host" exit 0 fi +printf '%s' "$host" exit 1 """ +def forge_probe(repo: str) -> str: + """The credential probe, for one repository path on the host.""" + return FORGE_PROBE_TEMPLATE.replace("__REPO__", shlex.quote(repo)) + + def probe_host(entry: dict, repo: str) -> list: - """§1a's questions, in §1a's order, stopping at the first NO. + """§1a's questions, stopping at the first NO. + + The repository is asked about BEFORE its forge, because which forge to + prove a credential against is a fact about that checkout's `origin` — a + remote task on a GitLab repository needs a GitLab credential, and asking + github.com about it is how one used to pass the probe and then fail at its + first `git push`. Returns one {check, ok, detail} per probe run. Nothing is spawned until every one of them passes: a remote worker that starts and then fails at its @@ -1982,16 +2026,6 @@ def probe_host(entry: dict, repo: str) -> list: return out out.append({"check": "reachable", "ok": True, "detail": "answers ssh, POSIX shell"}) - creds = ssh_run(entry, FORGE_PROBE) - if creds.returncode != 0: - out.append({"check": "forge", "ok": False, "detail": ( - "the host has no GitHub credentials of its own — neither an ssh key " - "nor a `gh` login. It cannot clone, fetch or push. Give that MACHINE " - "its own credentials; nothing here sends yours.")}) - return out - out.append({"check": "forge", "ok": True, - "detail": f"reaches GitHub with {creds.stdout.strip() or 'a credential'}"}) - quoted = shlex.quote(repo) check = ( f"if [ ! -d {quoted} ]; then printf no-dir; exit 1; fi\n" @@ -2009,6 +2043,24 @@ def probe_host(entry: dict, repo: str) -> list: "nothing local validates it.")}) return out out.append({"check": "repo", "ok": True, "detail": f"{repo} is a git checkout there"}) + + creds = ssh_run(entry, forge_probe(repo)) + if creds.returncode != 0: + # The probe names the host it tried, so a machine with a key for one + # forge and none for the other says WHICH — which is the whole reason + # this asks the repository rather than a constant. + tried = creds.stdout.strip() or "its forge" + detail = ( + f"the host has no credentials of its own for {tried} — neither an ssh " + "key nor a `gh` or `glab` login. It cannot clone, fetch or push. Give " + "that MACHINE its own credentials; nothing here sends yours." + ) + if tried.startswith("no forge: "): + detail = tried[len("no forge: "):] + out.append({"check": "forge", "ok": False, "detail": detail}) + return out + out.append({"check": "forge", "ok": True, + "detail": f"reaches {creds.stdout.strip() or 'its forge with a credential'}"}) return out @@ -2331,8 +2383,8 @@ def cmd_dispatch(args) -> int: if t.doc.get("host"): print(f" on host {t.doc['host']} — probed first, and not spawned" " until all three pass:") - print(" reachable and a POSIX shell / has its own GitHub" - " credentials / the repo is there") + print(" reachable and a POSIX shell / the repo is there" + " / it has its own credentials for that repo's forge") print(f" {shell_quote(create)}") if t.doc.get("host"): print(" ssh 'cat > /BRIEF.md' # the worker's" diff --git a/scripts/pane-selftest.sh b/scripts/pane-selftest.sh index 7ceb55e..03eb583 100755 --- a/scripts/pane-selftest.sh +++ b/scripts/pane-selftest.sh @@ -131,7 +131,7 @@ expect "the publish verdict is still drawn" "green" "$WIDE" # row reads as settled, which is the opposite of what it means, so the row says # the move in words instead of borrowing a colour that would claim one. expect "an open pull request says what to do about it" "open — review" "$WIDE" -expect "and the second pull request is named" "#52" "$WIDE" +expect "and a GitLab merge request is named in GitLab's own notation" "!52" "$WIDE" expect "a green one still says whose merge it is" "green — yours to merge" "$WIDE" # The order the ladder gives things up in, pinned: the METHOD is provenance and diff --git a/scripts/queue-selftest.sh b/scripts/queue-selftest.sh index bd195a9..2733e58 100755 --- a/scripts/queue-selftest.sh +++ b/scripts/queue-selftest.sh @@ -438,8 +438,8 @@ case "\$script" in printf fleet-posix-ok ;; *successfully?authenticated*) - [ -f "$sshstate/\$dest.noforge" ] && exit 1 - printf 'an ssh key' + [ -f "$sshstate/\$dest.noforge" ] && { printf github.com; exit 1; } + printf 'github.com with an ssh key' ;; *no-dir*) [ -f "$sshstate/\$dest.norepo" ] && { printf no-dir; exit 1; } @@ -4457,6 +4457,503 @@ else pass "no code path ran \`gh\` while a different forge was configured" fi +# --- 14. GITLAB: the second REAL adapter, over recorded `glab` output -------- +# +# Section 13 proves the seam with a forge that exists only in that section. +# This proves the adapter fleet actually ships for GitLab, and it is a +# different claim: the fake forge answers whatever fleet asks, while `glab` +# answers what GitLab decided to answer, in GitLab's own words and shapes. +# +# So the fixtures matter more than the code here. `scripts/fixtures/glab/` is +# real `glab` 1.117.0 output recorded from gitlab.com — its README says which +# command produced each file and which two answers are behind authentication +# and therefore CONSTRUCTED below rather than recorded. A fake `glab` on PATH +# replays them; nothing in this section reaches a network, and `gh` is a +# tripwire again, because a GitLab merge request is the one thing that must +# never be asked about with `gh`. +# +# What it proves: +# +# the recorded shapes parse — a fork's merge request is read as NOT ours, +# commits come back oldest-first out of a newest-first answer, and glab's +# two-stream error is read from the stream that carries the reason +# a pipeline for a commit that is no longer the head is NO check, not a pass +# a self-hosted instance round-trips: `GITLAB_HOST`, a subgroup path, and +# every call naming its host by full URL +# `squash_option: never` is a refusal fleet RECORDS, not a crash and not a +# merge by some other method +# the remote-host credential probe asks the repository's own forge + +gl="$tmp/gitlab" +mkdir -p "$gl/mrs" "$gl/api" "$gl/bin" +export FAKE_GLAB_DIR="$gl" +: >"$gl/calls.log" +: >"$gl/merged.log" +: >"$gl/gh-calls.log" + +fixtures="$PWD/scripts/fixtures/glab" + +# A `glab` that reads files instead of an instance. It knows only the four +# verbs the adapter uses, and it is deliberately literal about the two things +# recorded output taught us: `--jq .state` prints a bare word, and a failure +# puts its reason on STDOUT as JSON with a decorated box on stderr. +cat >"$gl/bin/glab" <<'PY' +#!/usr/bin/env python3 +import json +import os +import sys + +D = os.environ["FAKE_GLAB_DIR"] +argv = sys.argv[1:] +with open(os.path.join(D, "calls.log"), "a") as fh: + fh.write(" ".join(argv) + "\n") + + +def flag(name, default=None): + for i, a in enumerate(argv): + if a == name and i + 1 < len(argv): + return argv[i + 1] + if a.startswith(name + "="): + return a.split("=", 1)[1] + return default + + +def emit(doc): + sys.stdout.write(json.dumps(doc) + "\n") + raise SystemExit(0) + + +def refuse(message, recorded=None): + # A refusal is two streams, and which one carries the REASON is the thing + # an adapter gets wrong. `recorded` replays the pair exactly as glab wrote + # it; everything else is built to the same shape. + if recorded and os.path.exists(recorded + ".json"): + sys.stdout.write(open(recorded + ".json").read()) + sys.stderr.write(open(recorded + ".stderr").read()) + raise SystemExit(1) + sys.stdout.write(json.dumps({"error": {"message": message}}) + "\n") + sys.stderr.write("\n \n ERROR \n \n %s\n\n" % message) + raise SystemExit(1) + + +def load(number): + path = os.path.join(D, "mrs", "%s.json" % number) + return json.load(open(path)) if os.path.exists(path) else None + + +def opened(): + out = [] + for name in sorted(os.listdir(os.path.join(D, "mrs"))): + if not name.endswith(".json"): + continue + doc = json.load(open(os.path.join(D, "mrs", name))) + if doc.get("state") == "opened": + out.append(doc) + return out + + +if argv[:2] == ["mr", "view"]: + doc = load(argv[2]) + if doc is None: + refuse("failed to get merge request %s: 404 Not Found" % argv[2], + recorded=os.path.join(D, "missing") + if argv[2] == "999999" else None) + if flag("--jq") == ".state": + sys.stdout.write(str(doc.get("state") or "") + "\n") + raise SystemExit(0) + emit(doc) + +if argv[:2] == ["mr", "list"]: + emit([] if int(flag("--page", "1")) > 1 else opened()) + +if argv[:2] == ["mr", "merge"]: + if os.path.exists(os.path.join(D, "merge-refused")): + refuse("405 Method Not Allowed") + with open(os.path.join(D, "merged.log"), "a") as fh: + fh.write(" ".join(argv) + "\n") + sys.stdout.write("Merged!\n") + raise SystemExit(0) + +if argv[:1] == ["api"]: + path = argv[1].split("?")[0] + if path.endswith("/commits"): + name = "commits" + elif "/members/all" in path: + name = "members" + elif path.count("/") == 1: + name = "project" + else: + refuse("404 Not Found") + served = os.path.join(D, "api", name + ".json") + if not os.path.exists(served): + refuse("404 Not Found") + sys.stdout.write(open(served).read()) + raise SystemExit(0) + +refuse("unknown command: %s" % " ".join(argv)) +PY +chmod +x "$gl/bin/glab" + +cat >"$gl/bin/gh" <<'SH' +#!/bin/sh +echo "gh $*" >>"$FAKE_GLAB_DIR/gh-calls.log" +echo "gh: a GitLab merge request must never be asked about with gh" >&2 +exit 1 +SH +chmod +x "$gl/bin/gh" + +# The recorded fork merge request, under its own number, exactly as recorded — +# and the recorded refusal, replayed on both streams for merge request 999999, +# which is the number it was recorded against. +cp "$fixtures/mr-view.json" "$gl/mrs/3877.json" +cp "$fixtures/mr-commits.json" "$gl/api/commits.json" +cp "$fixtures/mr-view-missing.json" "$gl/missing.json" +cp "$fixtures/mr-view-missing.stderr" "$gl/missing.stderr" + +# CONSTRUCTED, and labelled: `GET /projects/:id` and `/members/all` are behind +# authentication, so these carry the field names from GitLab's REST API +# documentation and values this test chooses. The README beside the recordings +# says so too. +printf '{"id": 42, "path_with_namespace": "acme/group/widgets", "squash_option": "default_on"}\n' \ + >"$gl/api/project.json" +printf '[{"id": 7, "username": "letur", "access_level": 40}]\n' >"$gl/api/members.json" + +# Every merge request below is DERIVED FROM THE RECORDED ONE: the recorded +# object is loaded and named fields are overwritten, so each fixture keeps the +# real shape and only the facts under test are this test's invention. +glab_mr() { + python3 - "$fixtures/mr-view.json" "$gl/mrs" "$@" <<'PY' +import json +import sys + +recorded, out, number = sys.argv[1], sys.argv[2], int(sys.argv[3]) +doc = json.load(open(recorded)) +sha = "%040d" % number +steps = [{"step": s, "status": "completed"} for s in + ("intent", "rebase", "review", "test", "document", "lint", "push")] +steps += [{"step": "pr", "status": "running"}, {"step": "ci", "status": "pending"}] +payload = json.dumps({"head_sha": sha, "steps": steps}) +doc.update({ + "iid": number, + "id": 900000 + number, + "web_url": "https://gitlab.example.com/acme/group/widgets" + "/-/merge_requests/%d" % number, + "project_id": 42, + "source_project_id": 42, + "target_project_id": 42, + "state": "opened", + "draft": False, + "sha": sha, + "title": "change %d" % number, + "target_branch": "main", + "has_conflicts": False, + "detailed_merge_status": "mergeable", + "author": {"id": 7, "username": "letur", "name": "letur", "state": "active"}, + "description": "\n\n" % payload + + "\n".join("## %s\nx\n" % h for h in + ("Intent", "What Changed", "Risk Assessment", + "Testing", "Pipeline")), + "head_pipeline": {"id": 5000 + number, "name": "", "sha": sha, + "status": "success"}, +}) +for pair in sys.argv[4:]: + key, _, value = pair.partition("=") + doc[key] = json.loads(value) +json.dump(doc, open("%s/%d.json" % (out, number), "w")) +PY +} + +# --- 14a. the recorded shapes parse, and the fork is read as not ours ------- + +GLPATH="$gl/bin:$base_path" +env PATH="$GLPATH" FAKE_GLAB_DIR="$gl" python3 - "$PWD/scripts/lib" >"$tmp/gl-unit.tsv" <<'PY' +import json +import os +import sys + +sys.path.insert(0, sys.argv[1]) +import forge # noqa: E402 + +rows = [] + + +def claim(name, got, want): + rows.append(("PASS", name, "") if got == want + else ("FAIL", name, "wanted %r, got %r" % (want, got))) + + +gl = forge.GitLabForge() + +# The recorded merge request, through the public interface and the fake CLI. +ref = gl.parse_change_url("https://gitlab.com/gitlab-org/cli/-/merge_requests/3877") +claim("a /-/merge_requests/ URL on gitlab.com is a change request", ref is not None, True) +cr, why = gl.get(ref) +claim("and glab answers for it", why, "") +claim("its head commit is the recorded one", cr.head_sha, + "c152195ba6b110064690fca331b186c55a674fdf") +claim("its head branch is the recorded one", cr.head_branch, "patch-1") +claim("its base branch is the recorded one", cr.base_branch, "main") +claim("GitLab's `opened` is fleet's `open`", cr.state, "open") +claim("a merge request from a FORK is not ours", cr.head_is_ours, False) +claim("and the refusal line says where it lives", + "another project on gitlab.com" in cr.head_location, True) +claim("its failed pipeline is one failed check", + [(c.verdict) for c in cr.checks], ["failed"]) +claim("an undocumented detailed_merge_status is not read as mergeable", + cr.mergeable, "") +claim("commits come back oldest-first out of a newest-first answer", + [c.headline for c in cr.commits][0], + "chore(lint): add comment volume and overlap scripts") +claim("and the newest recorded commit is last", + [c.headline for c in cr.commits][-1], + "refactor: fix gocritic findings and delete comments that restate the code") + +# The recorded error: the reason is on stdout, and stderr's first line is a box. +missing = forge.ChangeRef(ref.repo, 999999, + "https://gitlab.com/gitlab-org/cli/-/merge_requests/999999") +gone, why = gl.get(missing) +claim("a merge request that is not there is a reason, not an exception", gone, None) +claim("and the reason is the one glab put on stdout", "404 Not Found" in why, True) +claim("not the decorated box it put on stderr", "ERROR" in why, False) + +# Hosts. +claim("a github.com pull request is not this adapter's", + gl.parse_change_url("https://github.com/Thurbeen/fleet/pull/1"), None) +claim("nor is a single-segment path, which GitLab has no such thing as", + gl.parse_change_url("https://gitlab.com/project/-/merge_requests/1"), None) +claim("an unconfigured self-hosted host is not ours either", + gl.parse_change_url( + "https://gitlab.example.com/acme/group/widgets/-/merge_requests/9"), None) + +os.environ["GITLAB_HOST"] = "https://gitlab.example.com/" +selfhosted = forge.GitLabForge() +ref = selfhosted.parse_change_url( + "https://gitlab.example.com/acme/group/widgets/-/merge_requests/301") +claim("GITLAB_HOST configures a self-hosted instance, scheme and all", + ref is not None, True) +claim("and a subgroup path is the whole path", ref.repo.path, "acme/group/widgets") +claim("whose first segment is the owner", ref.repo.owner, "acme") +for remote in ("https://gitlab.example.com/acme/group/widgets.git", + "git@gitlab.example.com:acme/group/widgets.git", + "ssh://git@gitlab.example.com/acme/group/widgets"): + claim("a checkout's origin names the project: %s" % remote, + selfhosted.repo_from_remote(remote), + forge.RepoId("gitlab.example.com", "acme/group/widgets")) + +# Pipelines. Each case is written into the fake CLI's own store and read back +# through `get`, so the interface under test is the one the queue calls. +mrs = os.path.join(os.environ["FAKE_GLAB_DIR"], "mrs") +recorded = json.load(open(os.path.join(mrs, "3877.json"))) + + +def pipeline_verdicts(pipeline): + doc = dict(recorded, iid=401, web_url=ref.url.replace("301", "401"), + source_project_id=42, target_project_id=42, head_pipeline=pipeline) + json.dump(doc, open(os.path.join(mrs, "401.json"), "w")) + got, _why = selfhosted.get(forge.ChangeRef(ref.repo, 401, doc["web_url"])) + return [c.verdict for c in got.checks] + + +head = recorded["sha"] +claim("a pipeline for a commit that is no longer the head is no check at all", + pipeline_verdicts({"id": 1, "sha": "0" * 40, "status": "success"}), []) +claim("and a merge request with no pipeline at all is no check either", + pipeline_verdicts(None), []) +for said, want in (("success", "passed"), ("skipped", "passed"), ("failed", "failed"), + ("canceled", "cancelled"), ("running", "pending"), + ("manual", "pending"), ("created", "pending")): + claim("pipeline %s reads as %s" % (said, want), + pipeline_verdicts({"id": 1, "sha": head, "status": said}), [want]) + +for verdict, name, detail in rows: + print("%s\t%s\t%s" % (verdict, name, detail)) +PY + +while IFS=$'\t' read -r verdict claim detail; do + if [ "$verdict" = PASS ]; then pass "$claim"; else fail "$claim" "$detail"; fi +done <"$tmp/gl-unit.tsv" + +# --- 14b. the whole queue, driven through the GitLab adapter ----------------- +# +# The recorded merge request and the pipeline cases above belong to gitlab.com +# and to no task; take them out of the fake CLI's store before the queue is +# asked what is open, and start the call log over so that the "nothing was +# aimed at gitlab.com" claim below is about this half of the section. + +rm -f "$gl/mrs/3877.json" "$gl/mrs/401.json" +: >"$gl/calls.log" + +glrepo="$gl/repo" +mkdir -p "$glrepo" +git -C "$glrepo" init -q -b main +git -C "$glrepo" -c user.email=t@t -c user.name=t commit -q --allow-empty -m base +git -C "$glrepo" remote add origin "https://gitlab.example.com/acme/group/widgets.git" + +glq() { + env PATH="$gl/bin:$tbxbin:$sshbin:$base_path" FLEET_QUEUE_DIR="$tmp/queue-gitlab" \ + GITLAB_HOST=gitlab.example.com \ + FLEET_AUTO_MERGE_REPOS="gitlab.example.com/acme/group/widgets" \ + "$QUEUE" "$@" +} + +gltopic="$(glq topic add on-gitlab --title 'Work on a self-hosted GitLab' \ + --prompt 'fleet must work on GitLab too')" + +for spec in 01:landed:301 02:conflicting:302 03:green:303 04:foreign:304; do + IFS=: read -r n slug num <<<"$spec" + glq add "$gltopic" "$slug" --title "A change that is $slug" --repo "$glrepo" \ + --branch "fix/$slug" --number "$n" >/dev/null + cat >"$tmp/queue-gitlab/$gltopic/$n-$slug/result.md" </dev/null + +out="$(glq collect 2>&1)" +expect "collect verifies a publish claim on a self-hosted GitLab" "01-landed" "$out" +expect "and it read a /-/merge_requests/ URL as a change request" \ + "merge_requests/301" "$out" +refute "its merge request is open, so nothing was reaped" "reaped" "$out" + +python3 - "$gl/mrs/301.json" <<'PY' +import json +import sys +doc = json.load(open(sys.argv[1])) +doc["state"] = "merged" +json.dump(doc, open(sys.argv[1], "w")) +PY +out="$(glq reap 2>&1)" +expect "a merged merge request lands the task" "landed" "$out" +expect "and releases the session that produced it" "reaped" "$out" + +out="$(glq shepherd --topic "$gltopic" --dry-run 2>&1)" +expect "shepherd names the self-hosted project, subgroup and all" \ + "acme/group/widgets on gitlab.example.com" "$out" +expect "and says what it would run, in glab's own flags" \ + "glab mr merge --yes --auto-merge=false --squash" "$out" + +out="$(glq shepherd --topic "$gltopic" 2>&1)" +if grep -q 'mr merge 303' "$gl/merged.log"; then + pass "a green, attested merge request is merged through the adapter" +else + fail "a green, attested merge request is merged through the adapter" \ + "$out$nl$(cat "$gl/merged.log")" +fi +expect "and the merge names the exact head it checked, so a race cannot slip in" \ + "--sha 0000000000000000000000000000000000000303" "$(cat "$gl/merged.log")" +expect "a conflicting one gets a fixer, in the base branch's own terms" \ + "conflicts with main" "$out" +expect "one whose head is in another project is left alone" "left-alone" "$out" +refute "and is never merged" "mr merge 304" "$(cat "$gl/merged.log")" + +# Every call carried the host. This is the whole self-hosted claim: a slug +# would have reached gitlab.com, and `RepoId` is host plus path for this reason. +refute "no call was ever aimed at gitlab.com" "gitlab.com" "$(cat "$gl/calls.log")" +expect "every call named the self-hosted instance by full URL" \ + "-R https://gitlab.example.com/acme/group/widgets" "$(cat "$gl/calls.log")" + +# --- 14c. a project that forbids squash is a refusal, not a crash ------------ +# +# GitLab's `squash` is not a merge method — it is a flag on the merge, and +# `squash_option: never` is the PROJECT setting that forbids it. So the +# mismatch section 13d proves at the forge level has a second form here, per +# project, and it must still be a sentence fleet records rather than a merge +# by whatever method the project does allow. + +printf '{"id": 42, "path_with_namespace": "acme/group/widgets", "squash_option": "never"}\n' \ + >"$gl/api/project.json" +glab_mr 305 'source_branch="fix/green"' +before="$(wc -l <"$gl/merged.log")" +out="$(glq shepherd --topic "$gltopic" 2>&1)" +expect "a project configured against squash says so in its own words" \ + "squash_option: never" "$out" +count_is "and nothing is merged while it forbids it" "$(wc -l <"$gl/merged.log")" \ + "$before" "$out" +printf '{"id": 42, "path_with_namespace": "acme/group/widgets", "squash_option": "default_on"}\n' \ + >"$gl/api/project.json" + +if [ -s "$gl/gh-calls.log" ]; then + fail "no code path ran \`gh\` against a GitLab merge request" \ + "$(cat "$gl/gh-calls.log")" +else + pass "no code path ran \`gh\` against a GitLab merge request" +fi + +# --- 14d. the remote-host probe asks the REPOSITORY's forge, not github.com -- +# +# The credential probe is plain shell that runs on somebody else's machine, so +# it is run here exactly as that machine runs it, with `git` and `ssh` stubbed. +# It used to name github.com flatly, which passes on a host that cannot reach +# the GitLab instance the checkout actually pushes to. + +probe="$tmp/probe" +mkdir -p "$probe/bin" +python3 - "$PWD/scripts/lib/queue.py" >"$probe/probe.sh" <<'PY' +import importlib.util +import sys + +spec = importlib.util.spec_from_file_location("fleet_queue_probe", sys.argv[1]) +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) +sys.stdout.write(mod.forge_probe("/srv/code/app")) +PY +cat >"$probe/bin/git" <<'SH' +#!/bin/sh +cat "$PROBE_ORIGIN" 2>/dev/null || exit 1 +SH +cat >"$probe/bin/ssh" <<'SH' +#!/bin/sh +echo "ssh $*" >>"$PROBE_LOG" +cat "$PROBE_BANNER" 2>/dev/null +exit 1 +SH +chmod +x "$probe/bin/git" "$probe/bin/ssh" + +probe_says() { + : >"$probe/ssh.log" + printf '%s\n' "$1" >"$probe/origin" + printf '%s\n' "$2" >"$probe/banner" + env PATH="$probe/bin:$base_path" PROBE_ORIGIN="$probe/origin" \ + PROBE_BANNER="$probe/banner" PROBE_LOG="$probe/ssh.log" \ + sh "$probe/probe.sh" +} + +expect "the probe proves a GitLab host against GitLab's own welcome" \ + "gitlab.example.com with an ssh key" \ + "$(probe_says 'git@gitlab.example.com:acme/group/widgets.git' \ + 'Welcome to GitLab, @letur!')" +expect "and it asked THAT host, not github.com" "git@gitlab.example.com" \ + "$(cat "$probe/ssh.log")" +expect "a self-hosted instance on a port is asked on that port" "-p 2222" \ + "$(probe_says 'ssh://git@gitlab.example.com:2222/acme/widgets.git' \ + 'Welcome to GitLab, @letur!' >/dev/null; cat "$probe/ssh.log")" +expect "GitHub's own banner still passes, unchanged" \ + "github.com with an ssh key" \ + "$(probe_says 'git@github.com:Thurbeen/fleet.git' \ + "Hi letur! You've successfully authenticated")" +expect "a repo whose origin cannot be read says THAT, not 'no credentials'" \ + "has no readable" \ + "$(probe_says '' 'Welcome to GitLab, @letur!' 2>&1)" + + +# The fixer above got a real worktree; take it back off the test repo, as +# section 13 does with its own. +git -C "$glrepo" worktree remove --force \ + "$tmp/queue-gitlab/.worktrees/${gltopic}__02-conflicting" 2>/dev/null + # The fixer above got a real worktree; take it back off the test repo so the # temp directory can be removed without leaving a stale registration. git -C "$frepo" worktree remove --force \ diff --git a/scripts/queue.sh b/scripts/queue.sh index 3e3d8dc..0086a22 100755 --- a/scripts/queue.sh +++ b/scripts/queue.sh @@ -45,8 +45,9 @@ # `no-mistakes`, `pr` or `push`, which name what the work must # LEAVE BEHIND rather than which tool made it. So "publish the way # your brief says" stops being an unverifiable instruction about a -# method: collect asks the forge for a pull request from this -# task's own branch (and, for `no-mistakes`, an attestation for the +# method: collect asks the forge for a change request — a pull +# request on GitHub, a merge request on GitLab — from this task's +# own branch (and, for `no-mistakes`, an attestation for the # commit that would merge), or asks git whether a `push` task's # commit reached the base branch. An artifact that is not there is # reported and the task is left OPEN; a check that could not run @@ -55,14 +56,14 @@ # `--how`: free text rendered into the brief and never parsed, # which is what lets a task name a publisher fleet has never heard # of. `add` takes both, defaulting to POLICY.md's frontmatter. -# `reap` asks the FORGE whether each concluded task's pull request has +# `reap` asks the FORGE whether each concluded task's change request has # merged, moves the ones that did to `landed`, and only then # deletes their sessions and worktrees. A `push` task has nothing # left to ask by this point — `collect` already confirmed its # commit reached the base branch before closing it — so it lands # in the same run reap follows. `collect` runs it, because # "delete each session as it closes out" was a documented MANUAL -# step and twenty gigabytes sat in a worktree whose pull request +# step and twenty gigabytes sat in a worktree whose change request # had merged the day before. It never touches a session thurbox # says is working, and never one a worker gave up in — that # session is the evidence. @@ -92,21 +93,22 @@ # read is `not checked` rather than a silent no. It adds no daemon, no poll and # no state: `collect` is still the only thing that closes a task. # -# AND THEN THE PULL REQUEST OUTLIVES THE TASK, which is what `shepherd` is for: +# AND THEN THE CHANGE REQUEST OUTLIVES THE TASK, which is what `shepherd` is for: # -# `shepherd` asks the FORGE for every open PR on the repos this queue's -# tasks name, DISPATCHES A FIXER for one that conflicts, fails a -# check, has a review asking for changes, or was declared +# `shepherd` asks the FORGE for every open change request on the repos this +# queue's tasks name, DISPATCHES A FIXER for one that conflicts, +# fails a check, has a review asking for changes, or was declared # `no-mistakes` and carries no attestation — and squash-merges # one that clears every gate. It is a fourth thing, after both # halves of completion. # # It reads the forge and not the task records because a task -# records ONE artifact, the first PR its worker reported: #25 was -# a SECOND pull request from a task still pointing at the merged -# #23, and a PR opened outside the queue was invisible the same -# way. A PR no task records is shepherded like any other; it just -# has no session to send a fixer into, and that is said out loud. +# records ONE artifact, the first change request its worker +# reported: #25 was a SECOND pull request from a task still +# pointing at the merged #23, and a PR opened outside the queue +# was invisible the same way. One no task records is shepherded +# like any other; it just has no session to send a fixer into, +# and that is said out loud. # # AND A FIFTH THING, WHICH IS FUEL. A worker that hits its agent's token limit # does not fail — it SITS. The hook that would have said `idle` never fires, so @@ -137,14 +139,15 @@ # # Three rules make it safe to run, and `--dry-run` shows all of them: # IDEMPOTENT the fixer it sent is recorded on the task; a second pass sees -# work in flight rather than a still-broken PR. `--force` to mean -# it anyway. +# work in flight rather than a still-broken change request. +# `--force` to mean it anyway. # NEVER GUESS no forge, no network, no thurbox — it says what it could not -# determine and carries on. A PR it could not read is never -# called broken, and never called ready. +# determine and carries on. A change request it could not read +# is never called broken, and never called ready. # NEVER TOUCH only artifacts recorded on this queue's own tasks, and it # A STRANGER merges only in the repos AUTO_MERGE_REPOS names, each of -# which names its forge (`github.com/owner/repo`). +# which names its forge (`github.com/owner/repo`, +# `gitlab.example.com/group/project`). # # Usage: # scripts/queue.sh topic add --title T --prompt 'the ask' # or --prompt-file F|- @@ -178,7 +181,7 @@ # scripts/queue.sh reap [--dry-run] # land what merged, release its session # scripts/queue.sh refuel [] [--dry-run] # the account's fuel first, then # restart the workers that ran dry against it -# scripts/queue.sh shepherd [--dry-run] # every open PR on the repo: fix or merge +# scripts/queue.sh shepherd [--dry-run] # every open change request on the repo: fix or merge # [--json] [--topic T] [--ref R] [--no-merge] [--force] # scripts/queue.sh run [] # refresh the run log(s) by hand # scripts/queue.sh list [--topic T] [--archived] [--all] # the lead's view: @@ -209,10 +212,14 @@ # because it injects into the lead's terminal whatever machine it comes from. # # Three probes run before anything is spawned, and one failure stops that task -# where it stands: the host answers ssh as a POSIX shell, it has GitHub -# credentials OF ITS OWN, and the repo is a checkout at that path. Fleet never -# sends credentials anywhere. POSIX hosts only — a Windows host (hosts.toml -# spells one with a non-tmux `multiplexer`) is refused by name. +# where it stands: the host answers ssh as a POSIX shell, the repo is a +# checkout at that path, and it has credentials OF ITS OWN for the forge that +# repo's `origin` names — GitHub or GitLab. The repo is asked about before its +# forge because which forge to prove a credential against is a fact about that +# checkout's origin, so it cannot be asked before the checkout is known to +# exist. Fleet never sends credentials anywhere. POSIX hosts only — a Windows +# host (hosts.toml spells one with a non-tmux `multiplexer`) is refused by +# name. # # THE RUN LOG IS PRODUCED, NOT REMEMBERED. `AGENTS.md` step 5 used to say # "record the run in orchestration/runs/ as it happens", and two consecutive @@ -284,9 +291,10 @@ # # Requires: python3 (with PyYAML) — the same dependency the rest of the gate # has. `dispatch`, `watch`, `reap` and `refuel` additionally need thurbox-cli, -# and `collect`, `reap` and `shepherd` ask the FORGE about a pull request — -# whichever `scripts/lib/forge.py` has configured, `gh` for GitHub — and -# `shepherd` needs git as well. `refuel` reads the account's quota window with +# and `collect`, `reap` and `shepherd` ask the FORGE about a change request — +# whichever `scripts/lib/forge.py` has configured: `gh` for GitHub, `glab` for +# GitLab — and `shepherd` needs git as well. `refuel` reads the account's +# quota window with # `quota-axi` (https://github.com/kunchenguid/quota-axi), which fleet neither # installs nor sends any credential to. A task that names a `--host` # additionally needs `ssh`. Every one of those degrades to "could not check"