feat(runner): perform a sequence of actions in one request with runner actions

.changeset/runner-actions-sequence.md

@@ -0,0 +1,5 @@
+---
+"@qawolf/cli": minor
+---
+
+Add `qawolf runner actions`, which performs a sequence of up to ten browser actions on a runner in one request. The answer says what happened to each action, where the sequence stopped and why, and saves a screenshot after the last action or after every action when asked.

skills/qawolf-cli/SKILL.md

@@ -132,6 +132,11 @@ that `url`; never guess a route and never send a repository link in its place.
 | `qawolf auth switch` | local | Choose which workspace to work in |
 | `qawolf auth whoami` | read | Show authentication status |
 | `qawolf automate` | write | Request automation for draft flows. First create a named local .flow.ts draft for every requested journey that does not already have a matching draft; never reuse a generic starter or placeholder. Each new draft must start with a JSDoc Goal: description, import flow from @qawolf/flows/web, and use export default flow(...); a comment-only file or direct test(...) call is not a valid draft. Commit and push all changes with Git to publish them, then list remote drafts to resolve every selected ID. Do not use patch to create or rename a selected flow. Finally make one automation request containing all requested flow IDs. |
+| `qawolf codeHostIntegration find` | read | List the workspace's code host integrations (GitHub or GitLab). A connected integration lets QA Wolf read repositories and receive the code host's webhooks. codeHostIntegration.listRepositories pages the repositories each integration covers. |
+| `qawolf codeHostIntegration listRepositories` | read | List the repositories the workspace's code host integrations cover, alphabetical by full name. The list reflects the last sync from the code host; codeHostIntegration.find lists the integrations themselves. |
+| `qawolf deployment find` | read | List the deployments QA Wolf has received for the workspace, newest first. A deployment arrives through a code host integration's webhook or through deployment.reportStatus, and is what a deployment trigger evaluates against. deployment.listTriggerEvaluations reports each trigger's verdict for one deployment. |
+| `qawolf deployment listTriggerEvaluations` | read | List the per-trigger verdicts recorded when a deployment was evaluated against the workspace's triggers. Each verdict says whether the trigger matched, did not match with the reason per condition, or was never considered and why. Verdicts are a snapshot from evaluation time: editing or deleting a trigger later does not change them. |
+| `qawolf deployment reportStatus` | write | Report a deployment lifecycle status. A deployment's first success report evaluates global triggers asynchronously, and it is the only report that does: a trigger added or unpaused later does not make an already reported deployment evaluate again. Report that deployment under a new providerDeploymentId to evaluate it against the current triggers. The response contains the deployment only, not the resulting runs. This route requires an organization or user API key. |
 | `qawolf doctor` | local | Diagnose problems running flows locally |
 | `qawolf email find` | read | List the workspace's inbox, or its sent mail, newest first. Read a message body with email.get. |
 | `qawolf email get` | read | Read one email of the workspace, with its plain text and HTML bodies. Use it to pull a sign-in code or a verification link out of a message. |
@@ -173,6 +178,7 @@ that `url`; never guess a route and never send a repository link in its place.
 | `qawolf run reattempt` | write | Request new attempts for a run's flows, in the same run. A flow is eligible once its result is failed or canceled and QA Wolf's automatic retries have finished. A fully investigated run no longer accepts reattempts. Attempts run with the latest flow code. Poll run.get for results. |
 | `qawolf run stop` | write | Stop a run, including its queued flows and automatic retries. Stopping is asynchronous and can update run-status messages and commit statuses in connected integrations. Repeated requests are safe, and finished runs keep their results. A run that is still being created returns not found; retry once run.get returns the run. If run.get returns a different runId, use that ID. Poll run.get for results. |
 | `qawolf runner act` | write | Perform one raw action on a runner's screen: click, double_click, scroll, move, drag, keypress, navigate or type. Use - to read a whole action as JSON from stdin. On a mobile runner only click (button left), drag and type have a touchscreen equivalent; the rest answer action-not-supported-on-mobile |
+| `qawolf runner actions` | write | Perform a sequence of up to ten raw actions on a runner's screen in one request, as a JSON array of the same actions `runner act` takes. Use - to read the array from stdin. Actions run back to back, so batch only steps whose targets are on the screen you last saw, and end the sequence at the step that changes the page. Each result carries an effect: performed, not-performed, or unknown when the runner stopped answering and the action may have landed |
 | `qawolf runner events` | read | Print a runner's journal, one entry per line. QA Wolf writes console, recorder, run-events, run-logs, run-status |
 | `qawolf runner exec` | write | Evaluate a snippet against a runner's live page. Use - to read the snippet from stdin |
 | `qawolf runner highlight-selector` | write | Highlight what a selector matches on a runner's live page, so the next screenshot shows it. Omit the selector to clear the highlight |

src/commands/__snapshots__/help.test.ts.snap

@@ -38,6 +38,8 @@ Commands:
                                Do not use patch to create or rename a selected
                                flow. Finally make one automation request
                                containing all requested flow IDs.
+  codeHostIntegration          QA Wolf public API codeHostIntegration commands
+  deployment                   QA Wolf public API deployment commands
   email                        QA Wolf public API email commands
   environment                  QA Wolf public API environment commands
   file                         QA Wolf public API file commands
@@ -303,6 +305,7 @@ Commands:
   events [options] <stream>                Print a runner's journal, one entry per line. QA Wolf writes console, recorder, run-events, run-logs, run-status
   screenshot [options]                     Save a JPEG of an interactive runner's screen to a file, or write it to stdout with --out -
   act [options] <action>                   Perform one raw action on a runner's screen: click, double_click, scroll, move, drag, keypress, navigate or type. Use - to read a whole action as JSON from stdin. On a mobile runner only click (button left), drag and type have a touchscreen equivalent; the rest answer action-not-supported-on-mobile
+  actions [options] <sequence>             Perform a sequence of up to ten raw actions on a runner's screen in one request, as a JSON array of the same actions \`runner act\` takes. Use - to read the array from stdin. Actions run back to back, so batch only steps whose targets are on the screen you last saw, and end the sequence at the step that changes the page. Each result carries an effect: performed, not-performed, or unknown when the runner stopped answering and the action may have landed
   exec [options] <file>                    Evaluate a snippet against a runner's live page. Use - to read the snippet from stdin
   inspect                                  Read one thing off a runner's live page (browser) or Appium session (mobile)
   import-package [options] <name>          Install a package into a runner's live run, so a snippet or a selection can import it

src/commands/runner/actions.register.ts

@@ -0,0 +1,72 @@
+import type { Command } from "commander";
+
+import { declareCommandKind } from "~/commands/commandKind.js";
+import { withAuthContext } from "~/commands/context.js";
+import { handleRunnerActions } from "~/domains/interactiveRunner/performActions.js";
+import type { SignalRegistry } from "~/shell/signals/createSignalRegistry.js";
+
+import { runnerDeps, runnerFlagDescription } from "./context.js";
+
+const actionsExamples = `
+Examples:
+  $ qawolf runner actions '[{"type":"click","button":"left","x":480,"y":260},{"type":"type","text":"hello@example.com"},{"type":"keypress","keys":["Enter"]}]' --screenshot after-login.jpg
+  $ echo '[{"type":"click","button":"left","x":1,"y":2},{"type":"type","text":"hi"}]' | qawolf runner actions -
+  $ qawolf runner actions '[...]' --screenshot-mode each --screenshot step.jpg
+  $ qawolf runner actions '[...]' --continue-on-failure`;
+
+type ActionsFlags = {
+  continueOnFailure?: boolean;
+  runner?: string;
+  screenshot?: string;
+  screenshotMode?: string;
+};
+
+function toScreenshotMode(
+  flag: string | undefined,
+): "each" | "final" | "none" | undefined {
+  return flag === "each" || flag === "final" || flag === "none"
+    ? flag
+    : undefined;
+}
+
+export function registerRunnerActionsCommand(
+  runner: Command,
+  signals: SignalRegistry,
+): void {
+  // One request for the steps a caller already knows, in place of one per step.
+  // The frames come back as files, never inline: a sequence's worth of base64
+  // is not something a terminal or a JSON reader wants.
+  declareCommandKind(runner.command("actions <sequence>"), "write")
+    .description(
+      "Perform a sequence of up to ten raw actions on a runner's screen in one request, as a JSON array of the same actions `runner act` takes. Use - to read the array from stdin. Actions run back to back, so batch only steps whose targets are on the screen you last saw, and end the sequence at the step that changes the page. Each result carries an effect: performed, not-performed, or unknown when the runner stopped answering and the action may have landed",
+    )
+    .option(
+      "--continue-on-failure",
+      "Carry on past an action that reached the runner and did not take effect; a runner that cannot be reached or a screen that cannot serve still ends the sequence",
+    )
+    .option("--runner <id>", runnerFlagDescription)
+    .option(
+      "--screenshot <path>",
+      "Save a JPEG of the screen after the last action to this file. With --screenshot-mode each, one file per action, with the action's index before the extension. - writes the final frame to stdout and moves the confirmation to stderr",
+    )
+    .option(
+      "--screenshot-mode <mode>",
+      "none, final or each. Defaults to final when --screenshot is given, none otherwise",
+    )
+    .addHelpText("after", actionsExamples)
+    .action((sequence: string, opts: ActionsFlags, command: Command) =>
+      withAuthContext(signals, (ctx) =>
+        handleRunnerActions(
+          ctx,
+          {
+            actions: sequence,
+            continueOnFailure: opts.continueOnFailure === true,
+            runner: opts.runner,
+            screenshot: opts.screenshot,
+            screenshotMode: toScreenshotMode(opts.screenshotMode),
+          },
+          runnerDeps(ctx),
+        ),
+      )(opts, command),
+    );
+}

src/commands/runner/index.ts

@@ -2,6 +2,7 @@ import type { Command } from "commander";
 
 import type { SignalRegistry } from "~/shell/signals/createSignalRegistry.js";
 
+import { registerRunnerActionsCommand } from "./actions.register.js";
 import { registerRunnerExecCommand } from "./exec.register.js";
 import { registerRunnerHighlightSelectorCommand } from "./highlightSelector.register.js";
 import { registerRunnerImportPackageCommand } from "./importPackage.register.js";
@@ -24,6 +25,7 @@ export function registerRunnerCommand(
   registerRunCommand(runner, signals);
   registerRunnerEventsCommand(runner, signals);
   registerRunnerInteractCommands(runner, signals);
+  registerRunnerActionsCommand(runner, signals);
   registerRunnerExecCommand(runner, signals);
   registerRunnerInspectCommands(runner, signals);
   registerRunnerImportPackageCommand(runner, signals);

src/core/messages/interactiveRunner/index.ts

@@ -1,4 +1,5 @@
 import { interactMessages } from "./interact.js";
+import { sequenceMessages } from "./sequence.js";
 import { lifecycleMessages } from "./lifecycle.js";
 import { listMessages } from "./list.js";
 import { runMessages } from "./run.js";
@@ -8,4 +9,5 @@ export const interactiveRunnerMessages = {
   ...listMessages,
   ...runMessages,
   ...interactMessages,
+  ...sequenceMessages,
 } as const;

src/core/messages/interactiveRunner/sequence.ts

@@ -0,0 +1,27 @@
+/** What `qawolf runner actions` says about a sequence. */
+export const sequenceMessages = {
+  actionsEmpty:
+    'The sequence holds no actions. Pass a JSON array of at least one, for example \'[{"type":"click","button":"left","x":480,"y":260},{"type":"type","text":"hello"}]\'.',
+  actionsInvalidAt: (index: number, error: string) =>
+    `Action ${index} in the sequence was refused: ${error}`,
+  actionsNotJsonArray:
+    'The sequence must be a JSON array of actions, for example \'[{"type":"click","button":"left","x":480,"y":260},{"type":"type","text":"hello"}]\'.',
+  actionsEachNeedsAPath:
+    "--screenshot-mode each writes one frame per action, so it needs --screenshot <path>; each frame goes to that path with the action's index before the extension.",
+  actionsEachToStdout:
+    "--screenshot-mode each writes one frame per action, which stdout cannot carry. Give --screenshot a file path.",
+  actionsPerformed: (count: number) =>
+    `Performed ${count} ${count === 1 ? "action" : "actions"}.`,
+  actionsPerformedScreenshotWritten: (count: number, path: string) =>
+    `Performed ${count} ${count === 1 ? "action" : "actions"} and wrote the runner's screen to ${path}.`,
+  actionsStoppedAt: (index: number, type: string, why: string) =>
+    `Action ${index} (${type}) ${why}`,
+  actionsLeftUnperformed: (count: number) =>
+    `${count} ${count === 1 ? "action" : "actions"} after it ${count === 1 ? "was" : "were"} not attempted.`,
+  actionsMayHaveHappened:
+    "It may have taken effect: take a screenshot before repeating anything from it on.",
+  actionsOutOfTime:
+    "was not reached before the sequence ran out of its time. Send the rest as a new request.",
+  actionsUnconfirmed: (errorMessage: string) =>
+    `has an unknown effect: ${errorMessage}`,
+};

src/domains/interactiveRunner/performActions.test.ts

@@ -0,0 +1,114 @@
+import { publicContractsV1 } from "@qawolf/api-contracts/v1";
+import { describe, expect, it } from "bun:test";
+
+import { makeAuthCtx, makeTestDeps } from "./deps.testUtils.js";
+import { handleRunnerActions } from "./performActions.js";
+import { sequenceCallOptions } from "./sequenceCallOptions.js";
+
+const aClick = { button: "left", type: "click", x: 480, y: 260 };
+const someTyping = { text: "hello@example.com", type: "type" };
+const anEnter = { keys: ["Enter"], type: "keypress" };
+const sequence = JSON.stringify([aClick, someTyping, anEnter]);
+
+const performed = (index: number) => ({
+  effect: "performed",
+  index,
+  outcome: "success",
+});
+
+describe("handleRunnerActions", () => {
+  it("sends the sequence in one request, asking for no frame when none is wanted", async () => {
+    const { callPublicApi, ctx, outputs } = makeAuthCtx();
+    callPublicApi.mockResolvedValue({
+      ok: true,
+      value: {
+        lastCompletedIndex: 2,
+        outcome: "success",
+        results: [performed(0), performed(1), performed(2)],
+      },
+    });
+
+    const result = await handleRunnerActions(
+      ctx,
+      {
+        actions: sequence,
+        continueOnFailure: false,
+        runner: "ci",
+        screenshot: undefined,
+        screenshotMode: undefined,
+      },
+      makeTestDeps(),
+    );
+
+    expect(result).toBeUndefined();
+    expect(callPublicApi).toHaveBeenCalledWith(
+      publicContractsV1.runner.performActions,
+      {
+        actions: [aClick, someTyping, anEnter],
+        id: "ci",
+        screenshotMode: "none",
+        stopOnFailure: true,
+      },
+      sequenceCallOptions,
+    );
+    expect(outputs().at(-1)?.humanMessage).toBe("Performed 3 actions.");
+  });
+
+  it("reads the sequence from stdin with -", async () => {
+    const { callPublicApi, ctx } = makeAuthCtx();
+    callPublicApi.mockResolvedValue({
+      ok: true,
+      value: {
+        lastCompletedIndex: 0,
+        outcome: "success",
+        results: [performed(0)],
+      },
+    });
+
+    await handleRunnerActions(
+      ctx,
+      {
+        actions: "-",
+        continueOnFailure: true,
+        runner: "ci",
+        screenshot: undefined,
+        screenshotMode: undefined,
+      },
+      makeTestDeps({ readStdin: async () => `${JSON.stringify([aClick])}\n` }),
+    );
+
+    expect(callPublicApi).toHaveBeenCalledWith(
+      publicContractsV1.runner.performActions,
+      {
+        actions: [aClick],
+        id: "ci",
+        screenshotMode: "none",
+        stopOnFailure: false,
+      },
+      sequenceCallOptions,
+    );
+  });
+
+  it("refuses a sequence with one bad action locally, naming its index", async () => {
+    const { callPublicApi, ctx } = makeAuthCtx();
+
+    const result = await handleRunnerActions(
+      ctx,
+      {
+        actions: JSON.stringify([
+          aClick,
+          { text: "a".repeat(201), type: "type" },
+        ]),
+        continueOnFailure: false,
+        runner: "ci",
+        screenshot: undefined,
+        screenshotMode: undefined,
+      },
+      makeTestDeps(),
+    );
+
+    expect(result?.exitCode).toBe(2);
+    expect(result?.error).toContain("Action 1 in the sequence was refused");
+    expect(callPublicApi).not.toHaveBeenCalled();
+  });
+});