Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ _Avoid_: Selector, locator, query
A Runtime Target match produced by applying a Finder during a Scenario run. A valid Finder resolution requires exactly one Finder Match; zero matches or multiple matches fail the step, and action-specific capabilities such as tapping, typing, or scrolling are validated when the action executes.
_Avoid_: First match, best match

**Finder Action Budget**:
The total time available for a Finder-backed action to synchronize with a Flutter frame and resolve exactly one Finder Match. `tap`, `type`, and targeted `scroll` use the runner's default 3000ms budget; `waitFor.timeoutMs` supplies the budget for that WaitFor Action. Frame synchronization consumes the same budget rather than adding a separate timeout.
_Avoid_: Finder timeout plus frame timeout, retry count

**Runtime Handle**:
An opaque runtime token returned with a Finder Match and accepted back by the Runtime Adapter for the immediately following action. Flutter Pilot may record it for diagnostics, but must not parse it, construct it, or treat it as stable identity.
_Avoid_: Widget id, key, Inspector id, stable reference
Expand Down Expand Up @@ -129,7 +133,7 @@ An action that waits until a Finder produces exactly one match. It does not wait
_Avoid_: Wait assertion, sleep

**Scroll Action**:
An action that moves a scrollable area by configured gesture drag deltas. It may target a specific scrollable with a Finder, or use the primary scrollable when no Finder is provided.
An action that moves a scrollable area by configured gesture drag deltas. It may target a specific scrollable with a Finder, or select the unique outermost visible scrollable on the dominant drag axis when no Finder is provided.
_Avoid_: Swipe

**Screenshot**:
Expand Down
8 changes: 4 additions & 4 deletions docs-internal/flutter-pilot-prd.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,12 +141,12 @@ The result is a reproducible bug report package that can be consumed by humans,
- `byKey` is not part of the current Scenario DSL because the calibrated `pilot_runtime` semantic Snapshot path does not expose Flutter key values reliably. Key-based Finders may be added later if the Runtime Adapter can obtain stable key data.
- `byType` accepts the `pilot_runtime` semantic Snapshot node type, such as `textField`, `button`, `text`, `scrollable`, or `header`. It does not accept Dart widget class names such as `TextField`, `FilledButton`, or app-defined wrapper widget classes.
- `byText` matches exact visible text. It does not perform contains, fuzzy, or regular expression matching in the first version.
- A Finder must resolve to exactly one widget before an action can execute. Zero matches fail the step as "Finder matched no widgets"; multiple matches fail the step as "Finder matched multiple widgets." Flutter Pilot does not automatically choose the first match.
- A Finder must resolve to exactly one widget before an action can execute. Zero matches keep polling until the Finder Action Budget expires; multiple matches fail the Step as "Finder matched multiple widgets." Flutter Pilot does not automatically choose the first match.
- The initial action set includes `tap`, `type`, `scroll`, `waitFor`, and `capture`.
- The `type` action means replacing text in a widget: clear existing text directly, then enter the configured text one character at a time. It is distinct from the `byType` Finder constraint.
- The `waitFor` action waits for a Finder to produce exactly one match before its timeout. Zero matches keep waiting until timeout, one match succeeds, and multiple matches fail the step. The first version does not support waiting for disappearance, enabled state, or disabled state.
- `waitFor.timeoutMs` defaults to `3000` when omitted. The first version supports per-step timeout overrides but no global timeout defaults in the Scenario.
- The `scroll` action accepts `deltaX` and `deltaY` as gesture drag deltas in logical pixels. Omitted deltas default to `0`. For example, `deltaY: -500` means dragging upward by 500 logical pixels, which usually reveals lower content. A Finder is optional for `scroll`; when omitted, Flutter Pilot scrolls the primary scrollable. When provided, the Finder must resolve to exactly one scrollable target. At least one of `deltaX` or `deltaY` must be non-zero, so `scroll: {}` and zero-delta scrolls are invalid.
- Finder-backed `tap`, `type`, and targeted `scroll` actions wait for a Finder to produce exactly one match within a default `3000ms` budget. Before polling, the runner waits up to `500ms` for the current or next Flutter frame; that frame wait consumes the same action budget. Zero matches keep polling every `50ms` until timeout, one match executes, and multiple matches fail the Step immediately.
- The `waitFor` action uses the same frame synchronization and Finder polling behavior. `waitFor.timeoutMs` defaults to `3000` when omitted and replaces that Step's default budget when specified; it does not add another timeout. The first version does not support waiting for disappearance, enabled state, or disabled state, and exposes no global timeout defaults in the Scenario.
- The `scroll` action accepts `deltaX` and `deltaY` as gesture drag deltas in logical pixels. Omitted deltas default to `0`. For example, `deltaY: -500` means dragging upward by 500 logical pixels, which usually reveals lower content. A Finder is optional for `scroll`; when omitted, Flutter Pilot selects the unique outermost visible scrollable matching the dominant drag axis and ignores nested scrollables on that axis. Multiple peer candidates remain ambiguous and require a Finder. When provided, the Finder must resolve to exactly one scrollable target. At least one of `deltaX` or `deltaY` must be non-zero, so `scroll: {}` and zero-delta scrolls are invalid.
- Capture directives support screenshots, semantic snapshots, widget summaries, logs, and labels. Runtime errors are collected as part of logs in the first version.
- Failed steps automatically trigger diagnostic capture even if the YAML did not request a capture at that point.
- Scenario execution produces a run directory containing a structured run report, an HTML timeline report, aggregated Step metadata, and capture artifacts.
Expand Down
6 changes: 4 additions & 2 deletions docs-internal/pilot-runtime-grill-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,10 @@ design memory, not a final PRD.
- `scroll` v1 uses pointer drag gestures and Flutter logical pixel deltas. It
does not use semantic scroll actions.
- Scroll with a Finder validates that the unique match can be scrolled.
- Scroll without a Finder targets the primary scrollable. If the primary
scrollable cannot be uniquely determined, the action fails.
- Scroll without a Finder targets the unique outermost visible scrollable on
the dominant drag axis. Nested scrollables on that axis are excluded from
primary selection and avoided when choosing the drag start; multiple peer
candidates fail as ambiguous.

## Widget Tree And Capture Artifacts

Expand Down
10 changes: 6 additions & 4 deletions docs-internal/pilot-runtime-prd.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ structured UI artifact exposed through `widgetTree`.
22. As a Flutter developer, I want tap to fall back to pointer center taps on calibrated platforms, so that targets without semantic tap actions can still be exercised.
23. As a Flutter developer, I want `type` to clear editable text directly and then enter the configured text character by character without simulating platform keyboard input, so that text entry is deterministic while still exercising per-character text changes.
24. As a Flutter developer, I want scroll deltas to remain Flutter logical pixel drag deltas, so that existing Scenario scroll semantics remain intact.
25. As a Flutter developer, I want untargeted scroll to use the primary scrollable, so that simple scrolling Scenarios remain concise.
26. As a Flutter developer, I want untargeted scroll to fail when the primary scrollable is ambiguous, so that Flutter Pilot does not pick an arbitrary scrollable.
25. As a Flutter developer, I want untargeted scroll to use the unique outermost visible scrollable on the dominant drag axis, so that simple scrolling Scenarios remain concise and nested views do not capture the gesture accidentally.
26. As a Flutter developer, I want untargeted scroll to fail when multiple outermost scrollables remain on the dominant drag axis, so that Flutter Pilot does not pick an arbitrary scrollable.
27. As a Flutter developer, I want Widget Tree capture to use Flutter Inspector summary tree data, so that artifacts preserve useful hierarchy without raw dump noise.
28. As a Flutter developer, I want Widget Tree JSON to be normalized, so that artifacts are stable and not tied to raw Inspector response envelopes.
29. As a Flutter developer, I want Widget Tree artifacts to include schema and source metadata, so that tools can version and interpret them safely.
Expand Down Expand Up @@ -114,7 +114,9 @@ structured UI artifact exposed through `widgetTree`.
- Tap execution prefers semantic tap actions and falls back to pointer center tap on calibrated platforms.
- Text entry supports editable text targets only and replaces existing text. It does not simulate keyboard or IME input.
- Scroll execution uses pointer drag gestures and Flutter logical pixel deltas. It does not use semantic scroll actions.
- Untargeted scroll resolves the primary scrollable and fails when that target cannot be uniquely determined.
- Untargeted scroll chooses the dominant drag axis, resolves the unique outermost visible scrollable on that axis, and fails when multiple peer candidates remain.
- Scroll gesture start selection avoids the bounds of nested scrollables on the same axis so the intended outer scrollable receives the drag.
- Finder-backed actions synchronize with the current or next Flutter frame before polling for a unique Finder Match. Frame synchronization and polling share one action budget; multiple matches fail immediately.
- Remove Snapshot from the new Scenario capture contract, Runtime Adapter contract, print diagnostics, and artifact language.
- Use `widgetTree` as the structured UI capture field, print diagnostic, and report artifact type.
- Write Widget Tree artifacts with a `widget_tree` filename suffix.
Expand Down Expand Up @@ -161,7 +163,7 @@ Major modules to build or modify:
- Runtime client tests should use fake VM Service responses for handshake, protocol mismatch, capability missing, Widget Tree capture, and error mapping.
- Runtime protocol tests should cover response decoding, version validation, capability validation, and structured runtime failures.
- Finder resolution tests inside `pilot_runtime` should use Flutter widget tests for visible matching, offstage exclusion, `byText`, semantic `byType`, `ValueKey<String>`, `byWidget`, strict AND combinations, and wrapper-child subtree evidence.
- Action execution tests inside `pilot_runtime` should use Flutter widget tests for semantic tap, pointer fallback, editable text clear/entry, targeted scroll, and primary scrollable resolution.
- Action execution tests inside `pilot_runtime` should use Flutter widget tests for semantic tap, pointer fallback, editable text clear/entry, targeted scroll, axis-aware outermost scrollable selection, nested-scrollable avoidance, and ambiguous peer scrollables.
- Widget Tree normalizer tests should use recorded Inspector summary tree fixtures and verify normalized schema, source, node fields, child structure, missing optional fields, and rejection of invalid required shape.
- Screenshot tests should verify returned MIME type and bytes shape where the chosen screenshot path can be faked; real screenshot quality should be covered by calibration smoke tests.
- Logs tests should verify the not-implemented payload and that `logs: true` does not fail a capture Step.
Expand Down
19 changes: 13 additions & 6 deletions docs-internal/scenario-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,8 @@ Flutter Pilot supports five Scenario actions: `tap`, `type`, `scroll`,
```

The Finder must resolve to exactly one Finder Match before the tap can execute.
Zero matches fail the Step. Multiple matches fail the Step.
Zero matches keep polling within the default `3000ms` budget. Multiple matches
fail the Step immediately.

### type

Expand All @@ -260,9 +261,10 @@ The action clears existing text and enters the configured text.

`deltaX` and `deltaY` default to `0`, but at least one must be non-zero.

A Finder is optional. When omitted, Flutter Pilot targets the primary
scrollable. When provided, the Finder must resolve to exactly one scrollable
target.
A Finder is optional. When omitted, Flutter Pilot selects the unique outermost
visible scrollable on the dominant drag axis and ignores nested scrollables on
that axis. Multiple peer candidates are ambiguous. When provided, the Finder
must resolve to exactly one scrollable target.

```yaml
- label: scroll_results
Expand All @@ -284,8 +286,13 @@ target.

`timeoutMs` is optional and defaults to `3000`.

`waitFor` waits until the Finder produces exactly one match. Zero matches keep
waiting until timeout. Multiple matches fail the Step.
`waitFor` first waits up to `500ms` for the current or next Flutter frame, then
polls every `50ms` until the Finder produces exactly one match. Frame
synchronization and polling share the `timeoutMs` budget. Zero matches keep
waiting until timeout. Multiple matches fail the Step immediately.

Finder-backed `tap`, `type`, and targeted `scroll` use the same synchronization
and polling behavior with a default `3000ms` budget.

### capture

Expand Down
5 changes: 5 additions & 0 deletions docs/guide/write-scenario.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ steps:

When a Finder has multiple fields, every configured field must match.

Finder-backed actions tolerate UI that appears asynchronously: Flutter Pilot
waits for a Flutter frame and polls for one unique match within the action's
timeout budget. Multiple matches fail immediately instead of selecting the
first match.

## Capture diagnostics

`capture` is a Step action. Use it as its own Step when you want the default
Expand Down
12 changes: 12 additions & 0 deletions docs/reference/scenario-dsl.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,12 @@ steps:
When several Finder fields are present, all constraints must match. Finder
fields are single strings.

Finder-backed actions first wait up to 500ms for the current or next Flutter
frame, then poll every 50ms until exactly one match is available. The frame wait
and polling share one total budget. `tap`, `type`, and targeted `scroll` use a
3000ms budget; `waitFor.timeoutMs` sets the budget for that `waitFor` Step.
Multiple matches fail immediately.

## Actions

Flutter Pilot supports these Scenario actions:
Expand Down Expand Up @@ -155,6 +161,12 @@ steps:
deltaY: -500
```

The dominant drag axis is vertical when `abs(deltaY) >= abs(deltaX)` and
horizontal otherwise. Without a Finder, Flutter Pilot selects the unique
outermost visible scrollable on that axis and avoids starting the gesture over
a nested scrollable on the same axis. Multiple peer scrollables are ambiguous;
add a Finder to choose one explicitly.

### waitFor

```yaml scenario
Expand Down
70 changes: 46 additions & 24 deletions lib/src/execution/scenario_runner.dart
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import '../target/target_device.dart';
///
/// It contains:
/// - the Runtime Adapter used for UI operations and captures
/// - the shared Finder Action Budget for tap, type, and targeted scroll
/// - the output directory where run artifacts are written
///
/// During `run`, it:
Expand All @@ -28,14 +29,17 @@ class ScenarioRunner {
required this.adapter,
this.recordingController,
this.targetDevice,
this.finderTimeout = const Duration(seconds: 3),
required this.outputDirectory,
});

static const Duration _waitForPollInterval = Duration(milliseconds: 50);
static const Duration _endOfFrameTimeout = Duration(milliseconds: 500);

final RuntimeAdapter adapter;
final RecordingController? recordingController;
final TargetDevice? targetDevice;
final Duration finderTimeout;
final Directory outputDirectory;

/// Execute Scenario Steps and write a run report.
Expand Down Expand Up @@ -577,6 +581,7 @@ class ScenarioRunner {
final FinderMatch match = await _resolveUniqueMatch(
finder,
actionName: actionName,
timeout: finderTimeout,
);
await operation(match);
return const _ActionExecutionResult();
Expand All @@ -586,50 +591,63 @@ class ScenarioRunner {
Future<FinderMatch> _resolveUniqueMatch(
Finder finder, {
required String actionName,
}) async {
final List<FinderMatch> matches = await adapter.resolveFinder(finder);
if (matches.isEmpty) {
throw _StepFailureException(
actionName: actionName,
message: 'Finder matched no widgets.',
);
}
if (matches.length > 1) {
throw _StepFailureException(
actionName: actionName,
message: 'Finder matched multiple widgets.',
);
}
return matches.single;
}

/// Poll a Finder until it has one unique match or the timeout expires.
Future<_ActionExecutionResult> _waitFor(
Finder finder, {
required Duration timeout,
}) async {
final Stopwatch stopwatch = Stopwatch()..start();
await _waitForEndOfFrame(stopwatch: stopwatch, timeout: timeout);
while (true) {
if (stopwatch.elapsed >= timeout) {
throw _StepFailureException(
actionName: actionName,
message: 'Finder matched no widgets before timeout.',
);
}
final List<FinderMatch> matches = await adapter.resolveFinder(finder);
if (matches.length == 1) {
return const _ActionExecutionResult();
return matches.single;
}
if (matches.length > 1) {
throw _StepFailureException(
actionName: 'waitFor',
actionName: actionName,
message: 'Finder matched multiple widgets.',
);
}
if (stopwatch.elapsed >= timeout) {
throw _StepFailureException(
actionName: 'waitFor',
actionName: actionName,
message: 'Finder matched no widgets before timeout.',
);
}
await Future<void>.delayed(_waitForPollInterval);
}
}

/// Wait for one frame without exceeding the Finder Action's total budget.
Future<void> _waitForEndOfFrame({
required Stopwatch stopwatch,
required Duration timeout,
}) async {
final Duration remaining = timeout - stopwatch.elapsed;
if (remaining <= Duration.zero) {
return;
}
final Duration frameTimeout = remaining < _endOfFrameTimeout
? remaining
: _endOfFrameTimeout;
await adapter
.waitForEndOfFrame(timeout: frameTimeout)
.timeout(frameTimeout, onTimeout: () {});
}

/// Poll a Finder until it has one unique match or the timeout expires.
Future<_ActionExecutionResult> _waitFor(
Finder finder, {
required Duration timeout,
}) async {
await _resolveUniqueMatch(finder, actionName: 'waitFor', timeout: timeout);
return const _ActionExecutionResult();
}

/// Execute a scroll action, resolving its optional Finder when provided.
Future<_ActionExecutionResult> _executeScroll({
required Finder? finder,
Expand All @@ -638,7 +656,11 @@ class ScenarioRunner {
}) async {
FinderMatch? match;
if (finder != null) {
match = await _resolveUniqueMatch(finder, actionName: 'scroll');
match = await _resolveUniqueMatch(
finder,
actionName: 'scroll',
timeout: finderTimeout,
);
}
await adapter.performScroll(match: match, deltaX: deltaX, deltaY: deltaY);
return const _ActionExecutionResult();
Expand Down
13 changes: 13 additions & 0 deletions lib/src/runtime/pilot_runtime_adapter.dart
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,19 @@ class PilotRuntimeAdapter implements RuntimeAdapter {
await _disposeClient?.call();
}

@override
Future<void> waitForEndOfFrame({required Duration timeout}) async {
try {
await _client.waitForEndOfFrame(timeout: timeout);
} catch (error) {
throw RuntimeOperationException(
operation: RuntimeOperation.waitForEndOfFrame,
message: error.toString(),
cause: error,
);
}
}

@override
Future<List<FinderMatch>> resolveFinder(Finder finder) async {
final List<PilotRuntimeFinderMatch> matches = await _client.resolveFinder(
Expand Down
Loading
Loading