Skip to content

Migrate bridge e2e tests from forEachEngine to regressionTest harness#126

Open
Copilot wants to merge 42 commits intonew_test_harnessfrom
copilot/migrate-bridge-package-tests
Open

Migrate bridge e2e tests from forEachEngine to regressionTest harness#126
Copilot wants to merge 42 commits intonew_test_harnessfrom
copilot/migrate-bridge-package-tests

Conversation

Copy link
Contributor

Copilot AI commented Mar 11, 2026

Migrates bridge package end-to-end tests from the legacy forEachEngine dual-run pattern to the stricter regressionTest harness, which adds serializer round-trip validation, traversal coverage checks, and GraphQL replay.

Serializer fixes

Fixed the serializeToolBlock function to properly round-trip tool self-wire patterns that previously broke parse → serialize → parse:

Fix Details
Expression self-wires const.x + 1, const.x * 5, const.x >= 18 now reconstruct from internal __expr_* fork handles
Ternary self-wires const.flag ? "POST" : "GET" using cond/thenValue/elseValue wire properties
Coalesce/fallback self-wires context.x ?? "5000" using fallbacks array with type: "nullish"
String interpolation self-wires "city={const.city}" reconstructed from __concat_* fork handles
Element-scoped tool handles Loop-scoped with blocks now serialize correctly inside array mapping braces

Test tool infrastructure

Added universal test tools in test/utils/bridge-tools.ts:

  • test.sync.multitool — Sync echo tool with .bridge = { sync: true } metadata
  • test.batch.multitool — Batch echo tool with .bridge = { batch: { maxBatchSize: 100, flush: "microtask" } } metadata; returns Error for items with _error set

Migrated files (12/28)

  • fallback-bug.test.ts — Restructured with test.multitool as controllable primary source to make all || fallback paths exercisable
  • chained.test.ts — 3-tool chain verified via test.multitool echo behavior
  • string-interpolation.test.ts — Combined 8 template tests into 3 bridge operations; formatter round-trip tests kept as standalone describe blocks
  • interpolation-universal.test.ts — Split into fallback and ternary regression groups
  • prototype-pollution.test.ts — Converted to direct engine loop (cannot use regressionTest — see bugs below)
  • ternary.test.ts — Parser/serializer tests unchanged; execution tests split across 4 regressionTest blocks + 2 direct-execution blocks for patterns with serializer issues; refFallback trace divergence resolved with function assertor
  • expressions.test.ts — 3 regressionTest blocks covering arithmetic, operator precedence, and safe flag propagation; short-circuit side-effect tests moved to test/legacy/
  • force-wire.test.ts — Force tool execution with test.multitool; timing and force catch null tests moved to test/legacy/
  • infinite-loop-protection.test.ts — Array mapping and non-circular chain via regressionTest; circular dependency as standalone forEachEngine; runtime-only ExecutionTree depth ceiling kept as standalone describe
  • tool-self-wires-runtime.test.ts — 9 scenarios covering constant wires, expressions, ternary, coalesce, string interpolation, and httpCall-style multi-tool bridges (unblocked by serializer fix)
  • native-batching.test.ts — 3 scenarios using test.batch.multitool for batch echo, trace/log validation, and partial failure with catch fallbacks (unblocked by batch tool + element-scoped tool serializer fix)

Legacy tests (test/legacy/)

19 files remain in test/legacy/ with a comprehensive README.md documenting per-file blockers and suggested migration approaches. These files still run via forEachEngine and are included in the test script. Common blockers:

  • Side-effect inspection — Spy tools, call-count tracking, timing assertions incompatible with data-driven harness
  • Custom tool metadata — Pipe syntax tests need transforming tools beyond echo
  • Direct API usageExecutionTree, BridgeRuntimeError property inspection, parseBridge calls
  • Compiler skips — Many tests use skip: engine === "compiled" for unsupported features

Serializer bugs discovered (not yet fixed)

Bug Impact
Traverses __proto__/constructor on wire path objects Produces a.__proto__ <- i.x[] as [object Object] {}
Drops alias references in `
Converts template strings in `
Renames iterator variables (ititem) Breaks ternary refs inside array mappings

Known remaining issues

  • Ternary.refFallback: runtime lazily skips fallback tool (0 traces) while compiler eagerly calls it (1 trace) — resolved with function-based assertTraces accepting both values
  • 19 files remain in test/legacy/ pending further serializer fixes and harness enhancements

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

@changeset-bot
Copy link

changeset-bot bot commented Mar 11, 2026

⚠️ No Changeset found

Latest commit: ce9f966

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 11, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bridge ce9f966 Commit Preview URL

Branch Preview URL
Mar 13 2026, 10:18 AM

Copilot AI and others added 4 commits March 11, 2026 14:38
Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
….ts to regressionTest

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
… - 2 trace assertions need engine-specific handling)

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Copilot AI changed the title [WIP] Migrate bridge package end to end tests to new test harness Migrate bridge e2e tests from forEachEngine to regressionTest harness Mar 11, 2026
@aarne
Copy link
Contributor

aarne commented Mar 11, 2026

@copilot

continue migrating the testcases.

any testcase that you are not able to migrate move it to test/legacy folder and create a README.md in that folder with all action items to do before we can migrate the tests

for compiler incompatibility issues just create a assertion to downgrade to runner

Copilot AI and others added 5 commits March 11, 2026 15:51
Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
…nTest

Migrated the following test groups to the regressionTest harness:
- expressions: execution (arithmetic, comparisons, tool source, chained)
- expressions: operator precedence
- boolean logic: not prefix
- parenthesized arithmetic
- safe flag propagation in expressions (with allowDowngrade)
- safe navigation with sync tool (with allowDowngrade)

Tests that cannot be migrated due to compiler limitations remain as
forEachEngine blocks:
- String literal comparison (serializer strips quotes)
- Array mapping with expressions (serializer bug)
- catch error fallback (compiler doesn't downgrade)
- and/or boolean logic (compiler generates broken __and/__or)
- Parenthesized boolean expressions (contain and/or)
- condAnd/condOr without safe navigation
- Safe flag on right operand of and/or

Side-effect tests (short-circuit behavior) moved to
test/legacy/expressions.test.ts per migration guidelines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Tests 1,2,4,5 migrated to regressionTest with test.multitool
- Tests 3 (timing) and 6 (runtime-skip) moved to legacy/force-wire.test.ts
- All 6 original test scenarios preserved

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…n items

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 11, 2026

@copilot

continue migrating the testcases.
...

Continued migration in commits d198728–a6cfc0b:

Migrated to regressionTest: expressions.test.ts, force-wire.test.ts, infinite-loop-protection.test.ts (plus ternary.test.ts fix)

Moved to test/legacy/: 19 files that can't be migrated yet due to serializer round-trip bugs, side-effect spying, custom tool metadata, timing assertions, or direct API usage. Created test/legacy/README.md with per-file blockers and action items.

All 1140 tests passing, 0 failures. The legacy files still run via the updated test script.

@aarne
Copy link
Contributor

aarne commented Mar 11, 2026

@copilot start taking legacy tests one by one and fixing them

Start with parser roundtrip

Then move to timing tests and validate via assertTraces function or from assertLogs

Create a new universal test tool for batching and use it in tests

Copilot AI and others added 5 commits March 11, 2026 16:53
…hql replay failures

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
…olation in tool self-wires

Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Move test/legacy/tool-self-wires-runtime.test.ts to
test/tool-self-wires-runtime.test.ts using the regressionTest pattern
with test.multitool from bridge-tools.ts.

All 9 test cases are consolidated into a single regressionTest block
with shared const declarations and per-scenario assertions. The
compiler handles all cases natively so allowDowngrade is not needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Migrate all 3 tests from forEachEngine to regressionTest pattern
- Fix serializer bug: loop-scoped tool handles now round-trip correctly
  - Add element property to HandleBinding type for tool handles
  - Mark element-scoped tools in parser (processLocalToolBindings)
  - Serializer skips element handles in header, emits inside array blocks
  - Serializer correctly identifies element-scoped tool wires
- Delete legacy/native-batching.test.ts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 11, 2026

@copilot start taking legacy tests one by one and fixing them

Start with parser roundtrip...

Progress in commits 19f6164–5736138:

Parser roundtrip fixes (15073cc): Fixed the serializer to properly round-trip tool self-wire expressions (const.x + 1), ternary (const.flag ? "POST" : "GET"), coalesce (context.x ?? "5000"), string interpolation ("city={const.city}"), and element-scoped tool handles in loop blocks. This unblocked several test migrations.

New test tools: Added test.sync.multitool (sync flag) and test.batch.multitool (batch flag) to bridge-tools.ts.

Migrated: tool-self-wires-runtime.test.ts (9 scenarios), native-batching.test.ts (3 scenarios + empty array coverage), fixed infinite-loop-protection.test.ts.

19 files remain in test/legacy/ — updated README.md with per-file blockers, suggested migration approach, and common patterns (allowDowngrade for compiler skips, assertTraces for call-count/timing, assertError for BridgeRuntimeError inspection).

@github-actions
Copy link
Contributor

github-actions bot commented Mar 12, 2026

🐰 Bencher Report

Branchcopilot/migrate-bridge-package-tests
Testbedubuntu-latest

⚠️ WARNING: No Threshold found!

Without a Threshold, no Alerts will ever be generated.

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
compiled: absolute baseline (passthrough, no tools)📈 view plot
⚠️ NO THRESHOLD
0.00 ns
compiled: array + SYNC tool-per-element 10📈 view plot
⚠️ NO THRESHOLD
0.01 ns
compiled: array + SYNC tool-per-element 100📈 view plot
⚠️ NO THRESHOLD
0.02 ns
compiled: array + tool-per-element 10📈 view plot
⚠️ NO THRESHOLD
0.01 ns
compiled: array + tool-per-element 100📈 view plot
⚠️ NO THRESHOLD
0.04 ns
compiled: chained 3-tool fan-out📈 view plot
⚠️ NO THRESHOLD
0.01 ns
compiled: flat array 10 items📈 view plot
⚠️ NO THRESHOLD
0.01 ns
compiled: flat array 100 items📈 view plot
⚠️ NO THRESHOLD
0.02 ns
compiled: flat array 1000 items📈 view plot
⚠️ NO THRESHOLD
0.12 ns
compiled: nested array 10x10📈 view plot
⚠️ NO THRESHOLD
0.03 ns
compiled: nested array 20x10📈 view plot
⚠️ NO THRESHOLD
0.05 ns
compiled: nested array 5x5📈 view plot
⚠️ NO THRESHOLD
0.01 ns
compiled: short-circuit (overdefinition bypass)📈 view plot
⚠️ NO THRESHOLD
0.00 ns
compiled: simple chain (1 tool)📈 view plot
⚠️ NO THRESHOLD
0.00 ns
exec: absolute baseline (passthrough, no tools)📈 view plot
⚠️ NO THRESHOLD
0.01 ns
exec: array + SYNC tool-per-element 10📈 view plot
⚠️ NO THRESHOLD
0.11 ns
exec: array + SYNC tool-per-element 100📈 view plot
⚠️ NO THRESHOLD
0.96 ns
exec: array + tool-per-element 10📈 view plot
⚠️ NO THRESHOLD
0.13 ns
exec: array + tool-per-element 100📈 view plot
⚠️ NO THRESHOLD
1.08 ns
exec: chained 3-tool fan-out📈 view plot
⚠️ NO THRESHOLD
0.06 ns
exec: flat array 10 items📈 view plot
⚠️ NO THRESHOLD
0.02 ns
exec: flat array 100 items📈 view plot
⚠️ NO THRESHOLD
0.10 ns
exec: flat array 1000 items📈 view plot
⚠️ NO THRESHOLD
0.87 ns
exec: nested array 10x10📈 view plot
⚠️ NO THRESHOLD
0.16 ns
exec: nested array 20x10📈 view plot
⚠️ NO THRESHOLD
0.30 ns
exec: nested array 5x5📈 view plot
⚠️ NO THRESHOLD
0.06 ns
exec: short-circuit (overdefinition bypass)📈 view plot
⚠️ NO THRESHOLD
0.01 ns
exec: simple chain (1 tool)📈 view plot
⚠️ NO THRESHOLD
0.01 ns
parse: large bridge (20 handles x 5 wires)📈 view plot
⚠️ NO THRESHOLD
1.35 ns
parse: simple bridge📈 view plot
⚠️ NO THRESHOLD
0.04 ns
🐰 View full continuous benchmarking report in Bencher

@aarne aarne marked this pull request as ready for review March 12, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants