ci: qualify the chat bundle in a browser off the critical path

Every heavy lane waited ~5 minutes for `chat-bundle`, of which ~4.5
minutes was the Playwright qualification, before downloading the
artifact. Publish the bundle once it is built and verified, and run the
same three browser smokes in a parallel `chat-bundle-browser` lane that
consumes that exact artifact. `checks` now requires the browser lane, so
`merge-gate` still cannot pass on a bundle that failed qualification.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: song <22676124+songoow@users.noreply.github.com>

Author: songoow

.github/workflows/python-tests.yml

@@ -105,20 +105,43 @@ jobs:
           cache-dependency-path: apps/presentation/dashboard/package-lock.json
       - run: python scripts/chat_bundle.py build --install
       - run: python scripts/chat_bundle.py verify --source
-      - name: Qualify the actual compiled UI before saving the artifact
-        working-directory: apps/presentation/dashboard
-        run: |
-          ./node_modules/.bin/playwright install --with-deps chromium
-          npm run smoke:personal-workspace-packaged
-          npm run smoke:chat-turn-acceptance-retry
-          npm run smoke:chat-upgrade
+      # Publish as soon as the bundle is built and verified so consumers start
+      # in parallel with browser qualification; `checks` still requires it.
       - uses: actions/upload-artifact@v7
         with:
           name: chat-bundle-${{ github.sha }}
           path: loopx/web/chat/
           if-no-files-found: error
           retention-days: 7
 
+  chat-bundle-browser:
+    needs: [changes, chat-bundle]
+    if: needs.changes.outputs.core_tests == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/download-artifact@v7
+        with:
+          name: chat-bundle-${{ github.sha }}
+          path: loopx/web/chat/
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+      - uses: actions/setup-node@v6
+        with:
+          node-version: "24"
+          cache: npm
+          cache-dependency-path: apps/presentation/dashboard/package-lock.json
+      - name: Qualify the actual compiled UI
+        working-directory: apps/presentation/dashboard
+        run: |
+          npm ci --ignore-scripts
+          ./node_modules/.bin/playwright install --with-deps chromium
+          npm run smoke:personal-workspace-packaged
+          npm run smoke:chat-turn-acceptance-retry
+          npm run smoke:chat-upgrade
+
   kernel-static-checks:
     needs: [changes, chat-bundle]
     if: needs.changes.outputs.core_tests == 'true'
@@ -320,19 +343,21 @@ jobs:
   checks:
     # Preserve the required check name while exposing independent failure lanes.
     if: always() && needs.changes.outputs.core_tests == 'true'
-    needs: [changes, kernel-static-checks, typescript-coverage, dashboard-acceptance]
+    needs: [changes, kernel-static-checks, typescript-coverage, dashboard-acceptance, chat-bundle-browser]
     runs-on: ubuntu-latest
     timeout-minutes: 2
     steps:
       - name: Require kernel and Dashboard qualification
         env:
+          BROWSER_RESULT: ${{ needs.chat-bundle-browser.result }}
           DASHBOARD_RESULT: ${{ needs.dashboard-acceptance.result }}
           KERNEL_RESULT: ${{ needs.kernel-static-checks.result }}
           TYPESCRIPT_RESULT: ${{ needs.typescript-coverage.result }}
         run: |
           test "$KERNEL_RESULT" = success
           test "$TYPESCRIPT_RESULT" = success
           test "$DASHBOARD_RESULT" = success
+          test "$BROWSER_RESULT" = success
 
   node-minimum-compatibility:
     needs: changes
@@ -707,7 +732,7 @@ jobs:
       - uses: actions/setup-python@v6
         with:
           python-version: "3.11"
-      - name: Verify the source-bound, browser-qualified Dashboard artifact
+      - name: Verify the source-bound Dashboard artifact
         run: python scripts/chat_bundle.py verify --source
 
   merge-gate:

docs/development/frontend-delivery.md

@@ -44,9 +44,12 @@ and freshness metadata, not a signature or a replacement for release attestation
 
 ## PR qualification and releases
 
-PR CI builds a clean bundle once, exercises its actual pages in a browser, then
-uploads `chat-bundle-<checkout SHA>`. Consumers download that qualified artifact;
-both frontend-only and mixed/backend PRs pass through this producer. On pull requests the checkout SHA is GitHub's tested merge commit, which
+PR CI builds and verifies a clean bundle once, then uploads
+`chat-bundle-<checkout SHA>`. Consumers download that one artifact, and
+`chat-bundle-browser` exercises its actual pages in a browser in parallel. The
+`checks` aggregate requires that browser lane, so `merge-gate` cannot pass on
+an artifact that failed browser qualification. Both frontend-only and
+mixed/backend PRs pass through this producer. On pull requests the checkout SHA is GitHub's tested merge commit, which
 may differ from the branch head. Generated-file Git cleanliness is no longer a
 qualification gate. Browser, integrity and workflow gates remain required.
 

tests/test_python_ci_workflow.py

@@ -37,7 +37,11 @@ def test_dashboard_acceptance_and_kernel_checks_run_independently() -> None:
     assert "python -m mypy" not in dashboard
 
     assert "if: always() && needs.changes.outputs.core_tests == 'true'" in aggregate
-    assert "needs: [changes, kernel-static-checks, typescript-coverage, dashboard-acceptance]" in aggregate
+    assert (
+        "needs: [changes, kernel-static-checks, typescript-coverage, "
+        "dashboard-acceptance, chat-bundle-browser]"
+    ) in aggregate
+    assert "needs.chat-bundle-browser.result" in aggregate
     assert "needs.kernel-static-checks.result" in aggregate
     assert "needs.typescript-coverage.result" in aggregate
     assert "needs.dashboard-acceptance.result" in aggregate
@@ -46,23 +50,27 @@ def test_dashboard_acceptance_and_kernel_checks_run_independently() -> None:
 @pytest.mark.parametrize("kernel", ["success", "failure", "cancelled", "skipped"])
 @pytest.mark.parametrize("typescript", ["success", "failure", "cancelled", "skipped"])
 @pytest.mark.parametrize("dashboard", ["success", "failure", "cancelled", "skipped"])
+@pytest.mark.parametrize("browser", ["success", "failure", "cancelled", "skipped"])
 def test_checks_aggregate_requires_every_parallel_lane(
-    kernel: str, typescript: str, dashboard: str,
+    kernel: str, typescript: str, dashboard: str, browser: str,
 ) -> None:
     gate = WORKFLOW.split("name: Require kernel and Dashboard qualification", 1)[1]
     script = gate.split("run: |", 1)[1].split("\n\n  node-minimum-compatibility:", 1)[0]
     result = subprocess.run(
         ["bash", "-e", "-c", script],
         env={
             **os.environ,
+            "BROWSER_RESULT": browser,
             "DASHBOARD_RESULT": dashboard,
             "KERNEL_RESULT": kernel,
             "TYPESCRIPT_RESULT": typescript,
         },
         capture_output=True,
         check=False,
     )
-    assert (result.returncode == 0) == (kernel == typescript == dashboard == "success")
+    assert (result.returncode == 0) == (
+        kernel == typescript == dashboard == browser == "success"
+    )
 
 
 def test_minimum_node_lane_exercises_sqlite_without_a_skip_list() -> None:
@@ -197,7 +205,10 @@ def test_merge_gate_runs_on_all_prs_and_checks_every_core_aggregate() -> None:
     for name, output in (("checks", "core_tests"), ("test-shard", "python_tests"), ("stage2c-suite", "stage2c_tests"), ("windows-powershell", "python_tests"), ("presentation", "presentation_tests")):
         job = WORKFLOW.split(f"  {name}:\n", 1)[1].split("    steps:", 1)[0]
         if name == "checks":
-            assert "needs: [changes, kernel-static-checks, typescript-coverage, dashboard-acceptance]" in job
+            assert (
+                "needs: [changes, kernel-static-checks, typescript-coverage, "
+                "dashboard-acceptance, chat-bundle-browser]"
+            ) in job
             assert "if: always() && needs.changes.outputs.core_tests == 'true'" in job
         else:
             assert "needs: [changes, chat-bundle]" in job
@@ -207,15 +218,23 @@ def test_merge_gate_runs_on_all_prs_and_checks_every_core_aggregate() -> None:
 def test_presentation_exemption_retains_real_frontend_checks_and_force_full() -> None:
     job = WORKFLOW.split("  presentation:\n", 1)[1].split("  merge-gate:\n", 1)[0]
     assert "name: chat-bundle-${{ github.sha }}" in job
-    producer = WORKFLOW.split("  chat-bundle:\n", 1)[1].split("  kernel-static-checks:\n", 1)[0]
-    assert "npm run smoke:personal-workspace-packaged" in producer
-    assert "npm run smoke:chat-turn-acceptance-retry" in producer
-    assert "npm run smoke:chat-upgrade" in producer
+    producer = WORKFLOW.split("  chat-bundle:\n", 1)[1].split("  chat-bundle-browser:\n", 1)[0]
+    browser = WORKFLOW.split("  chat-bundle-browser:\n", 1)[1].split("  kernel-static-checks:\n", 1)[0]
+    # The producer publishes a built, verified bundle; the browser lane
+    # qualifies that same artifact in parallel and `checks` requires it.
+    assert producer.index("chat_bundle.py verify --source") < producer.index("actions/upload-artifact")
+    assert "smoke:" not in producer
+    assert "needs: [changes, chat-bundle]" in browser
+    assert "if: needs.changes.outputs.core_tests == 'true'" in browser
+    assert "name: chat-bundle-${{ github.sha }}" in browser
+    assert "chat_bundle.py build" not in browser
     assert (
-        producer.index("npm run smoke:personal-workspace-packaged")
-        < producer.index("npm run smoke:chat-turn-acceptance-retry")
-        < producer.index("actions/upload-artifact")
+        browser.index("actions/download-artifact")
+        < browser.index("npm run smoke:personal-workspace-packaged")
+        < browser.index("npm run smoke:chat-turn-acceptance-retry")
+        < browser.index("npm run smoke:chat-upgrade")
     )
+    assert "continue-on-error" not in browser
     assert "scripts/chat_bundle.py verify --source" in job
     assert "status --short --untracked-files=all -- loopx/web/chat" not in job
     assert "continue-on-error" not in job
@@ -348,9 +367,11 @@ def test_four_shards_execute_each_test_once_and_merge_portable_coverage(
 
 
 def test_backend_and_mixed_prs_require_the_browser_qualified_artifact() -> None:
-    producer = WORKFLOW.split("  chat-bundle:\n", 1)[1].split("  kernel-static-checks:\n", 1)[0]
+    producer = WORKFLOW.split("  chat-bundle:\n", 1)[1].split("  chat-bundle-browser:\n", 1)[0]
     assert "needs.changes.outputs.core_tests == 'true'" in producer
-    for name in ("kernel-static-checks", "typescript-core", "dashboard-acceptance", "test-shard", "stage2c-suite", "windows-powershell", "presentation"):
+    aggregate = WORKFLOW.split("  checks:\n", 1)[1].split("    steps:", 1)[0]
+    assert "chat-bundle-browser" in aggregate
+    for name in ("chat-bundle-browser", "kernel-static-checks", "typescript-core", "dashboard-acceptance", "test-shard", "stage2c-suite", "windows-powershell", "presentation"):
         job = WORKFLOW.split(f"  {name}:\n", 1)[1].split("      - uses: actions/setup-", 1)[0]
         assert "needs: [changes, chat-bundle]" in job
         assert "name: chat-bundle-${{ github.sha }}" in job