Skip to content

CRITICAL: Release PR (develop→main) has NO E2E gate — Maestro/Visual skipped #174

Description

@TaprootFreak

Priority: CRITICAL — production rollouts ship un-E2E-tested

The auto-created Release PR (#94 Release: develop → main) reports all checks green, yet no end-to-end test runs against it.

Concrete check list on PR #94 right now

Check What it actually verifies Catches a runtime SDK mismatch?
typecheck TS compiles
lint, format static analysis
test (jest) unit tests, no native bridge
audit npm audit --audit-level=high
Bundle (mvp/full) expo export produces a JS bundle ❌ (compile ≠ boot)
CodeQL security scan
Analyze (javascript-typescript) CodeQL helper
Create Release PR the workflow that opened the PR

Missing: Maestro E2E, Visual Regression on an emulator/simulator.

Why it's skipped

.github/workflows/maestro-e2e.yml:

on:
  workflow_dispatch:
  pull_request:
    branches: [develop]    # ← only PRs targeting develop

PR #94 targets main ➝ Maestro never fires.

visual-regression.yml runs on all PRs but only does jest-image-snapshot diffs against pre-rendered Detox artefacts — it does not boot the app on a simulator either.

Why this matters now

Today we merged 8 Dependabot bumps (#160#167) with gh pr merge --admin. Each PR's Maestro CI passed in isolation against the pre-merge develop, but the combination has not been verified end-to-end. expo install --check on the resulting develop reports 10 packages outside Expo SDK 54's expected range (see #173).

If we merge PR #94 now, those mismatches go to main without any boot-the-app verification.

Required fix

Add Maestro (or at minimum a smoke subset) to the Release PR's required check list. Two reasonable options:

Option A — Run Maestro on develop → main PRs as well

# .github/workflows/maestro-e2e.yml
on:
  workflow_dispatch:
  pull_request:
    branches: [develop, main]   # ← include main

This is the smallest change. The release PR already exists, so adding main here means each push to develop fires Maestro on both the develop PR (if any) and on #94.

Option B — Fire on push to develop too

on:
  push:
    branches: [develop]
  pull_request:
    branches: [develop]

Runs Maestro on the squashed result of every merge. The release PR auto-tracks develop, so a passing push: develop Maestro run effectively certifies what #94 contains. Slightly more CI minutes (one extra run per merge) but it also closes the SDK-mismatch hole described in #173.

Recommended — both

Option B is the right invariant ("the head of develop is always E2E-green"). Option A is belt-and-suspenders so the Release PR's check list visibly carries the Maestro result, not just inherits it.

Asks

  • Decide Option A vs. B vs. both
  • Update maestro-e2e.yml trigger config
  • Add the Maestro jobs to branch-protection's required status checks on main if they aren't already
  • Do the same exercise for visual-regression.yml (it has the same gap)

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions