Skip to content

Add synthetic browser monitoring: the site was unusable while every health check passed #263

Description

@roman1887

Problem

demo.actyze.ai was unusable for an extended period and every health check said it was fine.

The failure in #261 was a 6.8 MB uncompressed JS bundle. A real browser could not finish loading within 60 seconds, but:

GET /health              -> 200
GET /api/dashboards      -> 401   (auth working)
kubectl get pods         -> all 1/1 Running, 0 restarts
helm deploy              -> success

Nothing was down. The pods were healthy, the API was correct, the deploy succeeded. The failure was entirely in time-to-usable, which nothing measured.

What is missing

Existing probes answer "is the process alive". Nothing answers "can a person actually use this". A synthetic check that drives a real browser and asserts a time budget would have caught this immediately.

Suggested approach

The repo already uses Playwright for E2E (frontend/playwright.config.js), so the tooling is in place.

Add a scheduled workflow that, against the deployed environment:

  1. Loads the site in a real browser
  2. Asserts the page reaches an interactive state within a budget (e.g. 15s)
  3. Asserts no console errors
  4. Optionally asserts the initial JS transfer stays under an agreed size

Sketch:

test('demo loads within budget', async ({ page }) => {
  const start = Date.now();
  await page.goto('https://demo.actyze.ai', { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('button', { name: /sign in/i })).toBeVisible();
  expect(Date.now() - start).toBeLessThan(15_000);
});

Run it on a schedule (every 15–30 min) and on deploy. Failures should be visible somewhere the team actually looks — a GitHub issue, or a notification.

Worth deciding

  • Where it runs. A GitHub Actions runner has good bandwidth and will be more forgiving than a real user on a poor connection. Consider throttling the network in Playwright to approximate a realistic client.
  • Whether it needs auth. Checking the login page alone would have caught this incident. Going further requires credentials in CI, which is a bigger decision.

Why this matters beyond the one incident

The load-time gap is the kind of failure that is invisible to infrastructure monitoring and obvious to a user. For a public demo used in evaluations, a prospective user hitting a 60-second blank page is worse than a clear outage.

Definition of done

  • A scheduled check that fails when the deployed site is not usable within an agreed budget
  • Failure surfaces somewhere the team sees it
  • Verified to actually fail — e.g. by pointing it at an artificially slow endpoint — rather than merely passing

Related: #261

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions