Cookie-auth capture fails for sites using Google SSO (headless chromium blocked by Google)

[barockok]

Symptom

Connecting a cookie-auth custom integration whose login page delegates to Google SSO fails at the Google login step — Google returns an error (typically "Couldn't sign you in" / "This browser or app may not be secure") and capture never completes. Reproducible / appears to always fail.

Root cause

The cookie-capture flow spawns Chromium in headless mode with Playwright's bundled binary:

packages/server/src/auth/cookie.ts:130-148

"--headless=new",
`--remote-debugging-port=${remotePort}`,
`--user-data-dir=${userDataDir}`,
"--remote-allow-origins=http://127.0.0.1",
"--no-first-run",
"--no-default-browser-check",
"--disable-features=TranslateUI",
"--window-size=1280,800",
loginUrl,

Google's account sign-in actively blocks headless and automation-controlled browsers (policy against "embedded/automated user-agents"). Signals that trip it here:

• --headless=new → default UA contains HeadlessChrome.
• Playwright/CDP-driven Chromium → navigator.webdriver = true, automation fingerprints.
• No --disable-blink-features=AutomationControlled, no UA override.

So even though the user drives the login through the portal screencast, Google sees a headless automated browser and refuses. This blocks every cookie integration gated behind Google SSO.

Options (in rough order of robustness)

1. Headed Chromium under a virtual display (Xvfb) instead of --headless=new. Real headed Chrome is far less likely to be flagged. Image already has the chromium + libs; add xvfb and run with DISPLAY.
2. Reduce automation fingerprint: override UA to drop HeadlessChrome, add --disable-blink-features=AutomationControlled. Helps but Google may still detect navigator.webdriver / CDP — partial.
3. Document the limitation: if neither lands, state clearly that Google-SSO-gated sites can't be cookie-captured; steer such integrations to a proper OAuth2 plugin instead.

Note: Google deliberately fights automated login; expect an arms race. Option 1 (Xvfb headed) is the most durable.

Acceptance

• A cookie integration whose login uses Google SSO can complete capture end-to-end, OR
• If unsupported, connect() / the connect page surfaces a clear "Google SSO login not supported for cookie auth — use OAuth" message instead of a generic Google error.
• Decision + approach recorded in docs/findings/.

Source

packages/server/src/auth/cookie.ts:128-149 (chromium spawn args)

[barockok]

Local verification (fingerprint probe)

Spawned chromium exactly like cookie.ts (raw process + CDP, no Playwright automation attached) under 3 configs, loaded the real Google sign-in page (accounts.google.com/ServiceLogin?service=mail), and read the signals Google inspects.

Signal	A: current (--headless=new)	B: opt2 (headless + UA override + --disable-blink-features=AutomationControlled)	C: opt1 (headed)
UA	HeadlessChrome/148 ⚠️	Chrome/131 ✅	Chrome/148 ✅
navigator.webdriver	false	false	false
window.chrome	true	true	true
plugins / langs	5 / en-US	5 / en-US	5 / en-US
sign-in page loads	yes	yes	yes
email input present	yes	yes	yes
blocked at this step	no	no	no

Findings

1. navigator.webdriver is already false in prod. The flow uses a raw chromium spawn with --remote-debugging-port and no --enable-automation, so the webdriver flag is never set. It is not the culprit — earlier assumption in the issue body was wrong on this point.
2. The only automation tell in config A is the HeadlessChrome UA token. --headless=new puts HeadlessChrome/<ver> in the UA string. Option 2's UA override removes it; option 1 (headed) removes headless entirely.
3. The email/identifier page loads in all three configs, including current headless — email input present, no block. So Google does not reject at page GET.

Limitation

Google's "Couldn't sign you in / this browser or app may not be secure" block fires at the password-submit step, not the email page. Exercising it requires entering real credentials, which wasn't done. So this verifies the signals each option changes, not the end-to-end login outcome.

Revised plan

• Fix lever is narrower than first stated: not webdriver, just the HeadlessChrome UA token (+ possibly headless itself).
• Try option 2 first — it's a one-line UA override (cheap, reversible). Wire behind a flag and test a real login against a live workbench (user types password in the screencast).
• If Google still blocks at password step (deeper headless heuristics), fall back to option 1: headed chromium under Xvfb in the Linux image.

Probe was a throwaway script (not committed); working tree clean.

[barockok]

Cannot reproduce — Google SSO login completes in the headless capture browser

Verified end-to-end against two real cookie-auth integrations whose login flows go through Google SSO:

• ng-mis — ngmis.amartha.id → NextAuth → Keycloak (sso.amartha.id) federated to Google.
• superset — Google SSO login.

Both completed Google sign-in inside the --headless=new capture browser (driven via the portal screencast), cookies were captured, and ng-mis executed end-to-end returning real data. If Google had blocked the headless browser, the login could not have completed and no session would exist.

The original concern ("Google blocks headless/automation browsers at the password step") does not reproduce here. The earlier fingerprint probe was inconclusive only because it couldn't perform a real credential step; these live logins settle it.

Caveat

This worked with --headless=new + a real user driving via screencast + Google Workspace accounts. Google's "browser may not be secure" block can be account-/flow-specific, so this isn't a guarantee for every Google-SSO site. Reopen if a specific integration's Google login actually gets blocked in the capture browser.

Closing as not-reproducible. (Note: required the --no-sandbox fix from #18 for the capture browser to start at all.)

[barockok]

Reopening — reproduced for superset (Google SSO directly)

Confirmed in the deployed build (v0.4.0, non-root, chromium wrapped with --no-sandbox): pairing superset (cookie auth) shows Google's own 500. That's an error. There was an error. Please try again later. page inside the capture browser, at the Google SSO step. The workbench did its part — chromium launched, screencast streamed, page loaded (no 500 in the workbench log; only 200/401). The 500 is Google's, returned to the headless/automated browser during OAuth sign-in.

Why my earlier close was wrong

ng-mis worked because its login is fronted by Keycloak (own login form) — it never hit accounts.google.com directly, so it didn't exercise Google's block. superset hits Google SSO directly and Google refuses the automated browser. So "ng-mis works" did NOT validate Google SSO; superset is the real test, and it fails.

Likely cause

Google blocks OAuth sign-in from automated/embedded browsers. The capture browser runs --headless=new driven over CDP — Google detects this and returns the generic 500 (or "browser may not be secure") at the authorize step.

Diagnostic to confirm browser-context (not a superset OAuth config issue)

Open the same superset login in a normal desktop browser on a machine that can reach it — if Google sign-in completes there but 500s in the capture browser, it's the headless/automation context (this issue), not superset's redirect_uri/client config.

Fix options (rough order of robustness)

1. Headed Chromium under Xvfb (virtual display) instead of --headless=new. Real headed Chrome is far less likely to be flagged. Image already bundles chromium + libs; add xvfb + run with DISPLAY.
2. Reduce automation fingerprint: UA override (drop HeadlessChrome), --disable-blink-features=AutomationControlled. Partial — Google may still detect CDP/automation.
3. If neither lands: document that Google-SSO-gated cookie integrations aren't supported via headless capture; prefer an OAuth2 plugin where the provider supports it.

Affects any cookie integration whose login is direct Google SSO (superset here). Keycloak/Flask/own-form logins are unaffected.

[barockok]

Correction: local works, only the cluster fails → environment, not headless detection

New evidence: the same workbench build (same --headless=new capture browser) completes Google SSO capture fine locally for both ng-mis and superset, but the cluster pod shows Google's generic 500. That's an error and never redirects back.

If Google were blocking the headless/automated browser, it would fail locally too. It doesn't. So this is not the headless-detection scenario this issue was originally about — it's a deployment-environment difference in the pod's egress to Google.

Leading hypothesis: egress proxy not applied to the spawned chromium

Clusters often force outbound through an HTTP(S) proxy. Workbench's own ctx.http (node fetch) honors HTTPS_PROXY, but startCookieSession spawns chromium with no --proxy-server, so the capture browser can't properly reach accounts.google.com → Google's OAuth flow dies (500 / no redirect). Local has no proxy → works.

Other env candidates: datacenter egress IP flagged by Google abuse-protection, or an egress NetworkPolicy/firewall breaking part of the OAuth redirect chain.

Diagnostics (in the pod)

env | grep -i proxy
curl -s https://ifconfig.me            # datacenter IP?
curl -so /dev/null -w '%{http_code}' https://accounts.google.com/

Likely fix

Pass the proxy to the chromium spawn in cookie.ts:
--proxy-server=${process.env.HTTPS_PROXY || process.env.HTTP_PROXY} when set.

This issue stays open as the tracker, but the cause is environmental (egress), not the original "Google blocks headless" theory.

[barockok]

Added CAPTURE_PROXY support (commit on main): sets --proxy-server on the capture chromium spawn, so an in-cluster capture can egress via a clean residential/ISP proxy and avoid Google's datacenter-IP 500. Set e.g. CAPTURE_PROXY=socks5://host:1080. Doesn't fix Google by itself — it's the hook to route around the flagged IP. Still need to provision the clean-egress proxy.