End-to-end tests that drive a real MetaMask against live Aave v3
Wallet connection · the full lending lifecycle · the failure modes that matter
TypeScript + Playwright · every transaction really signed, really settled on-chain · green in CI
Target: Aave v3 on Base Sepolia (Aave's only remaining testnet market).
In web3, wallet-driven flows are the hardest and least-automated part of dApp testing. MetaMask popups, network switching, transaction signing, security prompts, and irreversible on-chain state are exactly where most suites give up and start mocking.
This one doesn't mock. Every transaction here is really signed in a real MetaMask extension and really settles on-chain — and the assertions read the chain back to prove it.
Maintained by André Guerra — LinkedIn
| Suite | Tests | What it proves |
|---|---|---|
connection.spec.ts |
3 | Cold connect, reject the connection, disconnect — plus the add-network / switch-network handshake, asserting the wallet lands on the right chain |
edge-cases.spec.ts |
3 | User rejects a transaction, an over-sized supply is blocked before it reaches the wallet, and a wrong-network state disables actions instead of letting them fail |
lending-lifecycle.spec.ts |
1 (+3 quarantined) | supply against live Aave — a real two-transaction ERC-20 flow (approve + supply), verified against on-chain balances. borrow → repay → withdraw are present but test.fixme'd — see MV3 kills the wallet mid-run |
A single supply is two on-chain transactions and MetaMask's popup needs ~30s to boot for each interaction in headless — that's the honest cost of not mocking.
Also in here — a wallet-compatibility matrix (tests/matrix/). A separate, still-early harness that drives real wallets through connect / sign / reconnect / reject per dApp and records pass/fail with evidence, on its own CI workflow (matrix.yml). The verdict model is deliberately strict: a cell is only recorded once a result is stable across runs, so a flaky test never reports a false finding. See tests/matrix/STATUS.md for the build log.
The point isn't that things pass — it's what is asserted:
- Health factor. Borrowing must produce a health factor above 1 (the liquidation threshold), and taking on more debt must lower it. That number decides whether a user gets liquidated.
- On-chain, not the DOM. A row appearing in the UI only proves the frontend rendered.
utils/onchain.tsreads balances straight from the node — supplying must reduce the wallet balance, withdrawing must increase it. - The negative case. An over-sized supply must be blocked by the dApp and never reach the wallet at all. A UI that submits it and lets the chain revert has already failed the user — it costs them gas.
Written into the tests, because they're easy to get wrong:
- You cannot fully repay a loan with only what you borrowed. Interest accrues from the moment you borrow, so the debt is always a hair larger than the tokens received. A test asserting "debt hits exactly zero" is asserting something the protocol cannot do. So repay asserts the health factor rises.
- The chain is persistent. A wallet that borrowed in an earlier run still carries that position. Assertions must hold on any wallet state, not just a pristine one.
Synpress 4.1.2 — the standard tool for this job — is broken against the MetaMask it ships. It pins MetaMask 13.13.1 (the current release is 13.39.x) and most of its runtime had to be replaced. This was the bulk of the work, and it's the part worth reading.
| Problem | Reality | Fix |
|---|---|---|
| MetaMask 13.13.1 cannot broadcast transactions on Base Sepolia | It mangles the RPC payload (failed to decode param in array[0] invalid JSON input) on an endpoint viem talks to happily. The dApp just says "Transaction failed". Every transaction test was dead on arrival. |
Proved it with a control experiment (below), then upgraded to 13.39.1 |
synpress wallet-setup CLI |
Refuses to run on Windows; under WSL it hot-loops at ~96% CPU and never launches a browser | Replaced with scripts/build-cache.ts, which self-verifies the wallet really persisted |
metaMaskFixtures() |
Never types the password in headless — every test dies with Test timeout exceeded while setting up "page". Passes headed, fails in CI. |
Our own fixture, fixtures/metamask.ts |
MetaMask.importWallet() |
Targets an onboarding flow that no longer exists | utils/onboarding.ts |
connectToDapp() / switchNetwork() / getAccountAddress() |
Target elements MetaMask's redesign deleted | utils/metamask-actions.ts |
Synpress is now used for exactly one thing that still works: defineWalletSetup, which hashes the setup function to key the wallet cache.
The faucet kept reporting "Transaction failed" and it looked like a contract revert. Instead of guessing, I intercepted the exact eth_sendTransaction payload the dApp handed to the wallet, confirmed it was well-formed, and replayed that identical transaction through viem on the same RPC — where it succeeded immediately.
Same transaction, same chain, same endpoint, same wallet. Works via viem, fails via MetaMask. That isolates the bug to the wallet, not the dApp, not the chain, not the RPC — and it's what justified the version upgrade. (scripts/mint-test-tokens.ts is the surviving harness.)
The suite passed headed and failed in CI. Three reasons, none obvious:
- No popup window exists under
--headless=new. Waiting for one hangs forever. The pending request is still served atnotification.html, so we open that page ourselves. loadnever fires on MetaMask's pages, so a defaultgoto()burns its timeout — andpage.reload()has no timeout by default, so it hung indefinitely and silently ate the whole test budget.- MetaMask takes ~30s to boot its notification UI, sitting blank first. The instinctive fix — reload until it renders — is exactly wrong: each reload restarts that boot, so it never finishes.
Plus: never close-and-reopen that page to retry. Closing MetaMask's notification window is how a user rejects — retrying that way silently kills the request you're waiting for.
MetaMask's Blockaid scanner has no reputation data for testnet contracts, so it flags Aave's legitimate pool as a "Malicious address" — "you will probably lose your assets to a scammer" — and stacks two gates in front of every transaction:
"Malicious address"→ an informational modal ("Got it")"Your assets may be at risk"→ an acknowledgement checkbox, then a Confirm inside that modal (the footer Confirm merely reopens it)
Unhandled, transactions never sign and the dApp waits forever. Handled in metamask-actions.ts — with a warning not to copy that pattern anywhere near mainnet, where the alert may well be telling the truth.
More failure modes surfaced building this out — none of them in the dApp, and every one of them invisible to a mocked provider. A stub has no extension to fail to load, no version to drift, no service worker to die.
- Chrome dropped
--load-extensionfrom stable. Branded Chrome can no longer load an unpacked extension for automation — it silently ignores the flag, MetaMask never loads, and the wait for a service worker just times out. You need a Chromium (or Chrome-for-Testing) build. - Version drift renames elements out from under you. MetaMask changed the Connect button's
data-testidbetween13.13.1and13.39.1. A suite that clicked the old id found a button, clicked it, and hung — looking exactly like a broken dApp. The fix is to fall back to matching the button by its visible label when the testid misses. - MetaMask's own hardening blinds your tooling. LavaMoat scuttles globals on MetaMask's pages, so in-page evaluation (e.g. reading the DOM to debug) throws — precisely where you most want visibility. Screenshots survive it;
page.$$evaldoes not. - Headless MV3 kills the wallet mid-run — and drops in-flight transactions. On a long suite, Chrome terminates MetaMask's idle background service worker to reclaim memory. It restarts locked, and any transaction that was in flight is gone with the worker's memory. The dApp waits forever for a signature on a request that no longer exists. Re-unlocking and re-routing recovers a locked wallet; it cannot resurrect a lost request — which is why
borrow / repay / withdraw(deep into a 40-minute run) are quarantined, whilesupply(early, worker still alive) passes. The real fix is keeping the worker alive across the run.
Its modal is a MUI role="presentation" — never role="dialog", so every getByRole('dialog') locator silently matches nothing. The rest is data-cy, with traps:
| Element | Hook | Trap |
|---|---|---|
| Modal | Modal |
Not a dialog role |
| Approve step | approvalButton |
Separate from the action button — an ERC-20 flow is two transactions, two stacked buttons, one enabled at a time |
| Action step | actionButton |
Disabled until the approval lands |
| Amount field | (the only <input>) |
data-cy="inputAsset" is a red herring — it's the <h3> showing "USDC" |
| Supplied row | dashboardSuppliedListItem_USDC_Collateral |
Aave appends a state suffix; an exact match finds nothing |
All of it lives in utils/selectors.ts. One file to fix when Aave ships a redesign.
- Linux, macOS, or native Windows. Synpress's own CLI won't run on Windows, but this suite replaces it (
scripts/build-cache.ts) and its MetaMask fetch is now cross-platform (scripts/fetch-metamask.mjs, no bash), so native Windows PowerShell works. WSL2 also works (see below). - Node.js 18+ and pnpm
- A dedicated, throwaway wallet. Never one that holds real funds.
- Base Sepolia ETH for gas (not Ethereum Sepolia — different chain)
pnpm install
cp .env.example .env # add SEED_PHRASE + WALLET_PASSWORD
pnpm run wallet:address # the address to fund
pnpm run wallet:balance # check Base Sepolia gas + USDC
pnpm run mint:tokens # mint test USDC (faucet has a mint timelock)
pnpm run build:cache # import the wallet once; cached and reusedpnpm test # everything, headless (what CI runs)
pnpm run test:smoke # connection only — the quick check
pnpm run test:headed # watch it drive MetaMask for real
pnpm run test:demo # everything + video recording (slow; see below)
pnpm run report # ← open the reportTraces and screenshots are captured on every run, not just failures — these specs are the deliverable. pnpm run report opens an HTML report where each test gives you:
- A trace — Playwright's time-travel debugger. Scrub the whole journey with a full DOM snapshot before and after every action, plus the network calls behind each one. This is the best way to watch the flow.
- Named milestone screenshots, so it reads as a story rather than a log:
1. Wallet connected on Base Sepolia 2. Supply 100 USDC — before signing 3. Supply USDC — confirmed on-chain 4. Borrow 10 USDT — before signing … - Named steps (
test.step) — "Connect MetaMask to Aave", "Switch the wallet to Base Sepolia", "Supply 100 USDC as collateral" — so the report is legible to someone who has never seen the code.
CI uploads the same report as the playwright-report artifact on every run.
Video is opt-in (pnpm run test:demo). Playwright films every page in the context — which here means every MetaMask popup, 30+ clips per run — and it pushed a 3.8-minute test to 9 minutes. The trace already gives you a scrubable filmstrip, so paying that on every CI run buys almost nothing. Turn it on when you want footage to hand someone.
Test tokens are a precondition, not the thing under test — Aave's faucet enforces a mint timelock, so driving it every run would guarantee flakes.
mint:tokenshandles it out-of-band.
Windows: WSL2 setup
wsl -d Ubuntu
# Node must be a LINUX build — a Windows Node on $PATH via /mnt/c will fail.
curl -fsSL https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
source ~/.nvm/nvm.sh && nvm install 20 && corepack enable && corepack prepare pnpm@10 --activate
sudo apt-get update && sudo apt-get install -y libnss3 libnspr4 libasound2t64
cd /mnt/c/path/to/defi-wallet-e2e
pnpm install # from INSIDE WSL: a Windows node_modules carries
pnpm exec playwright install chromium # win32 binaries Linux can't executeWSL2 ships WSLg, so test:headed opens a real browser window.
├─ tests/
│ ├─ connection.spec.ts # connect / reject / disconnect + network handshake
│ ├─ lending-lifecycle.spec.ts # supply → borrow → repay → withdraw (serial)
│ └─ edge-cases.spec.ts # rejection, blocked actions, wrong network
├─ fixtures/metamask.ts # cached, unlocked wallet per test
├─ utils/
│ ├─ selectors.ts # every dApp selector, one file
│ ├─ metamask-actions.ts # every wallet popup, one file
│ ├─ onboarding.ts # imports the wallet (replaces Synpress)
│ ├─ onchain.ts # balances read from the node (viem)
│ ├─ networks.ts # Base Sepolia
│ └─ helpers.ts # behaviour-level flows
├─ scripts/
│ ├─ build-cache.ts # builds + verifies the wallet cache
│ ├─ fetch-metamask.mjs # downloads MetaMask (cross-platform: Win/mac/Linux)
│ ├─ mint-test-tokens.ts # mints test USDC on-chain
│ ├─ wallet-address.ts # the address to fund
│ └─ check-balance.ts # gas + token balances
└─ .github/workflows/e2e.yml
The design rule: every dApp selector lives in selectors.ts, every wallet interaction in metamask-actions.ts. MetaMask and Aave both ship UI changes constantly — when they do, there is exactly one file to fix.
GitHub Actions runs the e2e suites (connection, edge cases, lending supply) headless under xvfb on push / PR — real on-chain transactions, ~40 minutes, caching the MetaMask build and the Playwright browsers. The wallet-compatibility matrix runs on its own workflow (matrix.yml) so its experimental cells never gate this stable suite. borrow / repay / withdraw are quarantined (MV3 service-worker death — see above) and reported as skipped, not failed.
Required secrets (a throwaway wallet only): SEED_PHRASE, WALLET_PASSWORD.
A preflight step (pnpm run preflight) checks the wallet still has gas and test tokens before the browsers start — so a depleted wallet fails in ten seconds with an actionable message instead of twenty minutes later from inside a wallet popup. Each run supplies test USDC as collateral (and burns a little Base Sepolia gas); pnpm run mint:tokens tops the wallet up.
Every run uploads a playwright-report artifact containing the videos, traces and screenshots (below).
- Use a brand-new wallet created only for testing. It must never hold mainnet assets.
SEED_PHRASElives in.env(gitignored) and GitHub secrets. Never in the repo.- Only ever point this at testnets.
- Treat the seed as compromised the moment it touches CI — it's a throwaway, so that's fine by design.
- Keep MetaMask's MV3 service worker alive across long runs (keep-alive ping), then un-quarantine
borrow / repay / withdraw - Grow the wallet-compatibility matrix — more dApps, then more wallets (Rabby / OneKey / Phantom)
- A Foundry suite that forks the chain and tests the contracts directly (fuzz + invariant tests)
- Liquidation: push a position below health factor 1 and assert it's liquidatable
- Report the MetaMask 13.13.1 Base Sepolia broadcast bug upstream — MetaMask/metamask-extension#44598
- Aave — https://aave.com/docs · Synpress — https://docs.synpress.io · Playwright — https://playwright.dev