Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
5e24114
[change] a new structure for stable test class with isolated helper f…
ptrcja Sep 4, 2025
ac616f8
refactor: created helpers class for better code organization
ptrcja Sep 5, 2025
d021897
[change] improve helpers class methods and stable test adjustments
ptrcja Sep 7, 2025
8c1d108
fix: correct product page verification and option handling
ptrcja Sep 10, 2025
36da4f9
fix: verify checkout button instead of cart drawer for add to cart
ptrcja Sep 11, 2025
bec6ee6
refactor: apply proper test pattern - business logic in tests, implem…
ptrcja Sep 11, 2025
da7b4e5
fix: implement robust cookie banner handling
ptrcja Sep 12, 2025
e7ddb61
fix: resolve TypeScript type error for Signals overload
ptrcja Sep 14, 2025
43247e1
refactor: update flaky test class test cases
ptrcja Sep 16, 2025
f57a5b7
feat: created flaky detector class for test analysis
ptrcja Sep 16, 2025
afa055a
feat: create report generator class for test results
ptrcja Sep 24, 2025
124e730
feat: complete run-flaky-detection class implementation
ptrcja Sep 29, 2025
a0a970b
feat: complete flaky-detection workflow implementation
ptrcja Sep 30, 2025
8bb8cb8
feat: add HTML generation class and features
ptrcja Oct 4, 2025
81761ff
fix: resolve privacy banner issue using addLocatorHandler
ptrcja Oct 8, 2025
e0cad94
feat: add coffee stable test class for reliability testing
ptrcja Oct 27, 2025
dcb2594
feat: implement flaky test class with anti-patterns
ptrcja Oct 28, 2025
f378626
refactor(tests): optimize stable test suite with self-validating helpers
ptrcja Nov 5, 2025
2784e36
feat: refine helper implementations
ptrcja Nov 6, 2025
9fec26c
refactor: improve report clarity and remove search functionality
ptrcja Nov 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
290 changes: 265 additions & 25 deletions .github/workflows/flaky-detection.yml

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,7 @@ node_modules/
.DS_Store
.last-run.json
.cursorrules
index.html
ctrf-report.json
#examples
examples/
183 changes: 183 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# Playwright Flaky Test Detector

For [Coffee Store](https://coffee-e2e.vercel.app/) demo project ☕️

---

## Why I Built It

Flaky tests waste time and hide bugs. This provides a deterministic target and a repeatable way to measure stability and expose sync issues.

## What It Does

A Playwright-based detector that runs tests repeatedly, classifies them as Stable, Flaky, or Failing using simple statistics, and outputs HTML, JSON, and CSV reports.

- Runs Playwright tests N times (10, 20, 50, 100+)
- Analyzes results using statistical methods
- Generates interactive reports (HTML, JSON, CSV)
- Demonstrates best practices vs anti-patterns
- Provides actionable insights with confidence scores
- Calculates test suite health scores

## Data Flow

```mermaid
flowchart TB
Start([Start]) --> Config[Load config]
Config --> Loop{Run tests N times}
Loop -->|each run| Exec["Run Playwright
Write CTRF JSON
Store results"]
Exec --> Loop
Loop -->|done| Process["Aggregate by test
Compute rates, variance, patterns"]
Process --> Classify{Classify}
Classify --> Results["FLAKY / STABLE / FAILING
+ Confidence score"]
Results --> Reports["Generate reports
HTML | JSON | CSV"]
Reports --> End([End])
```

## Technical Description

### Features

**Detection Engine**

- Statistical analysis (failure rates, duration variance, patterns)
- CTRF integration for standardized parsing
- Confidence scoring based on evidence strength
- Configurable thresholds

**Reporting Suite**

- Interactive HTML
- CSV for spreadsheets

**Educational Value**

- Side-by-side stable vs flaky examples
- 20+ documented anti-patterns
- Reusable helpers following Page Object Model

## Stack

- Playwright, TypeScript, Node.js
- CTRF reporter and JSON artifacts
- Demo app: [coffee-e2e.vercel.app](http://coffee-e2e.vercel.app)

---

## Understanding Reports

### Detection Confidence

Shows confidence in test classification accuracy.

| Score | Meaning |
|-------|---------|
| 80-100% | Very high confidence |
| 50-80% | High confidence |
| 20-50% | Moderate confidence |
| 0-20% | Low confidence |

Calculated from: number of runs (40%), failure rate patterns (30%), duration variance (30%).

### Test Status

| Status | Criteria |
|--------|----------|
| STABLE | Pass rate = 100% |
| FLAKY | Failure rate 10-90% |
| FAILING | Failure rate ≥90% |

### Health Score

```
Health = (Stability × 40%) + (Reliability × 40%) + (Maintainability × 20%)
```

| Score | Action |
|-------|--------|
| 90-100 | Continue monitoring |
| 70-89 | Address flaky tests |
| 50-69 | Urgent attention needed |
| 0-49 | Critical - refactor suite |

---

## Resources

**Inspiration:**

1. [Detecting Flaky Tests in Playwright](https://ray.run/blog/detecting-and-handling-flaky-tests-in-playwright)
2. [Avoiding Flaky Playwright Tests](https://betterstack.com/community/guides/testing/avoid-flaky-playwright-tests/)
3. [Stability Over Luck - Run Tests 100 Times](https://medium.com/@daniel.wentland_49864/stability-over-luck-why-every-new-playwright-test-should-run-at-least-100-times-d85f67ff2845)

**References:**

- [Playwright Best Practices](https://playwright.dev/docs/best-practices)
- [Test Pyramid](https://martinfowler.com/articles/practical-test-pyramid.html)
- [CTRF Specification](https://ctrf.io/)


**Test App:**

- Live: [Coffee E2E - Vercel](https://coffee-e2e.vercel.app/)
- Controlled deployment with logs/database access
- Stable, deterministic test subject

---

## Usage

### Installation

**Prerequisites**: Node.js ≥18, npm, Git

```bash
# Clone
git clone https://github.com/yourusername/flaky-test-detector.git
cd flaky-test-detector

# Install
npm install

# Install browsers
npx playwright install

# Build
npm run build

# Verify
npm test
```

---

### Quick Start

```bash
npm run detect-flaky
```

### Commands

```bash
npm run detect-flaky:quick # 10 runs
npm run detect-flaky:thorough # 20 runs
ts-node src/run-flaky-detection.ts 100 # Custom runs

npm run report:open # Open HTML (macOS)
npm run report:serve # HTTP server
```

### Individual Tests

```bash
npx playwright test coffee.stable.refactored.spec.ts # Stable only
npx playwright test coffee.flaky.spec.ts # Flaky only
npm run test:headed # Headed mode
npm run test:debug # Debug
```
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"test:headed": "playwright test --headed",
"test:debug": "playwright test --debug",
"detect-flaky": "ts-node src/run-flaky-detection.ts",
"detect-flaky:quick": "ts-node src/run-flaky-detection.ts 5",
"detect-flaky:quick": "ts-node src/run-flaky-detection.ts 10",
"detect-flaky:thorough": "ts-node src/run-flaky-detection.ts 20",
"report:open": "open reports/analysis/flaky-report.html",
"report:serve": "npx http-server reports/analysis -o",
Expand Down
2 changes: 1 addition & 1 deletion playwright-report/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,4 @@
</body>
</html>
<script>
window.playwrightReportBase64 = "data:application/zip;base64,UEsDBBQAAAgIAM+bIlsbK7Bv7RMAABdhAAAZAAAAZjAwYWMxYTJjNjZiZTYwZmMzYzMuanNvbu1c/XLbRpJ/lTnUVonyiSAG32BWqbJlO3Gtz1EsJbk70/EOgAGJFQgwwEAS/fFPXuJeIm9xb5InuZoPAAMQlECaSrxXqz9sEMD0zPSvp7unuzEflChO8ItQmSqRpqEAIj2wbR/bWhQYgaGcsOev0BIrU6UgyE/wmOCCqMUKByoplBOF/iyU6ZsP7GorqTHyIssMQh1izw8wtBEONdo8JgkjvsjKJAR+nt0UGKA0BCgMQZBFEcaAZCBAOQE5TmLkJ2vlRFnl2T9wQMTQgkWeLeNyqZwoSRYgEmepMv3ABr914EmcYmWq2ydKkCXlMlWmzqcTJSxz0Rzq0DJOFJSmGWG36CzfnigEzcVVVpIgY/2XKb5d4YDgkA4NkQV/IcdFmQjubBAuCMrJZcza65pujTVvrOmX0Jka2hS6qq1b/61QGiRfK1ONNsArwWnBtCc4ynIMvs2yKzqj+yk6lGIzEgvafWSfx7ekzDGYKRyOfKYMoW5obeq6A/uov0RlGiwE0vkgwrBLWOLG2xMFEYKCxRKnRNwIsjIlypR2fxWvVjhUphFKCvxpp5dP+jgSZCnBt2QQRzxNbw8c9qJ4lmNEMBCUB9E12nSdP40dKzTHw3gBO7zQjV7pEMygdAdRNbpU/xDR2Jdxr9B1PKfzIxmYKZMBnPNUaJodKYLaPYDvqgPdRgdC+9P2qZwoRUp/E2WqAAB0B3wE9G8yARcErwCcAnmGi2yJKZDgJiYLsMqzFc7BDYoJKEiOCJ6vZykl4woyiD2jLdR5RrLR0YKQVTGdTKI8xuECkUINsuXR8VesGQCimfT3MyfogY+KzPafKN0oy0GSoRAUhI5vpqSY3GT5VRwmA0RY11Td6oiw7XoHxcHQ98PBgA0O0qQATosyxwVASVLdBzn+paQWG6AcgyhO42KBQ8Y1Q9+EgV49z/KXGQovKNdGRxL9+4EwjA4Q35c4XwM2LcCYk+WjI78kJEvfxOHpjLoBqzhaj1fBu3c+SlOcv3vnk3Qc4oDyaKa8PToeAJW3sWaMwyIleQ2GtgNSlmBVkKUFAas8vkbB+imfHDjlTN+bMRwOwxZ9xBEYCSxb/ahssKNj8DXQjsGHrRj2IurU7/WTTuLganT8VQv1M3rzYfG23M7StA6Lt9Pgrbs74L0PFkN5PAg4AZtbv0UlL0uolM1HR0/jYhkXBQ6rPgAH4aiD4DPm3YKZQrJv0TW+pLcH6Eyoarq5k6uyIy6mIeFiDsfF1BuN+c133z0F568fn12+OHs2BT8U3PsYJ/gaJwAVBc6Z988MiI+KOADBAgdXBeOsabS0Jt8GjCiBY1Vi1mjyvLJhkwHIcdRMk72xDYkn+Me4iP2hSHS9aKgfFApLMl7GDlBYcCsUtdnGCaYDoz6Fj8E1nzXw+e4nTgnOUUCHyrhm6X2IUEVzxnaTL+P0imJTs2/oUpKhsdqwt6nfpQXnmDxZv84SPDpK4vTq6AR8ACla4imY/MwpFH+ZxODTsRrFeUFG92s/qGpmxws2D7vKLHmV7eCX7I3EMO4OAE2AZW7zBwuc4IAwx1AYpzdBgori0emMRhjCMiDDTBBUNa+z/bUPi8GeProl+egv0oJgFIIsAij3Y5KjfA1IvMRZSU64Y06ZIiYO5nkc0iWHViuMcs7JHldd+IgXgpVbWEjlnDneU3AkFvAR+HSvB2ltdeVr6O5YUUGWZEs/Rh+vcLpGHzFZxNkqRh9XOC93XmTeRvDkoADbsAHYtYYDbGtb9efjMIzpCFDCLRYYNyAjAhKMCgKyFNeIt/Qr478Ne/ZmB2B4JTWjHaXiIH9ctGy9I1p7m1ZodrxP1zmsArY9ybbu4H7a7lbZeMb2hrVljQuQYxSutxlV2+tT5QxMrpnPuQh9hmFtkHG0DjKdDQR6s8hxdDpTJkJwi1rwxgHO82yM3pfJOEWkzFEynuNigcbZMo2ZLh+85nVV4yFYybAedsPv7LmNdKwGV86bLAVk0VrI1zG+ASEmKE64p+rYLQg3sdttY1GD5Qw3rnznp67GlkpDneNkrvrzsZ+g4GqAjdVVzensJowDA+Jq+1lZx7vfiaV9xVEcVGuuAGSBCIjTMA5YwGzBNx71WmTsdbUBFvcuvu5peF24XTvSXc0Pr18O0I26aupd3/SwgO25AXS3bwB/xHkcrcENPspxta6CLM/p7Kv1xcLejE33bv9+eP1yNJnNalU1m/Upq+GbQrfrzPbG1ML4WhU9jrMV27+qS3/sMvWXEEwN7wIVl/iWTMHR6wwVhErFIEC7xu6wgHr7RVrcytTxyFpOZ3SBmQHbiKvtyRyOkFst9SakI3e2T3CNA+tp9VvSHJ6wpV2A03YvHZXajdccUiY2+xoiJYazPfV1ADHxJG/Z3MFb/kwmc6BgH40zxuxTIImEIFuJxJ5erZCOrrd6AIgv4vd46Ko3nYdU41AzpWXvDMYTakZr2Rfxe3yYVV/xhoEGNXNj0Utd7bHmf+ZkrY4kUaKNMMpdPPyCr2a833q34ENGP6AmecqmsYN8fA6HOUZ2D4X2YpdoftZaF0LRdah7sY3TVUnekPUKn86UtFz6tEpjt92NaT3smoayKfeGYwbbpvyXEqUkJusXdMYbq/o+PnAQ4abRbpHdKyXG4dIbi9JHOIqTZHSkHx1/xbLkuHlON2m60i6rSBIwU/SZclic7W4ZymEj/1DX9lPee6Dymewerpr1tpGvM2YbJO9IlT3Bz1LKtnDYTqm7uT1sChPqlhRsGO4yQd1sLUcUhpfZGcqFd7MlaT1doGJM96KjmfI4DKsKvplyXOlV3erbOXWo83CS4OKOWlXgaHe06TeYADowXm+1zVyu8jjAfDNt3ibDTKALH1Sd6t5+JlCU19QVBwF+JoJ+d3lIXQYI0LwWLfbqOX2ztoVyB6z5GWfziK9IrM7VEzBTfv/1N4sWss2U/Q2lodWDqZbn38+rDTodxhT85YM8yE9/H1KYcJfoDhIDu7M/PuwGGRpwP227F7s4o9tJgM4a3SlsKJDrbmbuiBlSJE5nytnj15czZRAClmM9aPEPlKtBdogSwroaZDJpwoKssDnM0Q3OJ8ssREk75QbrapC7AoBtJu0X8oOiSGSH8i86EZFIYNOIyiRZsyI7HB68AE9XLddrA3vYmAI05Bzr8GDioSE6MBBbUBeYd/OreybBesBx7QPvAW1nr2wJtOmyE6y1KVQtj4O6AXFaYtnh2Dl/xZnJ0m2ceJvqDmVxsvU5E0SoRjijCdys5GZIzfJRkw0b1OR+xWmoptUxXZZxYNtlSytM13eAcA/YBsBxD7oC1WHVxPTHgIViqJ7XNU+mcdhSAuhILgLcgc008doUc+tS8ioQslRnPqADa+7KKqpKd1QNaL7jI/2NcjKhWk/Um0wB1DRN22qWOO+dA+XoLdXc+PZFMw6bpIdyNpelLmh2J6eDpv9PwazUNOi/MeCSjjMElJ2WpmnLgpVlxOmcsbp6zVuKK30pZL76rS8bUkIN9DQ6Vpv3ZW5Vz0fHEr10lr7klKaNRipInqVzWb38/utvEDqqq3E1JGIQs/SZ+Ppp2hSPvMYBjq/prUUchjidpWe0Dj3J5tNZWg8CgDHox5R/NCCkRfCpNWCZhMy/3YffTxUAD/zv/1TkQI6LLLnGbD/wV04asEKn05nyDnrz2FkTDVRXUG8uUXOZzJSv60H8dcLJfL11AHRqzadl4BolJVU1gqGK3E4Zvsjpxpvv4EhGULJtNziUfVzZsuqCloaWie9lVoUKcMRw6xDMJSXMtyzgmiZrYypl9eC6AZnHEcH5sG/kDE017Y52trzej7YughzjtFhk936wxYh2DKtzT2zgIT7CMjTVMuFODvQDfRp3yJFsvKydwIEj/ynLr3AOzhKM0nJ176Chqlldy32ozyfvI/6gX7IxM8WnsMRFQb+8+5fN+pfNenibNUu5hjdlr9OoK3Bqn5NICp+3OLwJ+/oLM2G8oQs+8gsed62ZZNZMIugWxGmU5UvEizVbYCACJj8UOC8mK0TiSZSgqzXzXschJiwwMSnyYMK+4Z/0OLfUqZ3qbgMaczNCHOX8u/vdywopL/iFVJvKJmVL242qzorXL/KyOJKxDRdv/E9ab0er6vjFF1p/RqvL+IXVgceZgrMsjeI5NW3VOERVAW8hRTe/RSn9AJXVEJ0AWllwws50+L7KnPE4b1Vd7EofKTiPpzxywRrw93g5jeiNZguXZULiVYIBukZxgqpq9S+5COzPrzyiJUT8wqjJ0PFLRL4GsB6vKam2yQT8R8VywSwwenZx/vrZxcV3k+cvXl4+e30MxhVc/AZbhVSbxOmck7QkknwCnNV1KrE1+B4YnrNbDQ6eLVFssJCp9mLhtNR2XyNZZ3lu6/VOPjjhhk7Mmc2gxsuTGn7i1R2NGIgbFabip6QYnCc9S4EWm4CRvAYEIMe8/RdaEPXnVuKwohp+4bZkvyHRiD7UvLbo/w3jFQhxhMqEAN3SvnkN6lRVY2YoKV5som2IOX2mW5qY5x0Czqg3bIawV74FuT42Q71HtOv3JamG0Bgg1Xy2tH2FAjQ3ZBpaHZmGdkumoazez6atsgz+whdR8XOA2pKdykZ4A73FKt1oWMXMrss+45I/7GDJ6PQLLMxgtRb8QgK8491ckIz5D9TzFd8aoTCk+6DqkCdqNbhPHKDKQ/hnKGL4YxLsLHfOL7qi4tW+YyUtN6igxHDYkhvD3IpOOyudZqRGAVQ3y1WICFd09DQLfvEFJLYfIGvKsqPswtQ6vIZas02lTVtbMNoAVhs2bsArROh3kiBOGzD4SQSTiRBHel/IDD3abEO6K34IDx6cUQ8ePH5fJuAV/9wNfEM/dwPfsc/dKpEXpxRMJu0cWruz3iSaaVYssKoZiXnX6qw1HXtjOrXPP0yRv2ERg1NWBvn2BIgJf3/5X1Og0wm11bzpiP4aNS/32aflTVc0kQwOG11PczbEH+mA6g6FXWHtBR9/IWvOO24d2HtWY0smk5YpqPlRbfwrBtZNoRggNwlWteAto4MA8zUnzCGUPvSDlineazB4yh9vEycqLGBCjf08b4TGsrYJjSDXKzAWkwB+6XQGXJQ+C6nwpy4P+1Qukni2bYwXVVsqA/VDVZV/sVcnv//622w2m4X/Tv9V+eWknpQnmNs+k0AQ788jay2Zrt59vGy7nuJ2x15wClCiUNsGquHrZtQ2tCnX1sGuBMA25EAZZBvyOttfRcv4m2YdIGunvgd4BXdWHXDqXDDahIaVIfD2n1mQMTCzzzL3PHz2p+W1WQabXxjVxf/jcCcQaYM7kwWiNzHsTkd/fMAUAOhqkkdI0O2LNMo2ubtCrXXyIg2Sknmuv//6m6c6BtXjBN3iQnKmXSg5JRWbeQe9ki3igu25o1uAuLJpTZ32WfdTSZdrtidubTiFwlQwqyPrDBHxE1pDPKn81y4zVuo7SG7df8wd7R2McjTHy2WuV/eC6sKvHq7m1S1o1VdhfRW1lccw56Zhsr3J5M4E+pldBQXc9u6CtdhmMu+eOaouvOpiGfXMHPfPvG2Hm/l5m/Nrj7J3et7mfuTovCMELR9EsKzq1hPBaujpbZmyW0eLbOzVqvBiY/fjp88ev6Rx9kLEFfh7Yh/C2R6HGCV13KHL9QT5OHkTZfnpTGFncI1fUJrUXwwRQSJ+Na7on84UkpfSqXiwij22mNjqs5+HdjPGmoliPiu0Zmp4ickiC+u51T1W0lWFD1s0zrLlKsH0pNMyDxaowCBKshuaLotDvuEqyiDARRGVyb/VJIVB00UUURgZXcQQ+bVevWNUF8zgPHrEfzA+iGs2uUfgb3gNLgjy44S6pefs3I8AF+ApXlJo6FGd4ZS3cHgL/sPlP6AKwDk/3PMnrubFyx5/Dqps5x17MRDfcTYRc2t1HlZrk6s2ijzxWofigizl4ilaQqll91Cxon3qzQTzmMaWg1B0Hl8THOAhtEdAVwFjYIJBNaKC84BHy0TX9MycsY/ocXtVnXsBRvVxOse8hSW1uJQ/VGna0OGXBc7H1VFo1Vw4BVuiUIftI5QkPgquqrNXY1yAsqDmi/p1om8ZXijgNVQAnuIIp0V8jSnQ8xwtlzXKUEaZb3arQ2XwbVwQnAbbjmrTdRlSevDeOT3Gu8BqjgLMZlkHuVdZwacqDt3mU9VlZIUMsnIGsKAZKDo7JhlqgEiwELPUZQh1AaGpAlYJkoNLXBBwQfIyoFUcfJK6DCNVgmN/PabFGSDIlg3ndRm7M77cafXAvHJ9QuyX7Bd/vQUURmmczqMyAdcoj5kw0WiBoCwjowtkLBWAVxl4TrO64BwRgvNUyJ0uo/Iqa7myl3xVCW4YWvvNBcpDmuINQYgTtObdG7D9kp/HhCQY/Oc5IgsWGmLq7BacXVw0csqb6u2mqCjKpUghIZ/5u/Gy4giPaT2a8B/cUVZojQqN+pQFPVsexQk7ZX3jXPZ2zcsHJRWn1suFWmIpXa5X9FG8RHM8WaXz+tB2hRqTCTJs3dM1z9B13Q5CzfQs13ccU7NMzdHcKHCRg4wQqrQprSgSXV3HIc42emF3JzfYX3a6saFhGi7SkK2HWuhEtm0jA1uWD30vgqbl6n7kOAZWWVupHybh4+a88HZ/9OZkifKrMLtJO136uoUC3XciD1vIDX3DwIbjh5Ht2Y7lhyYO/SByrVBdhnKHhK7HjY7QapUIuz95H6+6XQV2FCErCnUXByFG2AkNU0eeZnuGrZs6snTbN31dpU2lvgoSZuWWWa0SFNMp+Vm4VqbKttNLZ2lPcL4Tia1jt6ny6S2VsOyqLon69H9QSwMEFAAACAgAz5siW9YchjU1AgAAigQAAAsAAAByZXBvcnQuanNvbq2TzXLcIAzHX8Wjs+O1wcYfb9BLT5npoZODAJF1FoMH4ybpzr57B69nu00mtw4XEJJ+kvhzhokiaowIwxlQxRXtDx9OFBYYqksOS8QQH8eJYKjaRnS8ZlXH+i4HvQaMo3cwVJyLqig7noMZLS0w/Dxvu28aBjBliapCpoSQJEqjuOJw9fyOKW9iSEsPkZZYLDOpIi6QQzpeU6Xdl6kesDdNrTSrqJeKKoGkyxQ+RrslP/rV6kwG/7pQhk5nqHWmvDFEWfSZwhCzQHZEad8hhzn4F1JxL00dg5/GdYIcrFd7v9fmvizcjo5gYCIH5e06ORjayz/jYlXDc0DnfNxMqcunHCI+7zu/RuU3/urobSYVSafSMB53hxMMBu1COQRaVrsPCmNEdZzI7We3z1cFIrccfYRUk4vk4uP7nK7GCZ/pMLvnW3pIYjggF6xnZc8ZY0Lpsu6bTrZtXTZ12ZadUR22yHVVpNBLfkP9GjX5T5TNenglOX3AiIrXvMMSBdOlbo0QAjk1jaxkb6q66Zg0bcup2GLvOBSCDw8b5e1zV8l4mDCctH91H5CSNaiYbE1PDXZack68ldqIXrSN1DVpqUzX6GLS98AYUNEnEM6zHa+yOPwe548oJYzBxmjWkdKE1GpeM+xL0XPBaoYNE7KWrEihd6wlar9+0dVscXRweUpr+53pqc8QfUQLQ5XDTS9Dmd/LJ90Zi6f37WI5jfO8O93EdEkp7/SfRPT3B/x/XH59xk30lz9QSwECPwMUAAAICADPmyJbGyuwb+0TAAAXYQAAGQAAAAAAAAAAAAAAtIEAAAAAZjAwYWMxYTJjNjZiZTYwZmMzYzMuanNvblBLAQI/AxQAAAgIAM+bIlvWHIY1NQIAAIoEAAALAAAAAAAAAAAAAAC0gSQUAAByZXBvcnQuanNvblBLBQYAAAAAAgACAIAAAACCFgAAAAA=";</script>
window.playwrightReportBase64 = "data:application/zip;base64,UEsDBBQAAAgIAEWYLluzeOFSzQ8AAGFyAAAZAAAAMjllNTY4NjQ0ZTBmNDc2MWRkZjUuanNvbu1cbXPbuLX+K7j8sJKnskRQfFUn7SSOvcnUN94bK9uZrrMzNAlZrGlCJSEnbuz/fgcEKIIQJQG0lE079qdEJA/B85w3nHNwvhmzJEXvY2NiWAFyXN+1bWTObM+FcTxzjEF5/UN4h4yJUZDwOkXHBBXk+N4aFgsUDUlhDAz6S2FMfvtW/msjtWM4RlGAZsibhfbYdWdWENj08YSklP4JvlukiCAQ4dkMIbBY5tE8LBD4J17mGXowBsYix/9EEeELiuY5vkuWd8bASHEUkgRnxuRbueRty02TDBkT2x4YEU6Xd5kx8Z4GRrzMOQUIx/RimGWYlD/xb3tYlC8NCbrBOV1NjIooTxbsKf4+4+nzwCDhDX3m88DASxLhcrHo6wJFBMX0K0IyNya/GWd5guJ3ISnACfviX6ovPkvxF3AMLkuSYFry9/PAyFGxTDmr19ZbkDAn06R8mWVazrEZHEN7Cr2JCSdOMPRM+x8GpUHyB2Ni0gfQgn8aR+ANmuEcgXcY31JG7aboUIr1SsamE7TRvS7pnobRHMwxvlUi7a6R9tpInyVfyTJH4Mq4zvGXAuVXhhJ5v0keWk4b9fNwmUVzwEkrEQ5kwnZN+PPACAkJo/kdygj/IcLLjBgTODCK22SxQLExmYVpgZ60bh60cSTCGUFfiRJH/LHdXHg7Q05yFJYaWlJWoisBCf84fizCG6TGDNdsLtp3tzCDklUiKmmLD78zJz6E98kNXTHB4MoYKbEisD1JLuzA375ufVs8dmpbDN2nzV8zMIqM/p8YEwMAMLbBI6B/oxEQP26O71AJdkZvcvhN4ZcwISVawxtMcL83J2RRTEajGbXD85AUwwjf9Y7+XD4GAH9M+PudEXT5FUNk7mlp38GVQfAb9GtSJNfpTmGzJqY1tG3Z1O2Qiw789Wv+2qYGf72av3+nzJvhvGQgZfI1AjkK4wfGE7/BZObs+iWvy+XivN+7xvFD7+hoKDCov4XZm/44CEEbCO8LcM8ogxtE3jx8xCnq98IU5SROwhTf9AbgG8jCOzQBoy8I3IfpEoEHvMzBIk/uw+hhlICnIxXUfMlEQFMbteFwtCRJWtQCeDxH6QLlhQgeFJTD0gAP2uCxZBVkCpDMQJ+hcx1mGcqHSbFC4Qh824oD4zmsBD/CWUFAjCK6QvCqIiiw/HpJCM5EbvO7S/7++RCgqb9eBd5g3IQ3OBS6gYDuWANdv0K3UoQaX/6tOgCvMLbM1Q1NYlGaRLd9CboT+uOPg5pjSs4Kjg+llZYp4AY1cNOCayca20HlkEJ++QmgtEDCm8MoQgsivXizV3uXxDHKlJyaO94WYO8XCdE+agQPVmUfLafNdzGbxrwV+3A1Z8V5zkzlk9EarOl7cMurP9LX+Ea6DEpzyPaq16jf09p19gagfwRe/aWSRxYQlBTrTV0/LB6yCPS/sdjgaVDe8D6bYeFRkWucR34dXbyOY3Ay/XgG7hAJ45CE7YxTiXJ/RXkye1jFgArC6rnS3ge27mIFXXgX3qMp/VlJGdboe/sO8Gy3mw7YTdEvcaUr6/ckLvYGgIMsQGq7knUSwj6qOCsu9UdnlQaPEg0tsr3VPfuJtT3PkpDee6ztCI5h7GhAEay+lYVX9wn68jpN35ROEbziG5faW6ZJdiv4yt6v70//Dl6fn4OTi7Oz09PLHg20SjY6shfhODXeIMflQCcwZ3A5K0dDw4Q9SXbHraFtryxVv7crs1hLd2XCRCl/lo5s3kZ21R5j05aef1yE0xRFJZcVFMKXTZ8pJmj2oXK+8502SmIWQUfzeBah1jrmF4s9aB7PPaxpXeMNz9kRNzIS7DVr9BVDd5WvUsE7kNIZzvhQeItOz9PAuzMoinxWAZADxxzcJudGTcCnj+dKeubbUuY5OFjEPRb3rJYG41nAJyaT1mwWqPN2wTpMom389PG8P7q6GtXPFuX/SkRGO1HgRtjcCoCOqRsPPU8yddahRN+2BK9oamw/bVhBwJ1X6e7KdH5GQFKAFIcxisExiOYoui0xCglIUVgQgDMEFjmOlxEB1FowDlrtMG0xmBQyfHedhI+3KHsIHxGZJ3iRhI8LlC8fb5YhQXdhGrL8wCzJC9J/ftrwWX9cWMbgcX9bOmoXu0Q2jsU/msVc403RydbQQDNWcWwpVsk48SlmJvBkRZnpaNPXXCJ6EYT3YZKWe0u2HhWVCuR6GnRbC1P/t0T5AyhZuMWp6Uiewup8E8pZDv3aiJrGO4LGj3WkBQrBDVfe8yS7LXbHNTrM4hsNS3gZw+JVVXYRXj0sL+nuMSpRHK9b7P2DX5mdunoxHPZ4snKKvpL+6PLi/C24+DQdJSqSEowlSXF3lEG7C4rgnN1AQ1CqbAwXlTBHGTnBGQmTDOVUWmoEm1zh2AeN5+dhcYnT+GIpiECT5AZmVsIB/gLMPVt5JkButRtuiNDPiBrPr2TlC/crRyoiEsiVVvdAIuIKnsfW2Ci51gqI0QhQjlXRAGUMiJMcRSR9ALMc35UBQslRxvTx6tGGKaKdPG02YkifPGFQ9I/A4yPoMUfT6ygTHPvaj2lthg4CuT00LSlgdA4GubhX0oHcESFnbMIZIPOWSNCVExoinup7I47U7tSf+u7IHpq+lPjzfetQrA665WJdX2R1XW7nbF7ti9xAYvOWvRF/mG6Mdu+IauZ7psD8PUW7fsdo12tEu/6maHdDiKkb58pJ2KIki2JeKqksFfv5jCr66+qN7BYe/krmBWez5IY2IVVo4gVrLNwtuYFcPBAFd0sIVDrW3seL15fTnhTGKL1VskyHMky+IBeWrS4XPpOLEjXflxocchwW5OccLxdiNKFQA+di4FdiMBqBaf5Ad1AMcUa5Qg8kGcB5jHKAZ2CRoxnKURah7aHpFly21sJ7p5e/fDy9vLxQysPZw0AudegXndQQDOxuabhgLGp2UHnmGkVULHJUFPgNyUQYwU8/gfUbkuI0o1oYK/U61FAHjmxNxbe2JU73h+jZ+/Pp6UdVPOWGo0NpZCCECmON7J4SJxnPq0BhrSFhlqQE5RsBry/rwy1i7kkrFV57WMQv/vfDe2W83e+kv4IFHmv0lylwkXHb34Q2vsuSjVhXFzshzXGWI6XVCzVQvnz/j9MuDlRuIDtUMh6aYmiv3kEGTWflQaHpSh60SP6NujjQ3xm5SjR4EJX8G1mO+YbQYlpNeaumWI75c2kaO4OkSF8Jy+D7qCKEZqe9AzQDwZdCaK750hqCdm0TrmsrHMMcQknZxFc+U9+2QwlNUxVLZ2jKLYIH00soZmzVI1slRjKOW5sMK32CcmUr3NUNHV3pCvdxy3JXL1cC/jy8Rml/9K9lmJGEPDzyWLv6P00K4rwvyECxSDIuB0fltZW0JNliSX6jR7VeXRnZ8u6aHgj63DtSEg347DYtRdGwrE4hM1x1cpactyzJaFcMO0tQGuvufFaAWs2KUpPoLEnTfs/qSZCeJWkKrgzryvixELXk7pZDZd2hNe6UYNMHURUfxUQbtOwGOZ5MalCtskq/0hbuFvSlnFx5m0JWzhma4+/lWC1hkzrWCJI0mN2Rk+oGl+Pl7Lez2OtYhvYaZWhvYxl6Y+5LMzXnySVoZlN+YUQvGM224jNtLaYl8DDfcV6RiaR8GguWAfaOhjjuQdWEXj5RZx8qEw3FDikdqecdUqMROM0KCh0zzbQ/BLEvBaz7G5QOPsluStFstEdx8Q/jeIpPwpyIbVWrgENb+tsaptaKN+vnSF6/fQumF+Dk9cepWm3fGUL5cDM0zYN5j0ZHlYb3qDuqGBO4rFN+M3aJPVQSFLolGbjHXil3CKFUED9YtxoUT/pb6ocnoC0mKCFvKG4IN7Upb/PwS3VORLM9ibPV2Sdb187dHKoFDdrCkZRAQ2Z537PQgsb9wpewoDJadp+lGLPms8p9giSLkzIkBEJRLMlmmDHR24aO2GpwdRX/aVSGmO3X/0UeHoU49Q9vQONC4u+z9czrWIzzGsU4b2MxTnC5ul5eTtmFccxdPLNdbR7+lxxHCLFX0q5FvNzt6d0hdKQCiW2LJ/L3oYyOlAO0AvNwp4T9Tml76NSFNMjLq00t4hx9Zq86E2NH9EdNyopd6uu+/eTiw/T9h0+npYN/d3ryt4tP08fqH6wjo7nXlHs7dhPYHS8EQ1MG2w4OtpsRe7w0iqYd8VVCTa2yCt3tMYR6S0cwdBz5BP6hKmLQFfM1Grrlrrqt64Z3zj7WeH1M02ApjabB5RwvqDek9ywLgu9WtzLGbWizljs+VngWf726Gj1SE3yk2AwP3ZbWyo7GLxg6rnwW23QPB5Bw9MfTCPF4P1gdizThQSkqlwfCcvuKCpRxNFqOkNTdtCyQKBsdI/J4+rUsg65IP75FaXKP8jIL9n2DCw6zu89Ywu+YP/Ab+QN/Y/6gxblrxhS+nDlYMJJTfMIJtqYNZgTlKqOwrAk0h+5YLvfZrWeI9YYRcdJSRmJHCHGQGVH7XYnuzSjPcc7vK0hIloUxMRZhUZTj1J4xqE1aBf0J3xoTki8Zu7aPs/PN0I2hGQXXjg9j2/XcmTDO7l2YxSkCBU5jUFoU3osHbvIwQrNlmu55op0vRAXNiXaOSVt+/2MG2rHl7h61Nj70PDsrMP02unuYZ2cF7WnFvc2zaz3Fu495djsSCj/MPDvJILfPdOgwz06SkR2DgX6IcXby1iQYb+GF6jg7R4rv/O/NiT2Nsxvrj3Z6GWe3+RzyOJASuy/j7HbH5H/8ODtrbbf2Ms7uv2icnW1+p0L7yzi7faK2Ns7uUBXIl3F22wcL+muZ/Jdpdi/T7Bo8Osw0u85xmCt3M9qHyr6+zF36EeYuua4Ud7/MXdoC3P7mLnlQHuseHEzRXuYutU5OcKFUybf0t5wvc5d+/LlLOw+PaA5lWSvaQP1OzB2RTiD6Rg15WR2HbR5SFY54sOEqvDdmNVIHvHr1Srnfv+ZwUI9uYKW322TRp1WQAeh9wC01DILLG3trJ91bvRuVJno2sP+NzoWZbEBMb1yKKzfqOnsHD5piMK5RVjZF9KqDdvUxybwgfDaO5miM+oDdrpydugcbD91ALh6YcO8TiqHZ7aB/fZZw21AMaG5M2D1zJAbnd2t6rqPDsocelFvA9Ztfd/GbTjZezYHSOGO0OjxY+6pKRwGtvSLqrYo5/pI1j/1tdUTrqZBa72l/1QGdzO/Ns3LKQ5+2NpE/NpavgrgrD57Wj853Iu52SotD6IjWSsqKMjZQW70afhLGMQ+8G0OaOqa+IfQE5ya87qefwP/U/x0mWZQuY1T0exXve9vTO2+TQvVQhj30PFerrNcFHr9TKPBM/jAW16OF5MMZ4q6r4pgGlhzCyjw+dW2hoRDIAw+gOJy9ewtNSdr7EVpo9rmS/4gWms9P/w9QSwMEFAAACAgARZguW8iEp4i+AQAAhwQAAAsAAAByZXBvcnQuanNvbtVTO2/bMBD+K8LNtKEnZWkNEKRLUaABOgQaGPJoK6ZEgTy2NQz994KSAqdD0iVLt7uj7nvcB11hQBJKkID2CkJSEOaHdWd0Htp8ZuBJOHrsB4Q2q6v6wJsia4qsZqCCE9TbEdosa9Jqn/Kcge4Nemifrkv1RUELeYMVP/CyxFSXNc+U0hWsX34VETdyPBvcEXra/cz3fkK5Jw8M4mRFi9W7aLusQNmgxlqLsuBc501TxvWeTMS/s8NkkDCRVmvEZApOnoTH5MUGN+IFGEzOvqCkTZA8OTv0YQAGxsrN5WrpI7mmHxHasmQgrQnDCG09/32nIj6KcbS0jDZvl2khFYRH66IahV66flq3Nj6YOwYkjnGnY2ADSbuIxd8TSkIVXQg6QfsE965H9SDIJ3er42+vju+N/ZXsku8LZPK43DeinaElF5CBQx/MdnRBJORpwHHpu7mb2T+TOKSCqyyVzXN1yFTJa67fJPEgRmUw8daoxAZKJmdVkOSToxMSdTDmk8M4NO+FUaW8+K+z6JafM7ZXIEvCQJuzm4LYhPHWpgy0EefLUvlzP03b9JVvjohvTh95bsf/dDYG6Jx1q5s/UEsBAj8DFAAACAgARZguW7N44VLNDwAAYXIAABkAAAAAAAAAAAAAALSBAAAAADI5ZTU2ODY0NGUwZjQ3NjFkZGY1Lmpzb25QSwECPwMUAAAICABFmC5byISniL4BAACHBAAACwAAAAAAAAAAAAAAtIEEEAAAcmVwb3J0Lmpzb25QSwUGAAAAAAIAAgCAAAAA6xEAAAAA";</script>
Loading