A modular Playwright automation framework built with TypeScript, designed for extensibility and enhanced with AI-powered capabilities.
A reusable end-to-end automation testing framework built with Playwright and TypeScript, designed using the Page Object Model (POM) architecture and enhanced with AI-assisted failure analysis powered by Google Gemini.
The framework currently includes an AI extension that analyzes failed Playwright tests using Google Gemini, helping QA Engineers understand possible root causes faster through contextual failure analysis.
Playwright already provides excellent debugging artifacts such as screenshots, videos, and traces.
However, engineers still need to manually inspect those artifacts to determine why a test failed.
This project explores how AI can assist the debugging process by automatically collecting execution artifacts and generating an initial failure analysis.
The goal is not to replace engineering judgment, but to reduce investigation time and provide a faster starting point for debugging.
This framework is built around several core principles:
- Modular
- Reusable
- Extensible
- Maintainable
- AI-Enhanced
The architecture is intentionally designed so new capabilities (AI or non-AI) can be added without significantly changing the existing framework.
This repository focuses primarily on framework architecture and design rather than providing a large collection of automated test cases.
Only a small number of sample tests are included to demonstrate how the framework is intended to be used.
The emphasis is on building a reusable foundation that can be extended for real-world automation projects.
- Playwright + TypeScript
- Page Object Model (POM)
- Reusable Base Page
- Custom Playwright Fixture
- Custom Playwright Reporter
- Automatic HTML Snapshot Collection
- Browser Console Error Collection
- Screenshot on Failure
- Video Recording on Failure
- Playwright Trace Support
Current AI capability includes:
- AI-assisted failure analysis
- HTML-aware analysis
- Screenshot-aware analysis
- Console error analysis
- Root cause suggestions
- Confidence estimation
- Playwright best practice recommendations
playwright-ai/
│
├── .env.example
├── .gitignore
├── package.json
├── package-lock.json
├── playwright.config.ts
├── tsconfig.json
│
├── fixtures/
│ └── base-test.ts
│
├── pages/
│ ├── base.page.ts
│ ├── login.page.ts
│ └── dashboard.page.ts
│
├── reporters/
│ └── ai-reporter.ts
│
├── tests/
│ └── *.spec.ts
│
├── utils/
│ ├── ai-helper.ts
│ ├── ai-types.ts
│ └── step-logger.ts
│
└── README.md
Playwright Test
│
▼
Execute Test
│
▼
Test Passed?
│
┌────┴────┐
│ │
Yes No
│ │
│ ▼
│ Collect Execution Artifacts
│
│ • Playwright Error
│ • HTML Snapshot
│ • Screenshot
│ • Console Errors
│ • Video
│ • Trace
│
│ ▼
│ Build AI Prompt
│
│ ▼
│ Google Gemini
│
│ ▼
│ Root Cause Analysis
│
▼
Test Finished
When a test fails, the framework automatically gathers contextual information before sending it to Google Gemini.
The collected information includes:
- Playwright error message
- HTML snapshot
- Screenshot (image input)
- Browser console errors
- Video metadata
- Trace metadata
Gemini analyzes the available information and generates:
- Failure summary
- Possible root causes
- Supporting evidence
- Confidence level
- Severity assessment
- Suggested solution
- Playwright best practices
Note
The current AI prompt and analysis output are written in Bahasa Indonesia. They can be easily customized by modifying the prompt inside
utils/ai-helper.ts.
================ PLAYWRIGHT AI REPORT ================
Test
User should add product into cart
Status
FAILED
Playwright Error
locator('#cart') not visible
======================================================
Ringkasan
Element belum dapat diakses karena masih tertutup loading overlay.
Penyebab Kemungkinan
• Loading spinner belum selesai.
• Timing issue.
• Overlay masih aktif.
Confidence
93%
Severity
Medium
Solusi
Lakukan explicit wait terhadap loading spinner sebelum melakukan click.
Best Practice
Pastikan halaman berada pada state yang stabil sebelum melakukan interaksi.
The framework uses the official Google Gen AI SDK.
The AI model can be changed inside:
utils/ai-helper.ts
Example:
model: 'gemini-3.5-flash';You may replace it with any model available in your own Gemini account.
For example:
- gemini-3.5-flash
- gemini-3.1-pro
- gemini-3-flash-preview
- or any future supported Gemini model
| Technology | Purpose |
|---|---|
| Playwright | UI Automation |
| TypeScript | Test Development |
| Google Gemini | AI Failure Analysis |
| Node.js | Runtime |
| dotenv | Environment Variables |
git clone https://github.com/your-username/playwright-ai.git
cd playwright-ainpm installnpx playwright installCreate a .env file.
GEMINI_API_KEY=your_api_keyRun all tests
npx playwright testRun a specific file
npx playwright test tests/login.spec.tsRun headed mode
npx playwright test --headedRun UI mode
npx playwright test --uiWhen a test fails, Playwright generates:
- Screenshot
- Video
- Trace
Additionally, this framework collects:
- HTML snapshot
- Browser console errors
- AI-generated failure analysis
| Feature | Purpose |
|---|---|
| Network Request Analysis | Allow AI to detect failed API/resource requests |
| Step History Analysis | Help AI understand the sequence of events before failure |
| API Testing Module | Support API automation within the same framework |
| Visual Regression Testing | Detect unexpected UI changes |
| GitHub Actions CI/CD | Run automated tests on every push |
| Allure Report | Generate rich HTML reports |
| Cross-Browser Execution | Validate Chromium, Firefox, and WebKit in a single pipeline |
This project is intended for learning, experimentation, and portfolio purposes.
Feel free to use it! ❤️