Skip to content

Commit 54937ae

Browse files
hasanalituranhasanalituran
committed
feat: Update documentation to replace CI/CD integrations with CLI Reporter and add new feature guides
1 parent 3585c8c commit 54937ae

26 files changed

Lines changed: 328 additions & 257 deletions

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Official documentation for the Testream test management platform.
1111

1212
- Playwright Reporter: https://testream.github.io/reporters/playwright
1313
- .NET Reporter: https://testream.github.io/reporters/dotnet
14-
- CI/CD Integrations: https://testream.github.io/ci-integrations/setup
14+
- CLI Reporter: https://testream.github.io/reporters/cli
1515
- Jira Integration: https://testream.github.io/jira-integration/installation
1616

1717
## Packages

‎docs/features/dashboard.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
sidebar_position: 1
3+
---
4+
5+
# Dashboard
6+
7+
Monitor your test suite's overall health with an intuitive dashboard that displays key metrics, test result distributions, and recent test runs — all at a glance.
8+
9+
![Dashboard Overview](/img/jira/Dashboard-Screenshot.png)
10+
11+
## Test Metrics Cards
12+
13+
At the top of the dashboard, you'll see key metrics:
14+
15+
- **Total Tests** - Complete count of all tests in your suite
16+
- **Pass Rate** - Overall success percentage
17+
- **Failed Tests** - Number of currently failing tests
18+
- **Skipped Tests** - Count of intentionally skipped tests
19+
- **Flaky Tests** - Tests with inconsistent pass/fail behavior
20+
21+
## Test Results Distribution
22+
23+
The donut chart visualizes your test results breakdown showing passed, failed, and skipped tests. This provides an at-a-glance view of test suite health.
24+
25+
## Recent Test Runs
26+
27+
Below the metrics, you'll find a chronological list of recent test executions showing:
28+
29+
- Test run timestamp
30+
- Duration
31+
- Pass/fail counts
32+
- Quick action buttons
33+
34+
## Action Buttons
35+
36+
- **Releases** - Navigate to release-specific test results
37+
- **View Trends** - Access analytics and historical data
38+
- **View All Runs** - See complete test run history

‎docs/features/failure-inspection.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
sidebar_position: 4
3+
---
4+
5+
# Failure Inspection
6+
7+
Debug test failures efficiently with rich diagnostic information including error messages, stack traces, and visual artifacts.
8+
9+
![Failed Test Details](/img/jira/Failed-Test-Screenshot.png)
10+
11+
## Error Messages
12+
13+
Clear, formatted error output explaining why the test failed.
14+
15+
## Stack Traces
16+
17+
Expandable stack trace sections showing:
18+
19+
- File paths with line numbers
20+
- Function call hierarchy
21+
- Error propagation
22+
23+
## Artifacts
24+
25+
Visual debugging aids captured during test execution:
26+
27+
- **Screenshots** - Visual state at failure point
28+
- **Videos** - Recording of test execution
29+
- **Trace Files** - Detailed execution traces (Playwright trace viewer compatible)
30+
31+
Artifacts appear as thumbnail previews that you can click to view full-size or download.

‎docs/features/issue-creation.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
sidebar_position: 3
3+
---
4+
5+
# Issue Creation
6+
7+
Streamline your bug reporting workflow by creating Jira issues directly from failed tests — with pre-filled context and custom fields.
8+
9+
![Create Issue from Failed Test](/img/jira/Create-Issue-From-Failed-Test-Screenshot.png)
10+
11+
## Steps to Create an Issue
12+
13+
1. Click the **"Create Issue"** button next to any failed test
14+
2. Review the pre-filled issue form:
15+
- **Summary** - Auto-populated with test name
16+
- **Description** - Includes error message and test context
17+
- **Project** - Select destination Jira project
18+
- **Issue Type** - Choose Bug, Task, etc.
19+
3. The following custom fields are automatically populated:
20+
- **Test Run ID** - Links back to the specific test execution
21+
- **Test Name** - Full test identifier
22+
- **Environment** - Where the test failed (staging, production, etc.)
23+
4. Click **"Create"** to generate the issue
24+
25+
The created issue will contain all context needed for developers to investigate and fix the failure.

‎docs/features/pdf-reports.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
sidebar_position: 7
3+
---
4+
5+
# PDF Reports
6+
7+
Generate downloadable PDF reports that summarize your test run results — including pass rates, failed test details, error messages, stack traces, and screenshots.
8+
9+
![PDF Reports](/img/jira/Test-Run-Reports-Screenshot.png)
10+
11+
## What Are PDF Reports?
12+
13+
PDF Reports provide a shareable, offline-friendly summary of any test run. Each report is generated as a professionally formatted PDF document that you can download, share with stakeholders, or archive for compliance purposes.
14+
15+
## How to Generate a Report
16+
17+
1. Navigate to a [test run detail page](./test-run-details)
18+
2. Click the **"Generate PDF Report"** button
19+
3. The report is generated asynchronously in the background
20+
4. Once ready, click **"Download"** to save the PDF
21+
22+
## What's Included
23+
24+
Each PDF report contains:
25+
26+
- **Project Information** - Project name, test run metadata (branch, commit, environment)
27+
- **Test Summary** - Total tests, pass rate, and result breakdown
28+
- **Failed Tests** - Detailed listing with error messages, stack traces, and captured screenshots
29+
- **Passed & Skipped Tests** - Summary table of all remaining tests
30+
31+
> **[View an example PDF report](https://files.testream.app/reports/019c1925-214e-798c-b7c5-ec62b875ecd4/85cf3ae8-002b-45e8-aafb-dee7d1b7cb33/report.pdf)** to see what a generated report looks like.
32+
33+
## Report Statuses
34+
35+
Reports go through the following statuses:
36+
37+
| Status | Description |
38+
|--------|-------------|
39+
| **Queued** | Report generation has been requested and is waiting to be processed |
40+
| **Processing** | The report is currently being generated |
41+
| **Ready** | The report is complete and available for download |
42+
| **Failed** | An error occurred during generation — you can retry |
43+
44+
## Retry on Failure
45+
46+
If a report fails to generate, click the **"Retry"** button to request a new generation attempt.
47+
48+
## Availability
49+
50+
PDF Reports are available on **Starter**, **Pro**, and **Enterprise** plans.

‎docs/features/release-management.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
sidebar_position: 6
3+
---
4+
5+
# Release Management
6+
7+
Track test quality across your release cycle by linking test runs to Jira releases and monitoring quality metrics per version.
8+
9+
![Releases View](/img/jira/Releases-Screenshot.png)
10+
11+
## Release List
12+
13+
View all Jira releases with associated test runs, test run counts, and quality metrics including pass rate for each release version.
14+
15+
## Linking Test Runs to Releases
16+
17+
From any test run detail page:
18+
19+
1. Click the **"Link to Release"** button
20+
2. Select a Jira release version from the dropdown
21+
3. The test run will now appear filtered under that release
22+
23+
This helps teams assess release readiness and track quality gates.
24+
25+
## Link & Create Issue
26+
27+
You can also create a Jira issue at the same time you link a test run to a release.
28+
29+
![Link Test Run to Release](/img/jira/Releases-Create-Issue-Screenshot.png)
30+
31+
From the "Link Test Run to Release" modal:
32+
33+
1. Select a **Release version** from the dropdown
34+
2. Toggle **"Also create a Jira issue for this release"** to Yes
35+
3. Choose an **Issue type** (Epic, Bug, Task, etc.)
36+
4. Click **"Link & Create Issue"**
37+
38+
This creates the Jira issue and links the test run to the selected release in a single step.
39+
40+
## Filtering by Release
41+
42+
Click any release to see only its associated test runs. Use this to compare quality across versions and monitor readiness before deployment.

‎docs/features/test-run-details.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
sidebar_position: 2
3+
---
4+
5+
# Test Run Details
6+
7+
Click into any test run to see comprehensive metadata, results summary, and a list of failed tests with the ability to create issues directly.
8+
9+
![Test Run Details](/img/jira/Test-Run-Details-Screenshot.png)
10+
11+
## Metadata Section
12+
13+
Each test run displays key context in a two-column layout. Some fields are always visible while others appear only when data is available:
14+
15+
| Field | Details |
16+
|-------|---------|
17+
| **Branch** | Git branch name (always shown) |
18+
| **Started** | Timestamp of test execution (always shown) |
19+
| **Commit** | Short SHA (first 7 chars), links to the commit in your repository when a repository URL is available (shown when available) |
20+
| **Environment** | Test environment, e.g. staging, production (shown when available) |
21+
| **Duration** | Total execution time (always shown) |
22+
| **Tool** | Test framework name and version, e.g. "Playwright 1.40" (shown when available) |
23+
24+
## Results Summary
25+
26+
View aggregate statistics:
27+
28+
- Number of passed tests
29+
- Number of failed tests
30+
- Number of skipped tests
31+
- Overall success rate
32+
33+
## Release Linking
34+
35+
Associate test runs with Jira release versions using the **"Link to Release"** button. This helps track quality per release cycle.
36+
37+
## Failed Tests List
38+
39+
All failed tests are listed with:
40+
41+
- Test name
42+
- Error message preview
43+
- **"Create Issue"** button for each failure

‎docs/features/trends-analytics.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
sidebar_position: 5
3+
---
4+
5+
# Trends & Analytics
6+
7+
Track historical insights into your test suite's health with pass/fail trends, duration analysis, suite growth, and flaky test detection.
8+
9+
## Date Range Selection
10+
11+
Use the date range dropdown to control the time window for all charts on the page. Available ranges depend on your subscription tier — some options like "Last 90 days" may show **(Pro required)** and be disabled on lower-tier plans.
12+
13+
Default ranges available on all plans:
14+
15+
- Last 7 days
16+
- Last 14 days
17+
18+
## Test Results Trends
19+
20+
![Test Results Trends](/img/jira/Trends-Screenshot-1.png)
21+
22+
A grouped bar chart showing the percentage breakdown of test results per day:
23+
24+
- **Passed %** — tests that completed successfully
25+
- **Failed %** — tests that returned errors
26+
- **Skipped %** — tests that were not executed
27+
- **Flaky %** — tests that passed on retry after an initial failure
28+
29+
Hover over any bar to see exact percentages for that day.
30+
31+
## Average Test Duration
32+
33+
![Average Test Duration](/img/jira/Trends-Screenshot-2.png)
34+
35+
A horizontal bar chart displaying the average execution time in milliseconds per day. Use this to spot performance regressions or improvements in your test suite over time.
36+
37+
## Test Suite Growth
38+
39+
![Test Suite Growth and Top Flaky Tests](/img/jira/Trends-Screenshot-3.png)
40+
41+
A line chart tracking the total number of tests over time. This helps you understand how quickly your test coverage is expanding.
42+
43+
## Top Flaky Tests
44+
45+
*(Visible at the bottom of the screenshot above)*
46+
47+
A horizontal bar chart showing the top 10 tests ranked by flakiness percentage. If no flaky tests are detected, a success banner is displayed:
48+
49+
> **No flaky tests detected! All tests are stable.**

‎docs/getting-started/installation.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,5 +149,5 @@ You'll need a Testream API key to upload test results.
149149
## What's Next?
150150

151151
- Follow the [Quick Start Guide](./quick-start) for a complete setup walkthrough
152-
- Set up [CI/CD integrations](../ci-integrations/setup) for automated uploads
152+
- Set up [CLI Reporter](../reporters/cli) for automated uploads
153153
- Set up [Jira Integration](../jira-integration/installation)

‎docs/getting-started/quick-start.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,6 @@ You can also run tests first and upload results separately using the GitHub Acti
169169
## What's Next?
170170

171171
- Learn about [Playwright reporter options](../reporters/playwright)
172-
- Set up [CI/CD integrations](../ci-integrations/setup) for automated uploads
172+
- Set up [CLI Reporter](../reporters/cli) for automated uploads
173173
- Set up [Jira integration](../jira-integration/overview) to view results in Jira
174174
- Check out [.NET reporter](../reporters/dotnet) if you're testing .NET applications

0 commit comments

Comments
 (0)