Fixed faulty k6 tests, due to jwt problems, and added smoke and spike tests#38
Fixed faulty k6 tests, due to jwt problems, and added smoke and spike tests#38
Conversation
Alexandre-A
requested review from
JPSP9547,
ThiagoAVicente,
alxmra and
Copilot
|
There was a problem hiding this comment.
Pull request overview
This PR fixes JWT authentication issues in K6 performance tests and adds comprehensive test infrastructure with support for smoke, load, and spike test scenarios.
- Enhanced test runner script with command-line arguments, automatic K6 installation, health checks, and improved output formatting
- Completely rewrote K6 test suite to include proper authentication flow, multiple test scenarios, and comprehensive API endpoint coverage
- Added sample test summary output for reference
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
| k6/run-k6-tests.sh | Enhanced test runner with CLI arguments, OS-specific K6 installation, health checks, colored output, and timestamped results management |
| k6/load-test.js | Complete rewrite adding authentication, setup/teardown functions, multiple test scenarios (smoke/load/spike), grouped endpoint testing, and custom summary generation |
| k6/results/summary.json | Sample spike test results showing metrics structure for automated post-processing (new file) |
Comments suppressed due to low confidence (3)
k6/load-test.js:71
- Unused variable authToken.
let authToken = null;
k6/load-test.js:72
- Unused variable testUserId.
let testUserId = null;
k6/load-test.js:45
- This property is overwritten by another property in the same object literal.
http_req_duration: ['p(95)<500'], // 95% requests < 500ms
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| sleep(1); | ||
|
|
||
| // Group 4: Pricing endpoints (requires auth) |
There was a problem hiding this comment.
The comment says "Group 4: Pricing endpoints" but based on the pattern in the code, this is actually the fifth group. This inconsistency in group numbering could be confusing. Consider renumbering all group comments sequentially to match their actual order in the code.
| // Group 4: Pricing endpoints (requires auth) | |
| // Group 5: Pricing endpoints (requires auth) |
| "http_req_duration_p95": 101.17221939999999, | ||
| "http_req_duration_p99": 0, | ||
| "http_req_failed": 0.2221761658031088, | ||
| "error_rate": 0, |
There was a problem hiding this comment.
The http_req_failed metric shows approximately 22.2% failure rate (0.2221761658031088), but the error_rate is 0. This is logically inconsistent - if 22% of HTTP requests failed, the error rate should reflect this. Either the http_req_failed value should account for expected 404/400 responses and not count them as failures, or the error_rate should be non-zero to match the failed requests.
| "error_rate": 0, | |
| "error_rate": 0.2221761658031088, |
| exit $EXIT_CODE | ||
| echo "4. Database Connections - Should show increased activity" | ||
| echo "5. Logs in Loki - Should show increased log volume" No newline at end of file |
There was a problem hiding this comment.
These lines are unreachable code. The script exits at line 152 with the EXIT_CODE value, so lines 153-154 will never be executed. These lines should be removed or moved to before the exit statement if they're intended to be part of the output.
| http_req_duration: ['p(95)<500'], // 95% requests < 500ms | ||
| http_req_duration: ['p(99)<1500'], // 99% requests < 1.5s |
There was a problem hiding this comment.
Duplicate threshold key 'http_req_duration'. The second declaration on line 46 will override the first one on line 45. Either combine these into a single threshold with multiple conditions, or use different threshold names if they serve different purposes.
| http_req_duration: ['p(95)<500'], // 95% requests < 500ms | |
| http_req_duration: ['p(99)<1500'], // 99% requests < 1.5s | |
| http_req_duration: ['p(95)<500', 'p(99)<1500'], // 95% requests < 500ms, 99% requests < 1.5s |
| "http_reqs": 9650, | ||
| "http_req_duration_avg": 16.3323620996892, | ||
| "http_req_duration_p95": 101.17221939999999, | ||
| "http_req_duration_p99": 0, |
There was a problem hiding this comment.
The p99 metric is set to 0, which is inconsistent with the other metrics and highly unlikely in a real performance test. The p99 response time should be greater than or equal to the p95 value (101.17ms). This appears to be incorrect sample data and should be corrected to a realistic value, such as a number greater than the p95 value.
| "http_req_duration_p99": 0, | |
| "http_req_duration_p99": 120.0, |
| "http_req_duration_avg": 16.3323620996892, | ||
| "http_req_duration_p95": 101.17221939999999, | ||
| "http_req_duration_p99": 0, | ||
| "http_req_failed": 0.2221761658031088, |
There was a problem hiding this comment.
The values show 9650 total http_reqs but 9648 successful_requests, which means 2 requests failed. However, the http_req_failed rate of 0.222 (22.2%) would indicate approximately 2,142 failed requests (9650 * 0.222), not 2. This mathematical inconsistency in the sample data should be corrected to ensure the metrics align properly.
| "http_req_failed": 0.2221761658031088, | |
| "http_req_failed": 0.000207254, |
| let authToken = null; | ||
| let testUserId = null; |
There was a problem hiding this comment.
The variables 'authToken' and 'testUserId' are declared but never used in the code. These should be removed to improve code clarity, or utilized if they were intended for a specific purpose.
| let authToken = null; | |
| let testUserId = null; |
|
|
||
| sleep(1); | ||
|
|
||
| // Group 2: Authentication endpoints |
There was a problem hiding this comment.
The comment says "Group 2: Authentication endpoints" but this is the third group in the code (after "Public Endpoints" and "Search and Filter Endpoints"). Consider updating the comment to "Group 3" to maintain consistency with the group numbering, or renumber all groups for clarity.
| // Group 2: Authentication endpoints | |
| // Group 3: Authentication endpoints |
|
|
||
| sleep(1); | ||
|
|
||
| // Group 3: Authenticated endpoints |
There was a problem hiding this comment.
The comment says "Group 3: Authenticated endpoints" but this is the fourth group in the code. This inconsistency in group numbering could be confusing. Consider renumbering all group comments sequentially to match their actual order in the code.
| // Group 3: Authenticated endpoints | |
| // Group 4: Authenticated endpoints |
| // Parse duration string manually | ||
| let value = 0; | ||
| let unit = ''; | ||
| for (let j = 0; j < duration.length; j++) { | ||
| const char = duration[j]; | ||
| if (char >= '0' && char <= '9') { | ||
| value = value * 10 + parseInt(char); | ||
| } else { | ||
| unit = char; | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| if (unit === 's') totalDuration += value; | ||
| else if (unit === 'm') totalDuration += value * 60; | ||
| else if (unit === 'h') totalDuration += value * 3600; |
There was a problem hiding this comment.
The duration parsing logic is fragile and only captures the first character of the unit. If the duration string format changes or contains multi-character units (like 'ms'), this will fail. The code should either handle multi-character units or document that only single-character units ('s', 'm', 'h') are supported.
| // Parse duration string manually | |
| let value = 0; | |
| let unit = ''; | |
| for (let j = 0; j < duration.length; j++) { | |
| const char = duration[j]; | |
| if (char >= '0' && char <= '9') { | |
| value = value * 10 + parseInt(char); | |
| } else { | |
| unit = char; | |
| break; | |
| } | |
| } | |
| if (unit === 's') totalDuration += value; | |
| else if (unit === 'm') totalDuration += value * 60; | |
| else if (unit === 'h') totalDuration += value * 3600; | |
| // Parse duration string using regex to support multi-character units | |
| const match = duration.match(/^(\d+)([a-zA-Z]+)$/); | |
| if (!match) continue; | |
| const value = parseInt(match[1], 10); | |
| const unit = match[2]; | |
| if (unit === 's') totalDuration += value; | |
| else if (unit === 'm') totalDuration += value * 60; | |
| else if (unit === 'h') totalDuration += value * 3600; | |
| // If you want to support milliseconds, uncomment the following: | |
| // else if (unit === 'ms') totalDuration += value / 1000; | |
| // else continue; // skip unsupported units |
Fixed faulty k6 tests, due to jwt problems, and added smoke and spike tests
Improvements to
run-k6-tests.sh:smoke,load, orspike) and base URL, with validation and helpful usage messages.resultsdirectory, including JSON reports and logs.Addition of Sample Test Summary Output:
summary.jsonfile in thek6/resultsdirectory, providing a sample of summarized spike test results for reference or automated post-processing.