Early fail a run as soon as a test fails

[GregoryAlbouy]

Description

The current run workflow is:

1. Record all requests
2. Compute all stats
3. Run test suites

This issue suggests a new workflow:

1. Record all requests, and run the relevant tests* and stop if any fail
2. If not failed, compute all stats
3. If not failed, run remaining test suites

*: here a relevant test is a test against an absolute value, such as minimum response time. Relative values (such as average response time or status code distribution) do not make sense for an incomplete run.

Tasks

Suggestions

Notes

• Given the high complexity of the task and the low added value, we should consider a very low priority for that task.