The SWAT dev vs main results table does not seem to work correctly.
It shows these two runs
SV-COMP27_no-runtime-exception.Java.no-runtime-exception.Main
SV-COMP27_valid-assert.Java.valid-assert.Main
So it compares the different test categories (valid-assert and no-runtime-exception). But it should compare the different branches (main and dev) for each category. Something like this
SV-COMP27_no-runtime-exception_main
SV-COMP27_no-runtime-exception_dev
SV-COMP27_valid-assert_main
SV-COMP27_valid-assert_dev
It would also be great if we can compare runs from the same branch.
For example, the results page currently shows these results for dev:
2026-07-01 · 54dd68d — score 1357, correct 932/1782
2026-06-29 · 0e6cad5 — score 0, correct 0/1782
2026-06-25 · cdf7c4a — score 1353, correct 930/1782
Run 54dd68d passes two more cases than run cdf7c4a. But which ones? Can we add a feature to compare them directly?
The SWAT dev vs main results table does not seem to work correctly.
It shows these two runs
So it compares the different test categories (valid-assert and no-runtime-exception). But it should compare the different branches (main and dev) for each category. Something like this
It would also be great if we can compare runs from the same branch.
For example, the results page currently shows these results for dev:
Run 54dd68d passes two more cases than run cdf7c4a. But which ones? Can we add a feature to compare them directly?