What happened:
selected_and_all_and_picture assumes every test result has a Matrix metric.
For normal benchmarks with scalar metrics only, _draw_pictures() does this:
python
matrix = test_result.get("Matrix")
for key in matrix.keys():
So matrix is None and rank saving crashes after the benchmark result is already available.
What you expected to happen:
Rank CSV files should still be saved.
If picture mode needs Matrix, Ianvs should either skip picture generation for results without Matrix, or fail with a clear error.
How to reproduce it:
Use save_mode: selected_and_all_and_picture with a result that only has scalar metrics, for example:
{"accuracy": 0.9}
Actual error:
AttributeError: 'NoneType' object has no attribute 'keys'
Anything else we need to know?:
This is in core/storymanager/rank/rank.py, inside _draw_pictures().
What happened:
selected_and_all_and_pictureassumes every test result has aMatrixmetric.For normal benchmarks with scalar metrics only,
_draw_pictures()does this:So matrix is None and rank saving crashes after the benchmark result is already available.
What you expected to happen:
Rank CSV files should still be saved.
If picture mode needs Matrix, Ianvs should either skip picture generation for results without Matrix, or fail with a clear error.
How to reproduce it:
Use save_mode: selected_and_all_and_picture with a result that only has scalar metrics, for example:
{"accuracy": 0.9}Actual error:
AttributeError: 'NoneType' object has no attribute 'keys'
Anything else we need to know?:
This is in core/storymanager/rank/rank.py, inside _draw_pictures().