Analysis tools for the example benchmarking for CyRxnOpt.
cyrxnopt_analysis reads the results files from benchmarking CyRxnOpt with cyrxnopt_server, and computes summary metrics
for them, such as clearance rate, solve time, and average error relative to the
known optimum for each benchmark problem. Results are read from a directory of
run output, aggregated per benchmark function, and written out as CSV summaries
for further analysis.
Clone the repository and install it with pip:
git clone https://github.com/RxnRover/cyrxnopt_analysis.git
cd cyrxnopt_analysis
pip install .A set of results is analyzed with the analyze_results command-line tool,
installed as part of this package. It takes the optimizer whose results are
being analyzed and the directory containing the results files, and writes CSV
summaries to an output directory (analysis/{YYYY-MM-DD}_{optimizer} by
default).
At minimum, it needs the optimizer that produced the results and the directory
containing the results files to analyze. Run analyze_results --help for the
full set of options, including the clearance threshold, the file pattern used to
find results files, and where output is written. By default, CSV summaries are
written to analysis/{YYYY-MM-DD}_{optimizer}.
For example, continuing from the cyrxnopt_server benchmarking example (see
its README),
which runs the random optimizer and writes results to
data/random_optimizer, the results can be analyzed with:
analyze_results random ../cyrxnopt_server/data/random_optimizer/ -fp "results\.csv" -t 0.02This project uses pre-commit, please make sure to install it before making any changes:
# After cloning the repository
pip install pre-commit
cd cyrxnopt_analyzer
pre-commit install