EDBO+ command line interface + quality of life improvements - #15
Open
seanli9604 wants to merge 2 commits into
Open
EDBO+ command line interface + quality of life improvements#15seanli9604 wants to merge 2 commits into
seanli9604 wants to merge 2 commits into
Conversation
…ouple reaction scope from input data files (now, it's only for training)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I noticed that with EDBO+, it can be quite an inefficient process viewing and editing one large CSV file that contains all experiments (already done and pending). The need to work with an additional "priority" column also needlessly complicates the back-end code, and it is difficult to adjust the scope halfway through experimentation, so that the optimiser may find experiments with better acquisition (e.g. expected improvement) values.
For instance, if in the initial scope the "concentration" variable take up the possible values [0.05, 0.1, 0.5], after a number of experiments, it could be the case that a concentration of 0.2 actually results in a significantly higher acquisition function value (for some combination of the other variables), and would be picked by the sampler. But, this possibility will not be sampled by the current version EDBO+, unless the CSV holding all possible combinations of reaction conditions is manually or programmatically modified in an ad-hoc way.
So, I changed up the interface. Instead of requiring a large CSV containing all experiments, now the input file is a smaller CSV only containing experiments that are already done and a configuration file (config.json) containing the reaction scope (a dictionary mapping reaction variables to possible values), the objectives (a dictionary mapping each objective to a mode, i.e. max or min), and additional parameters required by EDBOplus().run().
Then, rather than holding all possible conditions in a CSV file, the full combination of reaction conditions are generated on the fly, with already trialed conditions removed. Now, only the batch of experiments proposed by EDBO+ is written to the input CSV (with user prompting). Once the experiments are done in the lab and objectives determined, the (empty) objective fields can be filled in, and EDBO+ run again to propose more experiments. As before, a separate CSV with prediction results for the entire reaction scope is also created.
The other change is the creation of a command line interface via edboplus.py, which takes the filename for the experimental CSV and the parameters in config.json and calls EDBOplus().run() with these parameters. This allows for much simpler and more dynamic usage of EDBO+, since the user can edit config.json and try different EDBO+ parameters to investigate how the prediction results differ.
Finally, I fixed the bug in calculating standard deviation/sigma (for the prediction CSV) in the expected_improvement() method in optimizer_botorch.py.
For more information, I have updated the README to include additional details about usage.