Exogenous Parameters

[sararuggeri]

Hi,

When building a custom surrogate model for a BotorchRecommender, some variables are important predictors for the target but should remain fixed during optimization since they are not part of the Campaign's searchable parameter space. There is currently no clean way to express this distinction.

I am currently approximating fixed parameters using NumericalContinuousParameter with a narrow bounds range (e.g. [x - 0.00001, x + 0.00001]), and can't find a more appropriate solution.

[Scienfitz]

Hey @sararuggeri

so you're using a custom surrogate? Why are you making these predictor-style numbers part of the searchspace? They can be added to the model input inside your custom surrogate class and wouldn't really need to be part of the searchspace at all.

Is it because your "measurements" still contain these numbers, ie they can differ from data-point to datapoint? Then maybe your request goes into the direction of what we call exogenous factors. Those are essentially numbers that enter the modeling process + are measured in the sense that they are decided when the targets are measured - however they are not controlled or "set" by the user/experimenter, they are merely "recorderd" and still need to enter the model.

Examples for this are for instance i) the temperature outside the lab or ii) the risk free return rate in financial modeling. Does this roughly sound like what you need?

We already thought about such exogeneous parameters. We would just implement a flag exogeneous to all parameters which would enable the retention of all interfaces + implement that they are passed to the surrogate but are not part of the searchspace. It would require that the values for such factors are declared by the user when using e.g. recommend.

[AdrianSosic]

Hi @Scienfitz and @sararuggeri. Agree, I think exogeneous parameters (or whatever we decide to call them) is exactly what is needed for this and they have been on the TODO list since ages. However, @Scienfitz: I feel that adding an exogenous attribute to all parameters is not the right approach, but we can certainly discuss. My reasoning: for an exogenous parameter, there is no need to put any restriction whatsoever, since you can't control it anyway and just have to accept the values it takes. So this means two things in particular:

• There is no need to define any ranges, values, etc (so all the stuff that our regular parameters expect)
• But even the distinction between discrete and continuous is sort of meaningless.

If you ask me, without having wrapped my head around all the details, I feel that exogenous parameter should probably become an isolated class that sits next to ContinuousParameter and DiscreteParameter. But let's find out together and finally implement it 🙃

[Scienfitz]

@AdrianSosic I think there are a few more possible complexities. Why do you assume we can't control the parameter automatically means we must accept any value?

An exogenous parameter could also be a category with a limited number of possible values. Example: the quarter(of the year). You have the need to define the possible categories to represent it (integer encoding here would impose spurious ordering). Also, why shouldn't a categorical exogenous parameter be custom or cheminformatics-encoded?

But I agree that there is no difference between a discrete or continuous numerical exogenous parameter. So at the very least we would need a categorical and numerical exogenous parameter.

Now one reason why I see benefit in "adding" the exogenous flavor to our existing classes: you're able to reuse all the encoding machinery we already have. Probably we would exclude task and fidelity parameters from being able to be exogenous.

So the steps to achieve everything needed are quite simple:

• Give some parameters the exogenous flavor OR create new exogenous parameters
• Exclude them from search space creation
• For predictive recommenders, require exogenous values to be set when calling .recommend
• Pass their values through to the surrogates

[sararuggeri]

Hi @Scienfitz and @AdrianSosic, , in my scenario, I have to assume certain features are fixed.
My goal is to keep these specific variables constant to simulate a static environment, and I have to assume those values are already given. While I don't want the model to include these uncontrollable features in its final recommendations, I believe dropping them completely would damage the prediction since they strongly influence the target variable.

A big inspiration could be the optimize_acqf method from BoTorch. The nice thing is it allows to set fixed_features, a map for features which should be fixed to a particular value during generation.