Constraint for batch proposal

[StefanPSchmid]

Dear baybe team,

I am coming to you with a question and/or improvement suggestion.

In my current project, I have the required constraint that for every experiment in my proposed batch, all values should be the same for one category. As an example, I have two possible temperatures (50°C and 100°C), but across my batch, I can only select 50°C or 100°C for all experiments. This paper also discusses it in Section 2.5: https://chemrxiv.org/engage/api-gateway/chemrxiv/assets/orp/resource/item/66f976b5cec5d6c142981e4e/original/highly-parallel-optimisation-of-nickel-catalysed-suzuki-reactions-through-automation-and-machine-intelligence.pdf

How I solved this problem within baybe is that for every possible value (temperature), I set a DiscreteExcludeConstraint so that the campaign is only able to recommend that temperature, let the campaign propose a batch of experiments and then determine the joint acqf value across this batch. Finally, I take this temperature with the best joint acqf value. Pseudo-code wise it roughly looks like this:

batch_size = 96
joint_acqf_value = -np.inf
proposed_experiments = None

for temperature in all_temperatures:
    temp_excluded = [x in x if x!= temperature]
    constraint = DiscreteExcludeConstraint(
            parameters=["temperature"],
            conditions=[
                SubSelectionCondition(selection=temp_excluded)
            ]
        )
    campaign.toggle_discrete_candidates(constraints=[constraint], exclude = True, complement = True)
    experiments = campaign.recommend(batch_size = batch_size)
    if campaign_tmp.joint_acquisition_value(experiments) > joint_acqf_value:
            joint_acqf_value = campaign_tmp.joint_acquisition_value(experiments)
            proposed_experiments = experiments

As far as I can tell, this corresponds roughly to what the above paper describes. Now to my question / improvement suggestion:

Do you know of a more elegant way to do this within baybe? If not, do you think it is a constraint worth implementing. If you give me a starter where this could fit, I would be interested in helping you out with this, as I think it might be suitable for hopefully more people (cannot work on it right now though, I have to admit).

Thanks for your answer,
Stefan

[AdrianSosic]

Hi @StefanPSchmid, what you refer to is often called an inter-point constraint, and yes, while your approach also solves the problem in a manual way, it absolutely makes sense to offer some convenience interface for this 👍🏼

In fact, there is already an open pull request, which however has been idling for quite a while. At the moment, I'm not perfectly up to date in terms of intended implementation for discrete spaces, though, since @AVHopp was the one looking into it. Perhaps he can:

• Give some insights on an expected timeline to finish/merge the PR
• Share his opinion on the discrete case?

[AVHopp]

Hello @StefanPSchmid let me check this for you in a bit more detail and come back to you hopefully today, but the latest during this week :)

[AVHopp]

Hello @StefanPSchmid , let me try to give some more details.

TL;DR: Will investigate this topic soon in more detail and keep you updated :) Hard to estimate though how complex it will be.

• The PR that @AdrianSosic mentioned is something that I want to get back to ASAP. I was unfortunately not able to focus on this in the past months, but this is my next work item. The reasons why this was not worked on were (i) it was for a long time blocked by another PR that touched relevant parts of the code that need to be changed for this (ii) me being busy with other projects and (iii) other topics being of higher priority. However, points (i) and (iii) are no longer valid and (ii) will become better soon :)
• The PR was meant as a first "easy" way to enable inter-point constraints by simply using what is already available in botorch. In fact, the optimize_acqf function of botorch which we also use supports inter-point constraints, which is what we wanted to enable. This is however only applicable to continuous spaces.
• Back when I worked on that actively, I also investigated the discrete setting. However, the optimize_acqf_discrete function does not seem to support inter-point constraints (at least they are not mentioned), which is the reason why I did not investigate this case further. But it seems that optimize_acqf_mixed offers inter-point constraints support, so maybe it is actually possible to do something here.

I will also have a look at your solution/the paper you mentioned but wanted to get back to you ASAP. In my opinion, inter-point constraints are definitely something that BayBE should offer, and I'll investigate this topic soon. I will keep you and this thread updated.

Also, just to be clear: Your implementation works for your use case, right?

[StefanPSchmid]

Hi @AVHopp

Thanks for the details, interesting to see that this is easier to implement for continuous than discrete. Also nice to hear that you are planning on picking it up - if I can help, please lmk, however I will most likely only have time for it when July is done.

My (pseudo-)code works and gives the right results.

[AVHopp]

Great, thanks for the update :) I might come back to your offer in August

[Scienfitz]

Hey @StefanPSchmid

thanks again for using baybe and raising these interesting feature requests. I think what you request is rather crucial for many real world batch experimenters and we should definitely support it in a convenient manner.

Relation to existing interpoint PR
I dont't think #345 would work for your problem, even if implemented and merged. The reason: As others pointed out it uses existing botorch machinery to enable a linear constraint (aka a weighted sum of things) that goes over the batch (for that reason we could also say intra-batch constraint). So what you can say with that constraint, is that the sum of all your temperatures should correspond to X.

Now the closest I can bring your requirement to that phrasing is The sum of all temperatures in the batch must be 96*temp1 OR 96*temp2. Besides not sure how to implement the OR this would also fail if the temperature numbers were such that there are other combinations (other than having 96 times purely temp1) of mixed temperatures allows for equalling 96*temp1 etc. So I'm not sure that PR will help with your request at all.

Phrasing your constraint
What your constraint actually says is something more general: Only allow N unique values for the temperature in the entire batch (with N probably being 1 for the most common cases).

This is also generalizable to non-numerical parameters, eg if you have a switch, and it can only be one of many settings for the entire batch.

Approach 1: Exclusion and loop (discrete space)
In a discrete space I can see how its fairly easy to deal with this. You already wrote the pseudo code for it:

• Iterate over the possible unique choices
• restrict the searchspace accoringly
• get a recommendation for each restricted space
• take the recommendations with the best batch acqf

Such a loop over sub-spaces is already used as part of the discrete cardinality constraint. There it was a bit more complicated, but in your case here I think the subspace generation is a bit easier.

Approach 2: Nonlinear constraint (continous space)
If I recall correctly there is no way of expressing the above constraint as a linear continuous interpoint constraint. The solution might be to not use a linear constraint but to rather phrase it as a non-linear constraint.

These work with indicator functions that tell which space is valid (indicator is 1) and non valid (indicator is 0). These functions also have a smooth rather than a harsh change of values, to allow for gradient movement required in the continuous optimization.

For this request, one would have to implement an indicator that is 1 if all values in the batch are a unique temp and 0 when they're not, with a smoothing. This usually can be achieved with each physicists favorite tools: gaussians (for equal conditions) and sigmoids (for inequality conditions). You just have to find a suitable inner condition that can be computed and corresponds to whether there is just one unique value. With this approach nearly any constraint can be enabled.

I have worked with tools that use exactly this for much more complex problems that we usually look at, and it works. But the tricky thing is that this smoothing needs to be set up correctly, otherwise it will break computationally and the optimizer wont find or explore well.

Potential Next Steps
I would strongly favor implementing approach 1 for now. The continuous and interpoint constraints are still in a rather strong baustelle and I wouldn't touch approach 2 until that's changed.

Besides that, the road to approach 1 is already very clear conceptually. Wec ould have something like a DiscreteNUniqueValuesConstraint (name to be found and improved) that triggers the loop you've pseudocoded above.

PS:
If I recall correctly you were in contact with Gerit some time ago. This could be a fantastic work item for an internship or industry experience placement! Feel free to reach out via mail if you're still interested in that.

[StefanPSchmid]

Hi @Scienfitz,

thanks for your answer. I agree that there should be a case by case basis, depending if it is a discrete or continuous case. If it is discrete, when my pseudo-code should be called, if it is continuous it sounds like the botorch optimiser can deal with this (at least from my understanding of the discussion with @AVHopp).

As far as I understood the discussion around PR #345 we had in this issue, the idea was to extend this PR to handle both continuous and discrete cases (with the appropriate case differentiation described above), then we don't have to go the route of your Approach 2, which sounds rather tricky. I am happy to try to help a bit once the NeurIPS rebuttal is over beginning of August :)

PS:
Thanks for the offer! I am/was indeed in contact with Gerit recently and I hopefully will be able to join his group in spring 2026, although I don't think the plan is to join the baybe team, from what I got in our discussions.

[Scienfitz]

Hmm I would rather not expand #345 any further, as its old and rather large (it should focus on its initial intended functionality). As outlined, I also think it actually has no concrete potential to solve the constraint requested in this issue here (unless someone can tell me how to phrase the uniqueness constraint you have here as a linear constraint with the means available to the botorch interface, I outlined some problems in my post above).

Besides, we currently have a always a split between discrete and continuous constraints (look eg at Cardinality). So you could simply implement the discrete one completely independent of anything else going on, that seems to me most frictionless and straightforward. We have already talked about grouping discrete and continuous constraint variants under simple dispatchers, but I wouldnt want to start this randomly with a newly implemented constraint and it needs to be done in other places.

PS: Cool, I didn't know that yet. Spring 2026 is still quite a bit ahead but we certainly look forward to welcoming you! We are usually flexible with what you want to work on, so if baybe also interests you then we can surely arrange (if not thats also OK).

[StefanPSchmid]

Okay thanks. That sounds like a good plan, if the constraints are anyways separated between Discrete and Continuous. I will then get in contact with @AVHopp and try to work on the Discrete constraint.