Probe generialized implementation

[nicolaefilat]

We ask for code review to check if the implementation is up to standards.

It requires HerbGrammar to be on branch fix-normalizing to pass tests.

[nicolaefilat]

I did a bit of a hack to add the HerbGrammar fix-normalizing branch to the CI as a dependency to make the CI pass to prove that the code works :)

[ReubenJ]

I did a bit of a hack to add the HerbGrammar fix-normalizing branch to the CI as a dependency to make the CI pass to prove that the code works :)

Please switch this back, @nicolaefilat. We don't want to mess with the CI to make it pass. We'll get the HerbGrammar branch merged and released before merging this.

[nicolaefilat]

@ReubenJ I reverted the CI hack :)

[ReubenJ]

Hackathon Plan

Steps

• Run a budgeted search
  • Gets an iterator with some limit (the low-level budget)
  • This is a normal iterator
• Collect (partial) successes
  • Function to define success, which is just % of examples solved by default
  • Function takes any kind of specification
  • Boolean output: should the program be saved or not?
• Modify the grammar
  • Function to modify the grammar based on the successes
  • Takes the successes (saved programs)
  • Modify the grammar: in place or return a new one, TBD
• Rinse and repeat
  • Function defining high-level budget
  • For now, just a parameter defining the number of overall runs of budgeted search

Optimization

• When evaluating a program during search, store the output of a rulenode on the specification in the rulenode's _val field
• Keep queue across grammar updates to avoid re-enumerating the search space that was already explored earlier

[THinnerichs]

Closing this in favor of MLFIterator and Herb-AI/Garden.jl#5.