Introduce Differential Evolution for optimising numeric constants in GP individuals

[morinim]

Introduce Differential Evolution (DE) as a generation-end refinement step for optimising numeric constants in GP individuals, instead of applying DE during offspring creation.

This yields a clean separation between:

GP → structural search
DE → numerical optimisation

DE acts as a population-level local search phase applied between generations.

Refinement policy

• Apply DE only to a subset of the population:

  • top 10-20% (elite) individuals;
  • optionally + small random subset (diversity).

• Optional scheduling:

  • every generation (cheap DE only);
  • every N generations (e.g. 2–5).

• Skip:

  • individuals without tunable parameters;
  • recently refined individuals;
  • oversized individuals (too many parameters).

Key design point

Lamarckian update: write optimised constants back into the genotype (offspring inherit improved parameter).

[morinim]

If the same numeric value appears multiple times in the SLP, the approach of dac9161 treats each occurrence as an independent parameter.

So:

x * 2.0 + 2.0

would normally produce two DE dimensions, not one shared constant.

This means the system is optimising numeric occurrences, not symbolic parameters.

[morinim]

Since the current SLP representation has no explicit tunable/non-tunable parameter distinction, the initial DE integration treats all active floating-point argument occurrences as optimisable numeric slots. The decision vector is extracted dynamically from the active SLP and written back after DE refinement. This keeps the prototype simple and avoids immediate representation changes, at the cost of optimising numeric occurrences rather than shared symbolic parameters.

[morinim]

After each generation, refine the current best individual and a small number of additional individuals sampled uniformly at random without repetition from the rest of the population. Each selected individual is replaced in place by its numerically optimised version (Lamarckian interpretation).

Probably we should skip individuals that have no optimisable numeric slots (and/or exceed a maximum allowed decision-vector size?).