Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,63 @@

## Recent Improvements (continued)

### NL-SHADE-LBC adaptive DE (CEC 2022 winner) — 2026-05-28
- [x] **New `NLSHADE_LBC` heuristic** in
`panobbgo/heuristics/nl_shade_lbc.py`; closes the *NL-SHADE-LBC*
DE-family follow-up in `planning/SELF_IMPROVEMENT_LOOP.md`.
Direct subclass of
:class:`~panobbgo.heuristics.nl_shade_rsp.NLSHADE_RSP` (Stanovov,
Akhmedova & Semenkin, CEC 2022 winner) adding **Linear Bias
Change** in the success-history memory update: the F / CR
Lehmer-mean order ``p`` is linearly scheduled across budget
progress instead of fixed at 2. Literature defaults
``p_F: 3.5 → 1.5``, ``p_CR: 1.0 → 1.5``, spread ``m_lbc = 1.5``;
at ``p = 2, m = 1`` the formula recovers the standard L-SHADE
Lehmer mean.
- **Why it matters.** NL-SHADE-LBC won the CEC-2022 single-objective
bound-constrained competition and is the direct NL-SHADE-RSP
descendant — the literature frontier as of the most recent CEC
competition we can mirror. Adds a fifth DE-family arm the bandit
can pick whichever wins on the current battery.
- [x] **Catalog wiring** — `default_structural_catalog` gains
`(NLSHADE_LBC, {"NP_init": 30, "k_rank": 3.0})` as a fifteenth
`add_heuristic` candidate; `default_catalog` gains six
LBC-specific rules (`NLSHADE_LBC.NP_init` integer_add,
`NLSHADE_LBC.p_F_init` / `p_F_final` / `p_CR_init` / `p_CR_final`
float_uniform, `NLSHADE_LBC.m_lbc` float_uniform).
- [x] **Deviations from the full CEC-2022 paper** (documented for
honesty): the adaptive binomial / exponential crossover blend
and the repetitive-generation bound-constraint handling are
*not* ported — the same async-pipeline limitations that motivated
omitting them from NL-SHADE-RSP apply here. Both queued as
follow-ups.
- [x] **CR-zero handling** — preserves the L-SHADE terminal sentinel
rule and filters strict zeros out of the LBC sum (because
``s^(p−m)`` with ``p < m`` is undefined at ``s = 0``).
- [x] **Backwards compatibility** — strictly safe. NLSHADE_LBC is
opt-in: not added to any default battery, so existing composite
baselines stay byte-identical. NL-SHADE-RSP / jSO / L-SHADE
base classes are untouched — only the LBC subclass overrides
`_update_memory`; verified by a regression test that
``NLSHADE_RSP._update_memory`` still produces the standard
L-SHADE Lehmer mean output.
- [x] **Tests** — `tests/test_heuristic_nl_shade_lbc.py` (30 tests):
construction validation, LBC schedule (endpoints, linear
midpoint, clipping, budget-unknown fallback), memory update
(anchor-bin skip, pointer mod (H-1), no-op on empty buffer,
[0,1] clamping, formula recovers Σ(w·F^3.5)/Σ(w·F^2.0) at
progress=0, p=2/m=1 recovers L-SHADE for both F and CR, CR=0
terminal sentinel, terminal-bin stays terminal, mixed-zero CR
filtered, uniform weights on zero-delta), pipeline (on_start,
smoke convergence, restart resets), inheritance safety
(NLSHADE_RSP unchanged), and registration.
- [x] **Documentation updated** —
`planning/SELF_IMPROVEMENT_LOOP.md` (new §13 entry, follow-up
idea seeded), `doc/source/guide.rst`,
`doc/source/guide_benchmarking.rst`,
`doc/source/guide_architecture.rst`,
`doc/source/heuristics.rst`, and this TODO entry.

### NL-SHADE-RSP adaptive DE (CEC 2021 winner) — 2026-05-25
- [x] **New `NLSHADE_RSP` heuristic** in
`panobbgo/heuristics/nl_shade_rsp.py`; closes the *NL-SHADE-RSP /
Expand Down
2 changes: 1 addition & 1 deletion doc/source/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Quick Navigation
* - **Interested in research?**
- Explore :doc:`guide_research` for related work, theoretical properties, and future directions
* - **Want to measure progress?**
- Read :doc:`guide_benchmarking` for the composite score, external baselines, parametrically randomised problems, the statistical acceptance rule (with paired and unpaired bootstrap sampling — paired is auto-selected for the rep-aligned randomized harness and shrinks the CI 3-10× over the historical independent-resample scheme), the autonomous self-improvement loop driver, the anti-cherry-pick guard, the hold-out validation set (single- and multi-seed, with bootstrap-CI aggregation across hold-out seeds), the adaptive Thompson-sampling mutation sampler with optional per-class structural bandit arms, the structural ``add_heuristic`` / ``drop_heuristic`` portfolio mutations, the categorical ``MutationRule`` kind (for discrete knobs like ``PSO.topology``, ``Sobol.scramble``, ``LSHADE.archive_factor``, ``LSHADE.F_schedule``, and ``NLSHADE_RSP.adaptive_archive``), the tri-topology PSO (``gbest`` / ``lbest`` / ``vonneumann`` — fully-connected, ring, and 4-connected 2-D toroidal grid per Kennedy & Mendes 2003 / Mendes 2004) candidate pool, the L-SHADE adaptive Differential Evolution heuristic (Tanabe-Fukunaga 2014) with two opt-in jSO refinements (the linearly-decreasing ``p_best`` schedule from iLSHADE / jSO Brest et al. 2016 / 2017 and the three-phase asymmetric F-cap from jSO Brest et al. 2017), the literature-faithful jSO heuristic itself (Brest, Maučec & Bošković 2017 — CEC-2017 winner — inheriting the L-SHADE F-cap machinery by construction), the NL-SHADE-RSP heuristic (Stanovov, Akhmedova & Semenkin 2021 — CEC-2021 winner — a jSO subclass adding non-linear population reduction, rank-based selective pressure, and a randomised adaptive archive), and the COBYQA derivative-free trust-region local optimizer (Ragonneau-Zhang 2023)
- Read :doc:`guide_benchmarking` for the composite score, external baselines, parametrically randomised problems, the statistical acceptance rule (with paired and unpaired bootstrap sampling — paired is auto-selected for the rep-aligned randomized harness and shrinks the CI 3-10× over the historical independent-resample scheme), the autonomous self-improvement loop driver, the anti-cherry-pick guard, the hold-out validation set (single- and multi-seed, with bootstrap-CI aggregation across hold-out seeds), the adaptive Thompson-sampling mutation sampler with optional per-class structural bandit arms, the structural ``add_heuristic`` / ``drop_heuristic`` portfolio mutations, the categorical ``MutationRule`` kind (for discrete knobs like ``PSO.topology``, ``Sobol.scramble``, ``LSHADE.archive_factor``, ``LSHADE.F_schedule``, and ``NLSHADE_RSP.adaptive_archive``), the tri-topology PSO (``gbest`` / ``lbest`` / ``vonneumann`` — fully-connected, ring, and 4-connected 2-D toroidal grid per Kennedy & Mendes 2003 / Mendes 2004) candidate pool, the L-SHADE adaptive Differential Evolution heuristic (Tanabe-Fukunaga 2014) with two opt-in jSO refinements (the linearly-decreasing ``p_best`` schedule from iLSHADE / jSO Brest et al. 2016 / 2017 and the three-phase asymmetric F-cap from jSO Brest et al. 2017), the literature-faithful jSO heuristic itself (Brest, Maučec & Bošković 2017 — CEC-2017 winner — inheriting the L-SHADE F-cap machinery by construction), the NL-SHADE-RSP heuristic (Stanovov, Akhmedova & Semenkin 2021 — CEC-2021 winner — a jSO subclass adding non-linear population reduction, rank-based selective pressure, and a randomised adaptive archive), the NL-SHADE-LBC heuristic (Stanovov, Akhmedova & Semenkin 2022 — CEC-2022 winner — a NL-SHADE-RSP subclass adding **Linear Bias Change** in the success-history memory update: the F / CR Lehmer-mean exponents are linearly scheduled across budget progress instead of fixed), and the COBYQA derivative-free trust-region local optimizer (Ragonneau-Zhang 2023)

Guide Contents
--------------
Expand Down
17 changes: 15 additions & 2 deletions doc/source/guide_architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,21 @@ Implemented Heuristics
Inherits the jSO mutation / schedule / anchor-memory machinery and the asynchronous pipeline
unchanged. The full CEC-2021 adaptive-crossover blend and success-ratio archive-probability
adaptation are intentionally not ported (the asynchronous generation model does not expose them
cleanly). All four DE-family arms (DE / L-SHADE / jSO / NL-SHADE-RSP) ship in the structural
catalog so the bandit picks whichever wins on the current battery.
cleanly).

- :class:`~panobbgo.heuristics.nl_shade_lbc.NLSHADE_LBC`: NL-SHADE-LBC refinement of NL-SHADE-RSP
(Stanovov, Akhmedova & Semenkin, CEC 2022 winner). Direct subclass of
:class:`~panobbgo.heuristics.nl_shade_rsp.NLSHADE_RSP` that adds *Linear Bias Change* in the
success-history memory update: the F / CR Lehmer-mean order ``p`` is linearly scheduled across
budget progress (``p_F: 3.5 → 1.5``, ``p_CR: 1.0 → 1.5``) instead of fixed at ``2``; the spread
between numerator and denominator exponents is held constant at ``m_lbc = 1.5``. At
``p = 2, m = 1`` the formula recovers the standard L-SHADE weighted Lehmer mean — both regimes
are reachable from the default catalog so the bandit can flip between them. Inherits NL-SHADE-RSP's
NLPSR / RSP / adaptive-archive machinery and the asynchronous pipeline unchanged. The CEC-2022
adaptive crossover blend and repetitive-generation bound-constraint handling are intentionally not
ported (see the heuristic docstring). All five DE-family arms (DE / L-SHADE / jSO / NL-SHADE-RSP /
NL-SHADE-LBC) ship in the structural catalog so the bandit picks whichever wins on the current
battery.

- :class:`~panobbgo.heuristics.pso.PSO`: Asynchronous Particle Swarm Optimization with the canonical
Clerc–Kennedy (2002) constriction-coefficient parameters (``w = χ ≈ 0.7298``, ``c1 = c2 ≈ 1.49618``).
Expand Down
11 changes: 9 additions & 2 deletions doc/source/guide_benchmarking.rst
Original file line number Diff line number Diff line change
Expand Up @@ -709,7 +709,7 @@ space with two new ops that change the *shape* of a
* ``add_heuristic`` — append a heuristic from a curated pool
(``Random``, ``Nearby``, ``NelderMead``, ``Center``,
``LatinHypercube``, ``Sobol``, ``Extremal``, ``PSO``, ``LSHADE``,
``JSO``, ``NLSHADE_RSP``, ``COBYQA``) to a target strategy. The pool ships *three*
``JSO``, ``NLSHADE_RSP``, ``NLSHADE_LBC``, ``COBYQA``) to a target strategy. The pool ships *three*
PSO entries — the default fully-connected ``gbest`` topology
(Kennedy-Eberhart 1995), the ring ``lbest`` topology with
``k_neighbors=2`` (Kennedy & Mendes 2002), and the 4-connected
Expand All @@ -734,7 +734,14 @@ space with two new ops that change the *shape* of a
Non-Linear Population Size Reduction (``NP(r) = round((NP_min −
NP_init)·r^(1−r) + NP_init)``), Rank-based Selective Pressure on the
differential ``r1`` draw (``k_rank`` default ``3``), and a randomised
per-generation archive cap. All four DE-family arms share the
per-generation archive cap. NL-SHADE-LBC (Stanovov, Akhmedova &
Semenkin 2022 — CEC-2022 winner) refines NL-SHADE-RSP further with
**Linear Bias Change** in the success-history memory update: the F /
CR Lehmer-mean order ``p`` is linearly scheduled across budget
progress (``p_F: 3.5 → 1.5``, ``p_CR: 1.0 → 1.5``) instead of fixed
at ``2``; the numerator / denominator exponent spread ``m_lbc`` is
held constant at ``1.5`` (``m = 1`` and ``p = 2`` everywhere recovers
the standard L-SHADE Lehmer mean). All five DE-family arms share the
``add_heuristic`` arm so the bandit picks whichever variant wins on
the current battery. COBYQA
(Ragonneau-Zhang 2023) brings the modern Powell-family
Expand Down
3 changes: 2 additions & 1 deletion doc/source/heuristics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Point generation algorithms (heuristics):
- **LSHADE**: Linear-population-reduction Success-History Adaptive DE (Tanabe & Fukunaga, CEC 2014 winner). Adapts ``F`` and ``CR`` per-trial via per-bin Cauchy / Normal memories that update each generation by the weighted Lehmer mean of successful triples; uses the ``current-to-pbest/1`` mutation with an external archive of replaced parents. The population shrinks linearly from ``NP_init`` (default 30) down to ``NP_min`` (default 4) over the strategy's evaluation budget — broad exploration early, focused exploitation late. Out-of-bounds components are repaired by midpoint reflection. Strictly stronger than the basic ``DE/rand/1/bin`` on multimodal benchmarks; opt-in via the structural mutation catalog.
- **JSO**: jSO (Brest, Maučec & Bošković, CEC 2017 winner) — direct subclass of :class:`~panobbgo.heuristics.lshade.LSHADE` that adds three literature-best refinements to the success-history adaptive DE machinery: a **weighted current-to-pbest-w/1 mutation** (the pbest direction is re-weighted by a phase-dependent ``F_w`` factor — ``0.7·F`` early, ``0.8·F`` mid, ``1.2·F`` late), a **linear ``p_best`` schedule** that decreases from ``0.25`` to ``0.125`` over the budget, and **Cauchy-F clamping** that limits sampled ``F`` values to ``0.7`` while ``progress < 0.6``. The history memory uses ``M_F = 0.3`` / ``M_CR = 0.8`` initial values and reserves the last bin (``H − 1``) as a frozen anchor at ``0.9 / 0.9`` that ``_update_memory`` never overwrites — a stable "moderately greedy" parameter source independent of live success history. Inherits LSHADE's asynchronous pipeline (per-slot pending dict, generation-by-count book-keeping, archive trimming, LPSR shrinking, warm restart) unchanged. Opt-in via the structural mutation catalog.
- **NLSHADE_RSP**: NL-SHADE-RSP (Stanovov, Akhmedova & Semenkin, CEC 2021 winner) — direct subclass of :class:`~panobbgo.heuristics.jso.JSO` that adds three further refinements on top of the jSO machinery: **Non-Linear Population Size Reduction** (``NP(r) = round((NP_min − NP_init)·r^(1−r) + NP_init)``, which drops the population faster than L-SHADE's linear schedule in the early phase), **Rank-based Selective Pressure** (the differential ``r1`` index is drawn with probability proportional to a fitness rank weight ``k_rank·(n−i)/n + 1``, biasing the mutation toward better individuals; ``k_rank`` default ``3``), and a **randomised adaptive archive** (the archive cap is resampled per generation in ``[0, round(archive_factor·NP)]``). Inherits jSO's weighted ``current-to-pbest-w/1`` mutation, linear ``p_best`` schedule, asymmetric F-cap, frozen anchor memory bin, and the asynchronous pipeline unchanged. The full CEC-2021 adaptive-crossover blend and success-ratio archive-probability adaptation are intentionally not ported (see the heuristic docstring). Opt-in via the structural mutation catalog.
- **DifferentialEvolution / LSHADE / JSO / NLSHADE_RSP complementarity**: Panobbgo carries all four DE-family arms in the structural catalog so the bandit can pick whichever wins on the current battery — basic DE for byte-identical legacy reproduction, L-SHADE for the established CEC-2014 high-water mark, jSO for the CEC-2017 refinement, and NL-SHADE-RSP for the CEC-2021 refinement. Each variant occupies a distinct exploration / exploitation regime and the literature shows real per-problem complementarity; on Panobbgo's small-budget battery NL-SHADE-RSP measures within noise of jSO (mean composite delta ≈ 0 across seeds, seed-dependent wins on both sides) — the CEC-DE refinements are large-budget specialists.
- **NLSHADE_LBC**: NL-SHADE-LBC (Stanovov, Akhmedova & Semenkin, CEC 2022 winner) — direct subclass of :class:`~panobbgo.heuristics.nl_shade_rsp.NLSHADE_RSP` that adds **Linear Bias Change** in the success-history memory update. The standard L-SHADE / jSO / NL-SHADE-RSP Lehmer mean uses fixed exponents (``s^2 / s^1``); NL-SHADE-LBC generalises the F / CR memory updates to ``Σ(w·s^p) / Σ(w·s^(p − m))`` with the *order* ``p`` linearly scheduled across budget progress — ``p_F(r) = (1 − r)·p_F_init + r·p_F_final`` (defaults ``3.5 → 1.5``) and ``p_CR(r) = (1 − r)·p_CR_init + r·p_CR_final`` (defaults ``1.0 → 1.5``). The spread ``m`` is held constant at the literature default ``1.5``; at ``p = 2, m = 1`` the formula recovers the standard L-SHADE weighted Lehmer mean. Inherits NL-SHADE-RSP's NLPSR, RSP, randomised adaptive archive, jSO's frozen anchor bin, and the asynchronous pipeline unchanged. The full CEC-2022 adaptive crossover blend and repetitive-generation bound-constraint handling are intentionally not ported (see the heuristic docstring). Opt-in via the structural mutation catalog.
- **DifferentialEvolution / LSHADE / JSO / NLSHADE_RSP / NLSHADE_LBC complementarity**: Panobbgo carries all five DE-family arms in the structural catalog so the bandit can pick whichever wins on the current battery — basic DE for byte-identical legacy reproduction, L-SHADE for the established CEC-2014 high-water mark, jSO for the CEC-2017 refinement, NL-SHADE-RSP for the CEC-2021 refinement, and NL-SHADE-LBC for the CEC-2022 refinement. Each variant occupies a distinct exploration / exploitation regime and the literature shows real per-problem complementarity; on Panobbgo's small-budget battery the CEC-DE refinements (RSP, LBC) measure within noise of jSO — they are large-budget specialists by construction.
- **PSO**: Particle Swarm Optimization with the canonical Clerc–Kennedy (2002) constriction-coefficient parameters — a swarm of particles tracks personal and global bests with momentum, providing exploration dynamics distinct from CMA-ES (covariance) and DE (recombination); supports IPOP-style warm restarts via the Restart analyzer. Three swarm topologies are available via the ``topology`` argument: the default fully-connected ``"gbest"`` (Kennedy-Eberhart 1995), the wrap-around ring ``"lbest"`` of width ``2·k_neighbors+1`` (Kennedy & Mendes 2002), and the 4-connected 2-D toroidal grid ``"vonneumann"`` (Kennedy & Mendes 2003; Mendes 2004) — three complementary information-diffusion regimes (instantaneous / one-hop linear / two-hop planar), with ``"vonneumann"`` widely cited as a stable middle ground that wins on a broader range of problem classes than either pure ring or pure star. Optional ``w_end`` enables the Shi–Eberhart (1998) linearly-decreasing inertia schedule (off by default; constant ``w`` reproduces the prior behaviour).
- **FeasibleSearch**: Actively searches for and repairs feasible solutions via line search towards feasibility.
- **ConstraintGradient**: Uses estimated gradients of constraint violations to find feasible regions.
Expand Down
2 changes: 2 additions & 0 deletions panobbgo/heuristics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
from .lshade import LSHADE
from .jso import JSO
from .nl_shade_rsp import NLSHADE_RSP
from .nl_shade_lbc import NLSHADE_LBC
from .repair import ConstraintRepair
from .cma_es import CMAES
from .pso import PSO
Expand All @@ -70,6 +71,7 @@
"LSHADE",
"JSO",
"NLSHADE_RSP",
"NLSHADE_LBC",
"ConstraintRepair",
"CMAES",
"PSO",
Expand Down
Loading