QPMaOEA

MATLAB implementation of the Q-learning driven prediction approach for dynamic constrained many-objective optimization.

This repository is prepared for the paper:

Miao Rong, Yunjie Wang, and Chen Peng, "A Q-learning driven prediction approach to dynamic constrained many-objective engineering system optimization", Expert Systems with Applications, 322, 132320, 2026. DOI: 10.1016/j.eswa.2026.132320

What is included

• QPMaOEA.m: paper-name entry point.
• CTAEA_RL_Dynamic.m: configurable MATLAB runner.
• D*_C*_DTLZ*.m: dynamic constrained DTLZ benchmark problems.
• PortfolioOptimization.m: dynamic engineering portfolio example.
• UpdateCA.m and UpdateDA.m: convergence/diversity archive updates.
• IGD.m and HV.m: metrics used by the runner.

The code follows the paper's core structure: a C-TAEA-style dual archive, dynamic constrained benchmark problems, and a Q-learning response mechanism with four actions: diversity-archive reinitialization, elite perturbation, historical elite reuse, and a hybrid response.

Requirements

• MATLAB R2021a or newer is recommended.
• Statistics and Machine Learning Toolbox is required for pdist2.

The project has been prepared and smoke-tested with MATLAB R2025b.

Relationship with PlatEMO

This repository is related to PlatEMO and includes a lightweight adapter that matches the official PlatEMO folder layout.

• It can run as a standalone MATLAB implementation through QPMaOEA(...).
• Several helper routines are adapted from PlatEMO and keep their original notices: MatingSelection.m, NDSort.m, OperatorGAhalf.m, UpdateCA.m, and UpdateDA.m.
• The PlatEMO/ folder follows the official PlatEMO layout and provides an ALGORITHM class (QPMaOEA_PlatEMO) plus PROBLEM wrappers (QPM_*).
• If this repository and a full PlatEMO checkout are both on the MATLAB path, keep this repository earlier on the path for standalone runs because helper names such as NDSort, UniformPoint, and OperatorGAhalf can overlap.

If a fully permissive MIT-only release is required, the PlatEMO-derived files should be replaced with original implementations or PlatEMO should be treated as an external dependency.

PlatEMO command-line example:

Algorithm = QPMaOEA_PlatEMO('save', 1);
Problem   = QPM_D1_C1_DTLZ1('N', 100);
Algorithm.Solve(Problem);

See PlatEMO/README.md for installation notes.

Quick start

From the repository root, run:

[PF, results] = QPMaOEA('Problem', 'D1_C1_DTLZ1', 'Seed', 1);

PF is the final archive objective matrix. Full archive matrices are also available as results.archiveObjectives, results.archiveDecisions, and results.archiveConstraints.

For a shorter smoke run:

[PF, results] = QPMaOEA( ...
    'Problem', 'D1_C1_DTLZ1', ...
    'N', 30, ...
    'Seed', 1, ...
    'MaxFE', 300, ...
    'ChangeInterval', 20);

Results are written to results/ by default. To disable files:

[PF, results] = QPMaOEA('SaveResults', false);

Configurable options

QPMaOEA(...) forwards options to CTAEA_RL_Dynamic(...).

Common options:

• Problem: one of D1_C1_DTLZ1, D1_C2_DTLZ2, D2_C1_DTLZ1, D2_C2_DTLZ2, D3_C1_DTLZ1, D3_C2_DTLZ2, or PortfolioOptimization.
• N: target population size. The final size can be adjusted by the uniform reference point generator.
• Seed: random seed for reproducible runs.
• MaxFE: optional maximum number of function evaluations.
• ChangeInterval: generations between environmental changes.
• TimeStep: dynamic time increment.
• OutputDir: directory for .mat, Pareto-front, and IGD files.
• SaveResults: set to false for in-memory runs.
• Verbose: set to false to reduce console output.

Reproducibility notes

• The runner sets MATLAB's random stream when Seed is provided.
• The portfolio example uses deterministic time-varying market data instead of drawing fresh random data during each evaluation.
• The included HV.m is a deterministic normalized approximation intended for comparable experiment logging in many-objective settings.

Third-party code

Several archive update, selection, sorting, and genetic operator routines are adapted from PlatEMO and retain their original notices in the source files. See NOTICE for details and citation requirements.

License

Original QPMaOEA code is released under the MIT License. Files that include third-party notices retain their original terms. See LICENSE and NOTICE.