NNML Paper Reproduction

NNML-only reconstruction of Neural Network-Assisted Multitask Learning for Multiobjective Optimization Problems With Heterogeneous Expensive Constraints (DOI 10.1109/TEVC.2026.3706453).

The implementation covers MW1-MW14 and LIRCMOP1-LIRCMOP14 at the supplement-specified D=10, including the NNML training/evolution loop, PlatEMO-aligned operators, and HV/IGD/IGD+ evaluation.

Reproduction status

The method and evidence gates are verified, but the complete paper experiment is still in progress. The current official checkpoint contains 320 of 448 runs (20 of 28 problems). It must remain labeled METHOD_VERIFIED_PENDING_FULL_SUITE, not PAPER_ALIGNED.

Item	Current checkpoint
Automated tests	84/84 passing
Official runs	320/448 (71.4%)
Problems with 16 runs	20/28
Aligned metric cells	37/84
Remaining runs	LIRCMOP7-LIRCMOP14, 128 runs

See the Chinese progress report and the paper alignment report for the detailed interpretation.

Setup

The pinned PlatEMO source is a Git submodule:

git submodule update --init --recursive
py -3.14 -m venv venv
venv\Scripts\python.exe -m pip install -r requirements-paper.txt
venv\Scripts\python.exe scripts\check_paper_deps.py

The current dependency lock is recorded in requirements-paper.txt.

Verification

venv\Scripts\python.exe -m unittest discover -s tests -v
venv\Scripts\python.exe scripts\verify_platemo_fixtures.py

Verified, hashed PlatEMO fixtures are committed under reference_data/platemo_oracles/. MATLAB or GNU Octave is needed only to regenerate those fixtures with scripts/build_platemo_fixtures.py; it is not needed for tests or NNML runs.

The official supplement and all 56 published NNML IGD/IGD+ cells can be revalidated with:

venv\Scripts\python.exe scripts\import_supplement_targets.py --supplement reference_sources\nnml_supplement.pdf

Add --write only when intentionally refreshing the pinned target data.

Run or resume the official experiment

venv\Scripts\python.exe scripts\run_paper.py --suite both --runs 16 --profile paper --output-dir results\paper
venv\Scripts\python.exe scripts\compare_paper_results.py --results results\paper --targets reference_data\paper_targets.csv
venv\Scripts\python.exe scripts\audit_paper_completion.py --results results\paper

The runner resumes completed run directories and rebuilds the run/summary CSV files after each completion. It uses the 16 frozen seeds in reference_data/paper_seeds.json.

Repository layout

Path	Purpose
nnml_repro/	NNML implementation, benchmarks, operators, metrics, and provenance
scripts/	experiment, comparison, audit, fixture, and diagnostic entry points
tests/	unit, protocol, oracle, and end-to-end smoke tests
reference_data/	frozen seeds, paper targets, evidence gates, and oracle fixtures
reference_sources/	pinned paper text, official supplement, and PlatEMO submodule
results/paper/	resumable official experiment checkpoint
docs/	protocol, benchmark-source, and progress audits

Generated smoke and diagnostic runs are intentionally ignored. Only the resumable results/paper/ checkpoint is versioned; superseded pilots and pre-fix artifacts are recoverable from Git history.

Known limitations

• The paper does not name the neural optimizer; Adam at 1e-3 is an audited reconstruction assumption.
• The authors' exact 16 seeds and complete source code are unavailable, so the goal is statistical rather than bitwise reproduction.
• The current checkpoint was generated with CPU-only PyTorch, whereas the paper reports an RTX 4090.

The authoritative protocol is NNML_PAPER_REPRODUCTION_WORKFLOW.md.