Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions docs/PAPER_OCP_VALIDATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,50 @@ All rows use `nfe=12`, `ncp=3`, `LAGRANGE-RADAU`, the policy initializer, and a
| `n_z=5` | optimal | ~6.19 h | Policy 1 -> Policy 2 |
| `n_z=20` | optimal/acceptable | ~6.19 h | Policy 1 -> Policy 2 |

## Paper-vs-LyoPRONTO Baseline Comparison

Issue #30 compares the paper-reference direct transcription against current
LyoPRONTO baselines before adding any LyoPRONTO-facing policy OCP API. The
closest comparable LyoPRONTO case is the `Tsh` benchmark: shelf temperature is
optimized while chamber pressure is fixed. The committed baseline data were
generated with the current SciPy optimizer and quasi-steady Pyomo FD/collocation
optimizers over the `baseline` 3x3 grid:

```bash
python benchmarks/grid_cli.py generate \
--task Tsh --scenario baseline \
--vary product.A1=16,18,20 \
--vary ht.KC=2.75e-4,3.3e-4,4.0e-4 \
--methods scipy,fd,colloc \
--n-elements 24 --n-collocation 3 \
--out benchmarks/results/baseline_Tsh_3x3_summary.jsonl
```

`Pch` data in `benchmarks/results/baseline_Pch_3x3_summary.jsonl` were also
checked as a non-comparable optimizer sanity run. They use chamber pressure as
the manipulated variable with a fixed shelf profile, so they should not be used
as a paper OCP comparison point. Their drying-time ranges were 18.12-23.67 h
for SciPy, 18.15-23.71 h for Pyomo FD, and 17.90-23.37 h for Pyomo collocation.

| Model/case | Comparable scope | Drying time | Active constraints/control behavior | Temperature behavior | Interpretation |
| --- | --- | --- | --- | --- | --- |
| Paper Problem 1 direct transcription | Shelf-only OCP, fixed chamber pressure, SI/Kelvin moving-boundary model | ~6.19 h | Policy 1 max heat input, then Policy 2 product-temperature tracking near 2.4 h | Max product temperature tracks 243 K after the switch; shelf temperature backs away from 273 K | Useful validation target for direct transcription and policy classification, but not a drop-in LyoPRONTO baseline |
| LyoPRONTO `Tsh` SciPy baseline | Shelf-only quasi-steady optimizer, fixed 0.1 Torr chamber pressure, cm/Torr/degC model | 12.19-14.47 h across the 3x3 grid; mean 13.33 h | Shelf profile optimizer with product-temperature limit as the active path constraint | Bottom product temperature tracks -25 degC at the constraint | Closest current control-space comparison; drying-time gap is dominated by formulation, units, scenario, and constraint differences |
| LyoPRONTO `Tsh` Pyomo FD | Same quasi-steady benchmark as SciPy, finite-difference transcription, 24 elements | 12.40-14.76 h; mean 13.58 h | Same active product-temperature constraint; no warm start in committed grid | Tracks the same -25 degC limit with 98.9% terminal drying target | Matches the SciPy baseline closely enough for LyoPRONTO regression use, but does not exercise paper Policy 3/Policy 2 structure |
| LyoPRONTO `Tsh` Pyomo collocation | Same quasi-steady benchmark as SciPy, collocation with effective mesh parity | 12.00-14.23 h; mean 13.11 h | Same active product-temperature constraint on a collocation mesh | Tracks the same -25 degC limit with small objective shifts versus SciPy | Confirms transcription effects are modest inside the LyoPRONTO model, separate from paper-vs-LyoPRONTO physics differences |
| Paper Problem 2 direct transcription | Shelf-only OCP with an interface-velocity path constraint | ~8.9 h first-pass coarse validation | Policy 3 interface-velocity tracking, then Policy 1 max heat input, then Policy 2 temperature tracking | Product temperature stays below 240 K after the velocity-limited phase | No current LyoPRONTO optimizer is directly comparable because the public quasi-steady APIs do not expose a flux or interface-velocity cap |

Recommendation: keep the paper-reference model validation-only and adapt the
policy constraints into LyoPRONTO instead of exposing the SI/Kelvin paper model
as a public optimizer. The paper model should remain in
`lyopronto.pyomo_models.paper_ocp` for upstream validation, convergence checks,
and policy-classification tests. The LyoPRONTO-facing work should proceed
through #31, which already converts the adapter requirements into an
implementation issue: add an optional flux/interface-velocity cap in LyoPRONTO
units, reuse the existing quasi-steady Pyomo optimizer infrastructure, return a
rich result object, classify active policies from LyoPRONTO trajectories, and
avoid changing existing SciPy or Pyomo optimizer behavior.

## Problem 2 First-Pass Tolerances

The Problem 2 validation is intentionally coarse at this stage. The paper
Expand Down
17 changes: 17 additions & 0 deletions tests/test_benchmarks.py
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,23 @@ def test_benchmark_results_gitignore_blocks_generated_outputs(repo_root):
assert allowed_result.returncode == 1, allowed_result.stdout


def test_paper_ocp_notes_cover_issue30_comparison(repo_root):
notes = (repo_root / "docs" / "PAPER_OCP_VALIDATION.md").read_text(encoding="utf-8")

assert "## Paper-vs-LyoPRONTO Baseline Comparison" in notes
assert (
"| Model/case | Comparable scope | Drying time | Active constraints/control behavior | Temperature behavior | Interpretation |"
in notes
)
assert "benchmarks/results/baseline_Tsh_3x3_summary.jsonl" in notes
assert "12.19-14.47 h" in notes
assert "12.40-14.76 h" in notes
assert "12.00-14.23 h" in notes
assert "Recommendation: keep the paper-reference model validation-only" in notes
assert "#31" in notes
assert "optional flux/interface-velocity cap" in notes


def test_compute_residuals_uses_native_bools_and_checks_product_temperature():
traj = np.array(
[
Expand Down
Loading