fix(experiments): build the published Box-Behnken design at six and seven factors - #501
Open
kgdunn wants to merge 7 commits into
Open
fix(experiments): build the published Box-Behnken design at six and seven factors#501kgdunn wants to merge 7 commits into
kgdunn wants to merge 7 commits into
Conversation
omars_properties checked that each factor reaches the middle level at least once, so that its pure quadratic is not the constant column a two-level factor gives. The mirror case was not checked: a factor left at the middle level in every run has the constant quadratic 0. Both are inestimable, and only the first was being caught. The consequence was worse than a permissive verifier. max_second_order_correlation skips constant columns, which is right on its own terms, but a pinned factor removes its own quadratic and all of its interaction columns from the comparison, so the score improves. generate_omars(selection_criterion= "min_second_order_correlation") minimises exactly that quantity, so it had a direct incentive to produce the degenerate designs the verifier was accepting, and did. Across a three-to-six factor sweep it returned one in roughly a third of cells, including a spurious perfect 0.000 at three factors in nine runs. The fix is to require an outer level as well as a middle one. With the verifier corrected the ILP rejects those candidates during the search and spends its budget on real ones instead: degenerate results fell from twelve to three, and two sizes that previously produced no usable design at all, three factors in nine runs and five factors in thirteen, now produce one. Five runs in three factors is the smallest case that shows the failure and is the new regression test: with the third factor pinned at the centre, is_omars returned True while the main-and-quadratic model matrix had rank 4 of 7. is_omars now returns False for matrices it previously accepted. Every such matrix has an inestimable quadratic, so no correct caller can be relying on the old answer, and it is corrected in place rather than deprecated. Verification: ruff check . and ruff format --check . clean mypy src/process_improve clean, 146 source files pytest -k omars: 291 passed, 2 skipped the 13-run definitive screening design still verifies Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
…even factors dispatch_box_behnken delegated to pyDOE3.bbdesign, which places a two-level factorial in every pair of factors whatever the factor count. Pairing reproduces the published design up to five factors, but from six factors upwards Box and Behnken (1960) use blocks of three factors taken from a balanced incomplete block design. Pairing therefore cost 60 and 84 design runs at six and seven factors against the published 48 and 56. The library was already inconsistent with itself about this: _BBD_RUNS in experiments.strategy.budget has always quoted 48 and 56, so estimate_rsm_runs promised a design the generator did not produce. The two now agree at every factor count from three to seven, which a new test asserts directly. Both the old and the new designs keep the main effects orthogonal to every second-order term and reach full rank on the second-order model, so nothing changes about what can be estimated. What changes is the run count, and with it the efficiency per run. The structural property test was encoding the pairing assumption, that every non-centre run has exactly two non-zero coordinates. It now checks the general invariant: a non-centre run varies the factors of one block and holds the rest at the centre level, with the block size depending on the factor count. Factor counts above seven still fall through to pyDOE3, which is documented at the call site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…Box-Behnken designs The table showed run budgets from 9 to 57 runs with no fixed reference at either end, so a reader could not see where the standard designs sit on it. The definitive screening design is the smallest member of the OMARS family and the Box-Behnken design is among the largest, so a table carrying both shows the span the family covers rather than only its middle. omars_trade_off_table(anchors=True) adds a DSD row above the budgets and a BBD row below them. The default is False, so the returned frame is unchanged for existing callers. Anchor run counts change from column to column, unlike a budget row where the count is the row label, so anchor cells lead with it: "46 Full df=25". Two supporting functions are public, definitive_screening_runs and box_behnken_runs, the second returning None where Box and Behnken published no design. Anchor rows are classified without the 2h + 1 parity gate. That gate is right for a budget row, where the question is whether any foldover has that run count with a single centre run, but a named design carries whatever centre replication its published form specifies. A Box-Behnken design has three or six centre runs, so its total is even from five factors upwards, and rejecting it as "not a design" would be wrong. The capability thresholds still apply unchanged: they are set by the number of distinct half-rows, which extra centre runs do not change. Every anchor cell was checked against the design the library actually generates, on run count, on the rank of the relevant model matrix, and on error degrees of freedom, for three to seven factors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
The figure script and the book's code block both need the anchor cells, and neither can build them from get_omars_trade_off_table_entry: a Box-Behnken design carries three or six centre runs, so its total is even from five factors upwards and the 2h + 1 parity gate correctly reports no design at that budget. Reaching into a private for that was the wrong shape. _reference_entry becomes omars_anchor_entry(design, n_factors), exported from process_improve.experiments alongside definitive_screening_runs and box_behnken_runs, with the design name validated and the difference from the budget path documented. Tests cover the rendering of every anchor cell, that error df is runs minus parameters, that the DSD never reaches Full, that the thresholds travel with an anchor as they do with a budget, and the case that motivated the split: 46 runs at five factors is no design as a budget and is the Box-Behnken design as an anchor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
… as rows The DSD row duplicated cells the table already had: a definitive screening design has 9, 13 or 17 runs, which are budgets on the table, so the row repeated them. It is now a marker appended to the cell it lands in, "Quad df=2 | DSD". The Box-Behnken row is now a marker too, but its run counts are not budgets, so the table adds a row for each: 15, 27, 46, 54 and 62. Each column ends there. Below its Box-Behnken cell every remaining row would say Full again on more runs, so those cells are left blank and the column closes. Reading down a column now shows the whole span of the family for that factor count, from the smallest design that exists to the standard response surface design that ends it, with no repetition at either end. omars_anchor_entry now returns the ordinary label, since it sits on the row of its own run count and no longer needs to carry it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
The entry described the earlier anchor-row form, which has been replaced by marking each standard design on the row of its own run count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
_REFERENCE_TAGS held the row labels for the DSD and BBD anchor rows. Those rows were replaced by markers appended to the cell each design lands in, so nothing reads it any more. CodeQL flagged it on the pull request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1daHPaXLs7PPRsq8CmMMm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dispatch_box_behnkendelegated topyDOE3.bbdesign, which places a two-level factorial in every pair of factors whatever the factor count. That reproduces the published design up to five factors, but from six factors upwards Box and Behnken (1960) use blocks of three factors from a balanced incomplete block design. Pairing therefore cost 60 and 84 design runs at six and seven factors, against the published 48 and 56._BBD_RUNSinexperiments.strategy.budgethas always quoted 48 and 56, soestimate_rsm_runspromised a design the generator did not produce. They now agree at every factor count from three to seven.Needed by pid-book #270, which anchors the OMARS trade-off table with a Box-Behnken row and so has to quote run counts that match the literature.
Test plan
pytest tests/test_design_properties.py tests/test_evaluate_design.py tests/test_strategy_budget.py tests/test_design_generation.py— 235 passed.ruff check .andruff format --check .both clean;mypy src/process_improvereports no issues in 146 files.Factor counts above seven still fall through to
pyDOE3, documented at the call site.Checklist
pyproject.toml— not bumped, per the maintainer's standing instruction for this series of PRs.ruff check .passesCHANGELOG.mdupdatedGenerated by Claude Code