Skip to content

generate_omars(n_runs=n, center_runs=c) returns n + c - 1 runs, not n #496

Description

@kgdunn

Summary

generate_omars(n_runs=n, center_runs=c) returns n + c - 1 runs, not n, whenever center_runs > 1. The parameter named n_runs is therefore not the number of runs in the returned design.

This contradicts the docstring, which documents n_runs as the "Exact (odd) run size" and describes the result as "a foldover [H; -H; 0] with 2*h + 1 runs".

Why it matters

It silently mis-sizes any comparison made by run count. Anyone benchmarking designs of "the same size", or building a run-count-versus-quality table, will compare a larger design against a smaller one without any error or warning. It cost us two apparently impossible results in a design comparison before we spotted it: a 15-run design was being scored against a 13-run reference.

Reproduction

import numpy as np
from process_improve.experiments import Factor, generate_omars

facs = lambda k: [Factor(name=chr(65 + i), low=-1, high=1) for i in range(k)]
SOLVER = {"time_limit": 30, "msg": False}

for k in (3, 4):
    for c in (1, 2, 3, 5):
        r = generate_omars(facs(k), n_runs=13, center_runs=c,
                           model="main_quadratic", solver_options=SOLVER)
        arr = r.design[r.factor_names].to_numpy(dtype=float)
        ncent = int((np.abs(arr).sum(axis=1) == 0).sum())
        print(f"k={k} n_runs=13 center_runs={c}: returned {arr.shape[0]} runs, {ncent} centre")

Actual

k=3 n_runs=13 center_runs=1: returned 13 runs, 1 centre
k=3 n_runs=13 center_runs=2: returned 14 runs, 2 centre
k=3 n_runs=13 center_runs=3: returned 15 runs, 3 centre
k=3 n_runs=13 center_runs=5: returned 17 runs, 5 centre
k=4 n_runs=13 center_runs=1: returned 13 runs, 1 centre
k=4 n_runs=13 center_runs=2: returned 14 runs, 2 centre
k=4 n_runs=13 center_runs=3: returned 15 runs, 3 centre
k=4 n_runs=13 center_runs=5: returned 17 runs, 5 centre

Returned runs = n_runs + center_runs - 1 in every case with center_runs > 1.

Expected

One of:

  1. n_runs means the total run count, and the extra centre runs are taken from within it (so center_runs=3, n_runs=13 gives h=5, N=13); or
  2. the parameter is renamed to reflect what it controls (it is currently the foldover size 2h+1, i.e. the design before the extra centre runs are appended), and the docstring is corrected.

Option 1 matches the documented contract and is what a caller sizing an experiment to a budget would expect. Either way, the docstring's "Exact (odd) run size" and "2*h + 1 runs" need updating, since with center_runs > 1 the returned design is neither.

Note

The parity claim also stops holding: with an even center_runs the returned run count is even, while the docstring says the run count is odd.

Found while building exhaustive ground-truth tables for OMARS designs. Unrelated to #494.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions