Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,8 @@ study.run(n_jobs=-1)

# 4. Inspect results
print(study.summary())
front = study.front() # non-dominated configs
hv = study.front_hypervolume(ref=...) # hypervolume indicator
front = study.front("benchmark") # non-dominated config indices
hv = study.front_hypervolume("benchmark", ref_point) # hypervolume indicator
```

### Protocols
Expand Down Expand Up @@ -156,7 +156,7 @@ factors = [
]

grid = build_grid(factors, method="sobol", n_samples=1024)
si = screen(run_fn, factors, method="morris", n_eval=3000)
si = screen(run_fn, factors, method="morris", n_trajectories=3000)
```

### Execution
Expand All @@ -176,9 +176,9 @@ results = run_adaptive(world, scorer, factors, observables, n_trials=600)
```python
from trade_study import extract_front, hypervolume, pareto_rank

front_mask = extract_front(results.scores, directions)
front_idx = extract_front(results.scores, directions)
ranks = pareto_rank(results.scores, directions)
hv = hypervolume(results.scores[front_mask], directions, ref=ref_point)
hv = hypervolume(results.scores[front_idx], ref_point, directions)
```

### Stacking
Expand Down
4 changes: 4 additions & 0 deletions docs/api/study.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ Multi-phase study orchestration.
::: trade_study.Study

::: trade_study.top_k_pareto_filter

::: trade_study.weighted_sum_filter

::: trade_study.feasibility_filter
11 changes: 11 additions & 0 deletions docs/api/viz.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Visualization

Plotting utilities for trade-study results.

::: trade_study.plot_front

::: trade_study.plot_parallel

::: trade_study.plot_scores

::: trade_study.plot_calibration
Binary file modified docs/assets/cstr_front.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/cstr_parallel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/cstr_selectivity.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
71 changes: 67 additions & 4 deletions docs/guide/bayesian.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,12 @@ metrics:

- **CRPS** (Continuous Ranked Probability Score) — a proper scoring
rule that rewards sharp, well-calibrated predictive distributions.
- **Energy score** — a multivariate proper scoring rule
complementary to CRPS.
- **Coverage (95 %)** — the fraction of test points whose true value
falls inside the 95 % posterior predictive interval.
- **RMSE** — root mean squared error of the posterior mean.
- **MAE** — mean absolute error of the posterior mean.

```python
--8<-- "examples/bayesian_study.py:world"
Expand Down Expand Up @@ -110,6 +113,16 @@ cost model for the number of observations:
--8<-- "examples/bayesian_study.py:annotation"
```

## Constraints

A `Constraint` defines a hard feasibility bound on an observable.
`feasibility_filter` builds a phase filter that drops any design
violating the constraints:

```python
--8<-- "examples/bayesian_study.py:constraint"
```

## Factor screening

Before running the full grid, Morris screening identifies which
Expand Down Expand Up @@ -157,6 +170,56 @@ overconfident nor excessively vague.

![CRPS strip plot](../assets/bayesian_crps.png)

## Quality indicators

`hypervolume` measures the dominated volume of the Pareto front;
`igd_plus` (inverted generational distance plus) measures how close
the front is to a reference set. Together they quantify front
quality — useful for comparing design methods or grid sizes:

```python
--8<-- "examples/bayesian_study.py:igd_plus"
```

## Feasibility filtering

Apply `Constraint` + `feasibility_filter` to keep only designs
meeting hard bounds (here, `coverage_95 >= 0.90`):

```python
--8<-- "examples/bayesian_study.py:feasibility"
```

## Weighted-sum filtering

`weighted_sum_filter` scalarises objectives into a single score
via min-max normalisation and weighted combination, then keeps
the top-K designs. Useful when you want a simple ranking
without full Pareto analysis:

```python
--8<-- "examples/bayesian_study.py:weighted_sum"
```

## Sobol grid

`build_grid(method="sobol")` generates quasi-random points with
better space-filling properties than LHS for low dimensions:

```python
--8<-- "examples/bayesian_study.py:sobol"
```

## Study workflow

The `Study` class orchestrates multi-phase workflows.
`Study.summary()` returns per-phase statistics and
`Study.stack()` computes stacking weights directly:

```python
--8<-- "examples/bayesian_study.py:study_workflow"
```

## Score-based stacking

`stack_scores` finds simplex-constrained weights that minimise the
Expand Down Expand Up @@ -222,9 +285,9 @@ rather than a design factor:

## What to try next

- Swap `method="morris"` for `method="sobol"` in `screen()` for
variance-based sensitivity indices.
- Use `Constraint` + `feasibility_filter` to enforce
`coverage_95 >= 0.90` before stacking.
- Try `stack_bayesian()` on models that expose log-likelihood
(requires `arviz`).
- Add more phases to the `Study` — e.g. a third phase that
re-evaluates with even higher fidelity.
- Experiment with different `weighted_sum_filter` weight vectors
to explore different trade-off preferences.
53 changes: 50 additions & 3 deletions docs/guide/cstr.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,25 @@ operating parameters with physical bounds:
--8<-- "examples/cstr_study.py:factors"
```

## Constraints

Hard feasibility bounds can be defined with `Constraint` objects.
`feasibility_filter` builds a phase filter that drops designs
violating any constraint:

```python
--8<-- "examples/cstr_study.py:constraint"
```

## Factor screening

Before committing to a full grid sweep, Sobol screening identifies
which factors influence the objectives most:

```python
--8<-- "examples/cstr_study.py:screening"
```

## Build the study phases

A `Study` chains multiple **Phases**. Phase 1 explores the design space
Expand Down Expand Up @@ -154,9 +173,37 @@ conversion–selectivity trade-off.

## What to try next

- Add a **screening** step with `screen()` to identify which factors
matter most before running the full grid.
- Use `run_adaptive()` for Bayesian optimization instead of LHS.
- Call `stack_bayesian()` on the Pareto front to compute model-averaging
weights.
- Save results with `save_results()` for later analysis.
- Combine `feasibility_filter` with `top_k_pareto_filter` across
successive phases for constrained multi-objective optimisation.

## Progress callbacks and parallel execution

`run_grid` accepts a `callback` for progress reporting and `n_jobs`
for parallel execution via joblib:

```python
--8<-- "examples/cstr_study.py:callback"
```

## Adaptive exploration

When the design space is too large for a grid sweep, use
`Phase(grid="adaptive")` for optuna-driven multi-objective
Bayesian optimisation. The `factors` argument on `Study` provides
the parameter bounds:

```python
--8<-- "examples/cstr_study.py:adaptive"
```

## Feasibility-constrained phases

Use `feasibility_filter` as a phase filter to enforce hard
constraints before passing designs to subsequent phases:

```python
--8<-- "examples/cstr_study.py:feasibility"
```
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ design-of-experiments studies:
| [Protocols](api/protocols.md) | Core types: `Observable`, `Direction`, `Scorer`, `Simulator`, etc. |
| [Design](api/design.md) | `Factor`, `build_grid`, `screen`, `reduce_factors` |
| [Runner](api/runner.md) | `run_grid`, `run_adaptive` |
| [Study](api/study.md) | `Phase`, `Study`, `top_k_pareto_filter` |
| [Study](api/study.md) | `Phase`, `Study`, `top_k_pareto_filter`, `weighted_sum_filter`, `feasibility_filter` |
| [Scoring](api/scoring.md) | `score`, `coverage_curve` |
| [Pareto](api/pareto.md) | `extract_front`, `pareto_rank`, `hypervolume`, `igd_plus` |
| [Stacking](api/stacking.md) | `stack_bayesian`, `stack_scores`, `ensemble_predict` |
| [Visualization](api/viz.md) | `plot_front`, `plot_parallel`, `plot_scores`, `plot_calibration` |
| [I/O](api/io.md) | `load_results`, `save_results` |
Loading
Loading