[Pyomo.DoE] Add simultaneous design of multiple experiments by smondal13 · Pull Request #3866 · Pyomo/pyomo

smondal13 · 2026-03-02T19:53:03Z

Fixes # .

Summary/Motivation:

This PR adds a new DesignOfExperiments.optimize_experiments() API in pyomo/contrib/doe/doe.py to support simultaneous optimization of multiple experiments in one workflow. The motivation is to provide a multi-experiment DoE interface with stronger initialization options, clearer mode handling (template vs. user-initialized experiments), and richer diagnostics/results than the existing single-experiment path.

Changes proposed in this PR:

Added API optimize_experiments() for multi-experiment DoE optimization.
Added Cholesky-based D- and A-optimality objective and Greybox-based D-, A-, E-, and ME- optimality objective
Implemented two operating modes:
- Template mode: pass one experiment and set n_exp.
- User-initialized mode: pass a list of experiments; n_exp is inferred/validated.
Added optional LHS-based initialization (initialization_method="lhs") with controls for:
- sample count, seed, candidate evaluation parallelism, combination fim metric scoring parallelism,
- worker count, chunk size, parallel threshold, and optional wall-clock budget.
Added symmetry-breaking constraints for multi-experiment solves:
- supports user-specified variable through sym_break_cons suffix,
- falls back to the first experiment input with a diagnostic warning when not provided.
Expanded output for this API:
- per-scenario and per-experiment results (designs, outputs, measurement errors, FIM/sensitivities),
- aggregated FIM metrics, timings, settings, names, diagnostics, and structured run_info.
Added JSON-safe serialization via _DoEResultsJSONEncoder for numpy/Pyomo-enum values when writing results_file.

Note:

Added new documentation.md which describes the API. This documentation is to help the reviewers to understand the API and will not be merged into Pyomo:main

Remove before merging

documentation.md
rb_multi.py

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…`experiment` argument is provided.

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.

…sing the `doe/reactor_multi_experiment.py`

…ultiexperiment.py` both for 3 experiments

…etric-uncertainty

…parametric-uncertainty

…riment

…nt for `optimize_experiments()` and `compute_FIM()`

… both of sensitivity and optimize_experiments()

…A-opt gave different result for grid and optimization

adowling2

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

smondal13 · 2026-05-28T04:26:11Z

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

@adowling2 I have addressed your comments. @adowling2 @mrmundt, I have resolved the conflicts. The conflicts were from @adowling2's recent PR #3867 merge into Pyomo:main.

smondal13 · 2026-05-28T04:42:42Z

@mrmundt @blnicho, the following two tests are failing. I think these two are not part of my PR

=========================== short test summary info ============================
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_any_solver - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_cplex - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set

blnicho · 2026-05-28T17:53:18Z

@smondal13 those failures were resolved in #3965 and I merged main into your PR to pick up the fix.

smondal13 · 2026-05-28T21:23:02Z

@blnicho @mrmundt this one is failing due to download issue

downloaded:
[FAIL] gjh
[ OK ] mcpp
Error: Process completed with exit code 1.

smondal13 · 2026-06-01T13:52:24Z

@blnicho @mrmundt Two tests are failing in this PR, which are not related to the PR.

blnicho

@smondal13 I still have 4 more files to go through but here are my comments so far.

blnicho · 2026-06-02T18:04:49Z

+        if hasattr(m, "sym_break_cons"):
+            m.sym_break_cons.clear()
+        else:
+            m.sym_break_cons = pyo.Suffix(direction=pyo.Suffix.LOCAL)
+        m.sym_break_cons[m.hour] = None


Why is this needed? Are you adding a new required sym_break_cons Suffix as part of this PR?

Yes — in multi-experiment design, sym_break_cons is an optional suffix that lets the user specify which experiment input should be used for symmetry breaking. If the user does not provide it, we automatically use the first variable in the experiment_inputs suffix.

We do this to reduce permutation symmetry and save solver time. In multi-experiment design, the order of experiments does not matter. For example, if we are designing two experiments with design variable T, then the designs (exp 1: T = 300 K, exp 2: T = 350 K) and (exp 1: T = 350 K, exp 2: T = 300 K) are equivalent. The symmetry-breaking constraint helps restrict these equivalent permutations so the solver only considers one ordering.

blnicho · 2026-06-02T18:08:01Z

+        m.temp = pyo.Var(initialize=self.temp, bounds=(280.0, 340.0))
+        m.temp.fix()
+
+        # Replace base Rooney-Biegler response with two-input synthetic variant
+        # used only for symmetry-breaking tests.
+        m.del_component(m.response_function)
+        m.response_function = pyo.Constraint(
+            expr=m.y
+            == m.asymptote * (1 - pyo.exp(-m.rate_constant * m.hour)) + 0.01 * m.temp
+        )
+        m.experiment_inputs[m.temp] = self.temp


If this is only used for symmetry-breaking tests, should this code be moved inside the if-statement below that is checking if the sym_break_flag is set?

We keep that code outside the sym_break_flag conditional because this helper is intended to be a two-input experiment in all cases. The flag only controls whether we add an explicit sym_break_cons suffix.

We still need the temp input and modified response when sym_break_flag=0, because that case is used to test the automatic fallback behavior when no explicit symmetry-breaking marker is provided. In that path, the code should infer the symmetry-breaking variable from experiment_inputs. Moving the two-input setup inside the conditional would remove that coverage.

blnicho · 2026-06-02T22:38:38Z

+        # The exact number of initialization solves is implementation-dependent,
+        # but they must all occur before the one final main-solver call.


Shouldn't this number be deterministic and known for this test?

In this test, init_solver is also used during finite-difference scenario construction and the square initialization solve, so it is expected to be called multiple times (deterministically) before the final optimization solve.

Right but my point is that for this specific example problem you should know exactly how many times init_solver will be called so why does the comment imply that the exact number is "unknowable" and the test check for a positive number of calls instead of the exact number?

You’re right. For this test, the init_solver call count is deterministic, so the assertion should be exact rather than a lower bound. I have added self.assertEqual(init_solver.calls, 11) instead of the previously used assertGreaterEqual.

…cstring for clarity and update test cases to use make_ipopt_solver() directly

… reflect deterministic behavior

blnicho

@smondal13 here is my next set of comments. I have 2 more files to look at.

Co-authored-by: Bethany Nicholson <blnicho@users.noreply.github.com>

… make_ipopt_solver directly

Co-authored-by: Bethany Nicholson <blnicho@users.noreply.github.com>

- Update the square-solve stub helper to use direct solver attribute access so the merged test helper matches the latest upstream cleanup. - Preserve the local DOE error-test change that calls make_ipopt_solver() directly, which keeps the solver setup explicit and avoids the extra wrapper.

…zation test with deterministic FIM input

…g test with fixed expected points

smondal13 added 30 commits January 14, 2026 10:03

Add _spec.md

ed6707a

Add optimize_experiment()

eed6329

Add experiment_list in the __init__ and a DeprecationWarningif …

4861662

…`experiment` argument is provided.

Rename finite difference scenario_blocks to fd_scenario_blocks an…

bf5eb91

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.

Add new objective method for multiple experiment.

f70e461

Add complete doe.py for multi experiment without testing /example

7c7332b

Solve the optimize_experiments() for both 1 and 2 experiment case u…

f5ba35e

…sing the `doe/reactor_multi_experiment.py`

Solve doe/examples/rooney_biegler_multiexperiment.py and `reactor_m…

79d09cc

…ultiexperiment.py` both for 3 experiments

Add plot and the grid best and optimal point points to the same point

fc0f8a5

Add minor changes

1a80a6c

Merge branch 'change-a-optimality' into add-multiexperiment-and-param…

948e77c

…etric-uncertainty

Add trace objective

b83ce4f

Merge branch 'add-comp-efficient-tests' into add-multiexperiment-and-…

63488f0

…parametric-uncertainty

Resolve the test failing issue for scenario_blocks does not exist.

6a53dc9

Correct docstring

4fa587c

Add trace and pseudo_trace in the plotting function for multiexpe…

422f0c2

…riment

A-optimality gives different results for rooney_biegler_multiexperime…

4bb38dc

…nt for `optimize_experiments()` and `compute_FIM()`

Add new directory for multiexperiment protype examples

974598b

Solve the error FIM not positive definite when prior_FIM is passed

c38875b

Solve rooney_biegler_multiexperiment.py

62f28ed

Add FIM prior in multiexperiment

42b8744

Add multiexperiment_prototype.ipynb that gives the same result from…

26ea0a6

… both of sensitivity and optimize_experiments()

When no prior was used, D-opt and trace FIM gave he same result, but …

cb19d00

…A-opt gave different result for grid and optimization

Solve 2 experiment case with same result from both grid and optimization

e56276a

Merge branch 'Pyomo:main' into add-multiexperiment

0f7cc9e

Add local_name of sym_break_cons

03bfb89

Add log10 codiition number

e9d33b7

Add lhs iniatialization method and test

0833295

Add tests for optimize_experiments(). All tests passed

ca0a4dd

Remove cherry-picked example artifacts from DOE test changes

02aa81a

adowling2 approved these changes May 28, 2026

View reviewed changes

Comment thread pyomo/contrib/doe/tests/test_doe_build.py Outdated

Comment thread pyomo/contrib/doe/tests/test_doe_build.py

Comment thread pyomo/contrib/doe/tests/test_doe_build.py

smondal13 added 2 commits May 27, 2026 22:49

Merge branch 'main' into add-multiexperiment

dc1bfc4

Address Alex's review comments and resolve conflict

0980905

Merge branch 'main' into add-multiexperiment

a20061b

mrmundt added the AT: PRE-TEST INSPECTED label May 29, 2026

pyomo-autotest removed the AT: PRE-TEST INSPECTED label May 29, 2026

Merge branch 'main' into add-multiexperiment

8346a48

smondal13 moved this from Ready for design review to Ready for final review in ParmEst & Pyomo.DoE Development May 31, 2026

Merge branch 'main' into add-multiexperiment

e0a9d5f

blnicho reviewed Jun 3, 2026

View reviewed changes

smondal13 added 2 commits June 3, 2026 07:46

Implement Reviewer comments. Refactor RooneyBieglerMultiExperiment do…

fd51a4c

…cstring for clarity and update test cases to use make_ipopt_solver() directly

Update test case for optimization experiment initialization solves to…

9f69e27

… reflect deterministic behavior

blnicho reviewed Jun 3, 2026

View reviewed changes

smondal13 and others added 12 commits June 4, 2026 12:50

Merge branch 'Pyomo:main' into add-multiexperiment

18293e3

Apply suggestions from code review

9faf46b

Co-authored-by: Bethany Nicholson <blnicho@users.noreply.github.com>

Refactor solver initialization in TestDoEErrorsRequiringSolver to use…

4daeb23

… make_ipopt_solver directly

Refactor TestOptimizeExperimentsAlgorithm methods to be static

2b55cf4

Apply suggestion from @blnicho

85c83cd

Co-authored-by: Bethany Nicholson <blnicho@users.noreply.github.com>

refactor(TestOptimizeExperimentsAlgorithm): enhance Cholesky regulari…

75590e5

…zation test with deterministic FIM input

simplify LHS combo test by using fixed expected points

c4ff2a1

simplify LHS sampling by using fixed expected points

a968fb3

refactor(TestOptimizeExperimentsAlgorithm): simplify LHS combo scorin…

dde1951

…g test with fixed expected points

Hard-code the tests

cbfc1f3

Merge branch 'main' into add-multiexperiment

62b6bc7

		# The exact number of initialization solves is implementation-dependent,
		# but they must all occur before the one final main-solver call.

Conversation

smondal13 commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes # .

Summary/Motivation:

Changes proposed in this PR:

Note:

Remove before merging

Legal Acknowledgement

Uh oh!

adowling2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smondal13 commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smondal13 commented May 28, 2026

Uh oh!

blnicho commented May 28, 2026

Uh oh!

smondal13 commented May 28, 2026

Uh oh!

smondal13 commented Jun 1, 2026

Uh oh!

blnicho left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blnicho left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

smondal13 commented Mar 2, 2026 •

edited

Loading

smondal13 commented May 28, 2026 •

edited

Loading