Orthogonal Array

Given an upper and a lower bound for each of your input variables, Design Explorer needs to decide (1) how many times to run your analysis code, and (2) what values of the input variables to run it for each time. In Design Explorer, the goal is to minimize the number of times the analysis code has to be run while at the same time ensuring that the design space is adequately covered. A particular combination of runs is generally called an "experiment" or "experimental design".

The statistical literature provides several different of experimental designs to chose from. These experimental designs can be divided into two general types: classical experimental designs and space filling experimental designs. Roughly speaking, classical experimental designs use a number of runs in replication (repeated runs at the same settings) and "pseudo-replication" to mitigate the effects of random noise in experimental outcomes. In contrast, "space filling" experimental designs for non-random output (as from a computer simulation) take into account that, though there may be random errors in the responses (e.g., numerical error), replication will not mitigate the effects of the random error since one always obtains the same response at the same input settings. The figure below illustrates the contrast between an 18 run classical DoE design (Central Composite Design) and an 18 run space filling design (IMSE Optimal Design) in which design variables x1 and x2 and the combinations of their settings are plotted.

The experimental designs used by Design Explorer are of the "space-filling" type; specifically they are orthogonal arrays (OAs) [1][2]. OAs have an appealing space-filling property called strength such that if there are m distinct values for each variable and the OA is strength k, then every subset of k variables is a grid in the m values. These properties give one confidence that the designs are infiltrating" the design space well. The figure below illustrates an OA with strength two in three variables. Note that the design points form a grid when projected onto any of the three coordinate planes.

Depending on the number of design variables (n) in your problem, Design Explorer will automatically construct an appropriate orthogonal array. Design Explorer will attempt to construct this array with a maximum of (n+1)*(n+2)/2 or 10*n runs (whichever is larger), and with a strength of at least two.

References

[1] Owen, A.B., "Orthogonal Arrays for Computer Experiments, Integration and Visualization", Statistica Sinica, 2:439-452, 1992.

[2] Booker, A. J., "Design and Analysis of Computer Experiments," 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis & Optimization, St. Louis, MO, (Sept. 2-4, 1998) pp. 118-128. AIAA-98-4757.