Method Selection Wizard

The Method Selection Wizard is accessed by pressing the Choose button in the Probabilistic Analysis Tool. The Method Selection Wizard takes the information already provided by the problem definition and then asks a series of questions to help select a probabilistic method that is appropriate for the problem you are trying to solve. If the answer to a question can be drawn from the problem definition, it will be skipped by the wizard. If you have already answered all of the wizard questions once, you will immediately be presented with the Method Comparison page. Once the information required on a particular screen is complete, use the Next > button to navigate to the next screen. The < Back button provides the ability to go to the previous screen and edit some information if needed. The Cancel button can be used to close the Method Selection Wizard at any given time, which will then bring the Probabilistic Analysis Tool in focus.

How to Use the Probabilistic Analysis Tool

The method selection process is generally guided by the accuracy of the method and how efficient it is. Monte Carlo is the best method as it can attain most accurate solution but it will require very high number of model evaluations. If each model evaluation takes very long, the efficient analytical methods can be used. NESSUS EGRA method combines surrogate based approach and sampling techniques and is known to come up with solutions which are highly accurate amongst the analytical methods. Thus NESSUS EGRA method should be used if Monte Carlo cannot be afforded. The NESSUS EGRA method may require slightly more model evaluations than other analytical methods thus if pressed for time NESSUS AMV+ method can be used. NESSUS AMV+ and NESSUS FORM methods have been observed to provide similar accuracy on results but NESSUS FORM requires slightly more model evaluations. NESSUS MV method is the least accurate and most efficient but should be generally used only to get an idea of the solution and should never be relied upon for complex response functions. NESSUS MV can be used reliably only for linear response functions. NESSUS AMV+ - IS and FORM - IS methods seek to improve the efficiency of probability estimates by focusing the samples in the important region of the design space, where failure is more likely to occur. Particularly for small probabilities of failure, basic Monte Carlo sampling can require a very large number of samples, because the vast majority of the samples will lie in the safe region. NESSUS AMV+ - MAIS and FORM - MAIS are variations of importance sampling that allow for the use of multiple sampling densities, making them better suited for cases where multiple sections of the limit state are highly probable. MAIS methods require the MPP as a starting point. MAIS efficiency over Monte Carlo increases as the probability of failure decreases.

Welcome Page

The welcome page summarizes the problem definition (responses and design variables) as they have been specified in the Probabilistic Analysis Tool GUI. The number of responses and the number of design variables will be listed from the Probabilistic Analysis Tool form.

How to Use the Probabilistic Analysis Tool

Number of Responses Page

If the responses have not been already specified, the wizard will ask if the problem definition has more than one response specified. The analytical methods work with one response at a time, and thus when using these methods the number of evaluations will increase proportional to the number of responses. Generally, analytical methods require around 100 evaluations for dealing with one response. If there are more responses, that number will increase but will still be lot smaller than the sampling based methods. It is not necessary for the Monte Carlo method to run individually for each response, as it just generates random samples from a joint distribution of the design variables.

Model Speed Page

The wizard asks for an estimation of how long your model takes to run. If your model takes a long time to run, then methods that minimize the number of model executions will be given a higher ranking. The analytical methods generally require very few model evaluations to come up with a probability, and thus should be used for long running models. Methods like Monte Carlo could require days of analysis to compute probabilities for long running models, as they generally require a very large number of model evaluations.

Failed Runs in Model Page

The wizard asks for whether or not it is possible for your model to have failed runs. Some models have the possibility of throwing errors (or failing to converge) for some input values. Not all methods are capable of handling these run failures. The Monte Carlo method can handle failed runs, as it simply ignores those runs in the calculation of probabilities. The analytical methods do not support failed runs as they try to converge on a Most Probable Point (MPP) using gradient optimization algorithms. Suppose there is failure in a model if possible the model should return an error code or response value which would lie in the failure zone. Suppose in an analysis failure is represented by stress being greater than 200 MPa. So any failure occurring in the underlying model should return a value greater than 200 MPa, thus it lies in the failure zone. This method would thus mean that the failure region would represent both model failures and design failures. The approach still would not work for MPP based methods but may work for EGRA. The results in this case should be used with caution.

Response Value specification Page

The wizard tries to gather information about where the specified upper or lower bound lies on the distribution of the response variable. This can be a guess, as it may not always be known. If the response value is in the tail of the distribution (i.e. it is in a very low probability zone), the 'Beyond 3 standard deviations of the mean' should be selected. If the response value is expected to lie beyond one standard deviation of the mean in either direction, the second button should be selected. If the response value is expected to lie in the middle 60-70% of the distribution, the 'Within 1 standard deviation' option should be selected. The default option here is 'Don't Know'. The response to this question is influential only for Monte Carlo. Monte Carlo may easily require greater than 100,000 model evaluations for predicting rare event probabilities with acceptable error. The number of required function evaluations decreases as the response value gets closer to the mean. The analytical methods are generally insensitive to the value of the response variable, and generally converge in a couple hundred evaluations.

Smooth and Non-Smooth Responses Page

The wizard will ask whether or not the response variables vary smoothly as a function of design variables. Methods that use derivatives (the slope of the response function) to determine search directions for computing the MPP may not be reliable for non-smooth response functions. In the case of MPP based methods, non-smooth response functions could lead to longer convergence times. Because it does not need to search for an MPP, the Monte Carlo method is capable of handling non-smooth responses.

Method Comparison Page

All of the methods installed are listed in the Method table, and they are rated based on their capabilities and the answers provided to the wizard. By default, only the overall rating for the method is presented. Selecting any of the methods displays a small description of the method in the window in lower part of the screen. The description also contains the approximate number of evaluations required by the method. The possible ratings are:

– Suitable
– Possibly Suitable
– Unsuitable
– Unknown

Thus, as per the above image, the Monte Carlo method is Possibly Suitable but the rest of the methods are Suitable. In order to help you choose between Suitable methods we are providing some subjective information about estimated accuracy and also estimated function evaluations required through Estimated Accuracy and Estimated Function Evaluations Range (min to max) columns. For more information about why a method has a particular rating, check the Show rating details checkbox. This displays the following columns:

Rating - This is the same rating as was displayed in the condensed table
Model Speed - Set based on the reported speed of your model, the location of the response value in the distribution, and the number of function evaluations the method typically requires. If the method requires a number of evaluations and your model is slow, the method will be shown as Possibly Suitable. Tooltips are shown for the Model Speed column to let you know the approximate number of model evaluations. (see Model Speed Page, Response Value specification Page)
Multiple Responses - Set based on whether you have specified multiple responses and whether or not the method directly supports them. If this column is shown as Possibly Suitable, refer the tooltip message. For analytical methods, the wizard lets the user know that each response will be dealt with individually. Even in this case, however, the number of model evaluations are usually extremely small compared to the Monte Carlo method. (see Number of Responses Page)
Smooth Responses - Set based on whether or not your responses are smooth and whether or not the method works well with non-smooth responses (see Smooth and Non-Smooth Responses Page)
Failed Runs - Set based on whether or not your model might have failed runs and whether or not the method can handle failed runs (see Failed Runs in Model Page)
License - Set based on whether or not a license is available for the method

For the Possibly Suitable, Unsuitable and Unknown ratings, tooltips are available for information about why that rating was given.

When a method is selected in the table, a brief description of the method is displayed below the Method table and the method's detailed help file can be opened by clicking on the Open method help in external viewer… link.

When Finish is clicked, the method you have selected in the Method table will be applied to the Probabilistic Analysis Tool.