Sensitivity Analysis

Modeling and Uncertainty

Mathematical models are representations of variable relationships based on effect statistics, and are used for their predictive ability. As part of these models, there are estimates of values and unknowns because the data have not yet been collected. Once the data have been collected, the justification of the model (i.e., success or failure) depends on its ability to correctly predict the variability of observed values in the dependent variable. They are not designed to better understand the variables in terms of expectations and preconceived notions; rather, models are supposed to be used to test the set of variables and assumptions at part of the overall model. From the example in the previous section, years spent in school (X) is thought to have a direct influence on annual income (Y), so the purpose of the model should be to represent that relationship no matter what data are collected. If the model is accurate in predicting the change in the dependent variable (Y) for a data set, then it should be able to predict the relationship given any data set.

Sensitivity analysis is generally carried out with another statistical test known as uncertainty reduction. The goal of sensitivity analysis is to accurately determine how sensitive the observation of the dependent variable is (or how much the values change when the sample parameters change). Uncertainty analysis essentially accomplishes the same goal as sensitivity analysis; however, sensitivity analysis is putting a value on how susceptible to change the output is when there is a change in input, and uncertainty analysis is generating the range of possible observations or values in the output based on the given set of input. For instance, the input from the previous example includes the parameters that define years spent in school, along with output variables that explain how people earn annual income. Uncertainty analysis could be used to generate the actual range of possible observed values of the output (i.e., how much money participants made). A sensitivity analysis of the same input and output produces values that could illustrate how susceptible annual income is to changes in the parameters of years spent in school. Once the most sensitive parameters have been discovered, it would prove beneficial for the researcher to further investigate exactly which parameters are most useful to the prediction of the output (i.e., annual income). Sensitivity and uncertainty analysis should be carried out concomitantly to evaluate parameter estimates, better understand the model’s power to predict the change in the output based on the input, and generate actual values for unknown portions of models.

...