Parameter Mean Squared Error

Calculation of the MSE

If an estimation procedure is repeated a number of times, one should be able to calculate the average squared deviation of an estimator from the expected value of a given parameter across all replications. Let x denote the expected value of a parameter and xi denote the estimator of x from the ith, i = 1, … , T, replication. Then, the MSE for the estimator can be written as

$$\text{MSE}= \frac{1}{T} {\displaystyle \sum }_{i=1}^T{\left({\widehat{x}}_i-x\right)}^2.$$

At times, one may need to estimate a set of parameters. For example, in item response theory calibration, the ability parameters for a group of [Page 1204]examinees need to be estimated. Let X be a vector of the expected values of N parameters and Xi be a vector of estimators for the ith, i = 1, … , T, replication. Then, the MSE for the estimators can be written as

$$\text{MSE}= \frac{1}{T}\frac{1}{N} {\displaystyle \sum }_{i=1}^T{\displaystyle \sum }_{j=1}^N{({\widehat{x}}_{ij}- x_j)}^2,$$

where xij and xj are the jth elements in Xi and X, respectively.

There are also occasions in which researchers are interested in the accuracy of the output of a function with respect to its expected value. For example, a psychometrician attempts to examine the accuracy of the equating results obtained by using the estimated linking coefficients and related equating functions. Let fs be the “true” function of s, which is built upon the expected values for all related coefficients. Let fis denote the estimated function of s, in which all coefficients are estimators from the ith, i = 1, … , T, estimation. Then, the MSE for the estimated function can be expressed as

$$\text{MSE}= \frac{1}{T}{\displaystyle \sum }_{i=1}^T{\left[{\widehat{f}}_i\left(s\right)-f\left(s\right)\right]}^2.$$

In a study using MCMC simulation methods, the MSE is typically viewed as an index that summarizes the total errors occurring during a given statistical process (e.g., estimation, equating, and scaling). In fact, based on the types of source, there exist two types of error: systematic and random. The former relates to constant inaccuracy and is also known as bias, whereas the latter is unpredictable and occurs only by chance. Correspondingly, the MSE can be further broken down into two components to represent the systematic and the random error, respectively. Equation 1, for instance, can be rewritten as

$$\begin{array}{l}\text{MSE}= \frac{1}{T} {\displaystyle \sum }_{i=1}^T{\left({\widehat{x}}_i-x\right)}^2\\ ={\left({\overline{x}}_i-x\right)}^2+ \frac{1}{T}{\displaystyle \sum }_{i=1}^T({\widehat{x}}_i-{\overline{x}}_i{)}^2, \end{array}$$

where

$${\overline{x}}_i=\frac{1}{T}{\displaystyle \sum }_{i=1}^T{\widehat{x}}_i.$$

As shown in Equation 4, the MSE is the sum of squared bias, (xi − x)2, and sample variance, $\frac{1}{T}{\displaystyle \sum }_{i=1}^T{({\widehat{x}}_i-{\overline{x}}_i)}^2$. The square roots of the sample variance are the empirical standard errors of the estimator, which indicate the consistency of the estimator and estimation methods.

...