Ordinary Least Squares

Least Square Approach to Analysis

The least square approach provides a set of assumptions for the estimation of statistical parameters, like the estimation of a mean. The mean, also known as expected value, constitutes the value where the sum of squared error, Ʃ(X − M)2, is the smallest for that set of data, where X is the raw score and M is the arithmetic mean, Ʃ(X/N), where N is the number of scores in the analysis. The impact of this set of assumptions and mathematical operations provides the basis for most of the assumptions and practices employed in statistical analysis.

For example, the estimations involved in the independent groups t-test involve a term in the denominator that is the weighted average of the variances:

$$t=\frac{M_1-M_2}{\frac{\left(s_1^2\left[n_1-1\right]+s_2^2\left[n_2-1\right]\right)/\left(n_1+n_2-2\right)}{\left(n_1+n_2\right)/\left(n_1\times n_2\right)}},$$

[Page 1151]where s indicates the variance for each variable and each sample size is designated by the term n, the means for each group are indicated by M (M1 and M2) and the variance or sum of squared error is represented by the term s ( $s_1^2$ and $s_2^2,$ one calculated for each group). The sample size for each of the two groups is represented by n (n1 and n2). The estimate essentially examines the difference between groups, indicated in the numerator, versus the within-group variability, weighted by sample size.

The same can be applied to the correlation coefficient implied by the formula used by Pearson. The formula for the correlation coefficient provides for a numerator that generates the covariance between two variables compared with the level of variability for both

...