Least Squares, Methods of

The least-squares method (LSM) is widely used to find or estimate the numerical values of the parameters to fit a function to a set of data and to characterize the statistical properties of estimates. It is probably the most popular technique in statistics for several reasons. First, most common estimators can be cast within this framework. For example, the mean of a distribution is the value that minimizes the sum of squared deviations of the scores. Second, using squares makes LSM mathematically very tractable because the Pythagorean theorem indicates that, when the error is independent of an estimated quantity, one can add the squared error and the squared estimated quantity. Third, the mathematical tools and algorithms involved in LSM (derivatives, eigendecomposition, and singular value decomposition) ...