Matrix Algebra

Basic Operations

[Page 1029]A matrix transpose (written MT or M’) flips a matrix to exchange rows and columns, which can be written as (MT)ij = Mji. M is symmetric if MT = M. Matrix addition or subtraction is defined for matrices of identical size and shape, and it adds or subtracts corresponding entries; thus (A + B)ij = Aij + Bij.

A matrix can always be multiplied by a number: (kA)ij = kAij. A product of matrices, AB, is defined when the number of columns of A equals the number of rows of B and consists of sums of cross-products of rows and columns: ${(\text{AB})}_{ij}={\displaystyle {\sum }_k{\text{A}}_{ik}{\text{B}}_{kj}}$. AB need not equal BA, and they need not both be defined. An identity matrix I is a matrix such that for any M of the correct size, MI = M or IM = M. The identity matrix has a special diagonal structure: Iij = 1 for i = j and 0 for i ≠ j. While there is no matrix division in general, some square matrices A have a multiplicative inverse A−1 such that A−1 A = AA−1 = I. Others do not have inverses; this happens if the matrix is not of full rank (i.e., if some row or column can be written as a linear combination of other rows or columns). This is called collinearity, rank deficiency, or singularity.

Vectors and Matrices

Vectors are matrices with only one row or column. A column vector v is often interpreted as the coordinates of a point in r-dimensional space and can be pictured as an arrow from the origin to the point. Mv represents some operation (e.g., rotating, stretching) on this v, depending on the structure of M.

The product uTv of two vectors is a single number, the sum of their cross-products. Thus, vTv is a sum of squares and is the squared length of v’s arrow.

For a matrix M, it is often possible to find vectors v such that Mv = λv for a constant λ. Then, v and λ are called an eigenvector and eigenvalue of M. Usually, eigenvectors are scaled to be of standard length (vTv = 1) when calculating eigenvalues. Eigenvalues and eigenvectors provide important information about M and are related to methods of decomposing (factoring) matrices into products of simpler matrices. Techniques such as principal component analysis are based on performing such a decomposition of the covariance matrix of a set of observed variables in order to study their interrelationships.

...