Eigendecomposition

Notations and Definition

There are several ways to define eigenvectors and eigenvalues. The most common approach defines an eigenvector of the matrix A as a vector u that satisfies the following equation:

When rewritten, the equation becomes

where λ is a scalar called the eigenvalue associated to the eigenvector.

In a similar manner, a vector u is an eigenvector of a matrix A if the length of the vector (but not its direction) is changed when it is multiplied by A. For example, the matrix

has the eigenvectors

[Page 305]

and

We can verify (as illustrated in Figure 1) that only the lengths of u1 and u2 are changed when one of these two vectors is multiplied by the matrix A:

and

For most applications, we normalize the eigenvectors (i.e., transform them so that their length is equal to one):

For the previous example, we obtain

We can check that

and

Traditionally, we put together the set of eigenvectors of A in a matrix denoted U. Each column of U is an eigenvector of A. The eigenvalues are stored in a diagonal matrix (denoted Λ), where the diagonal elements give the eigenvalues (and all the other values are zeros). The first equation can be rewritten as follows:

or also as

[Page 306]

For the previous example, we obtain

It is important to note that not all matrices have eigenvalues. For example, the matrix does not have eigenvalues. Even when a matrix has eigenvalues and eigenvectors, the computation of the eigenvectors and eigenvalues of a matrix requires a large number of computations and is therefore better performed by computers.

Digression: An Infinity of Eigenvectors for One Eigenvalue

It is only through a slight abuse of language that we can talk about the eigenvector associated with one given eigenvalue. Strictly speaking, there is an infinity of eigenvectors associated with each eigenvalue of a matrix. Because any scalar multiple of an eigenvector is still an eigenvector, there is, in fact, an (infinite) family of eigenvectors for each eigenvalue, but they are all proportional to each other. For example,

...