Eigenvalues and Eigenvectors in MATLAB: Video Tutorial

This video tutorial reviews eigenvalues, eigenvectors and Singular Value Decomposition (SVD), and implements them in MATLAB. It is part of the Numerical Computations in MATLAB course.

The main ideas

For a square matrix $A$ of size $n \times n$, a nonzero vector $v$ is an eigenvector when multiplying it by $A$ only scales it. The scale factor $\lambda$ is the matching eigenvalue:

An $n \times n$ matrix has $n$ eigenvalues over the complex numbers when repeated values are counted with their multiplicities. It need not have $n$ independent eigenvectors. Two useful properties connect the eigenvalues to the matrix. Their product equals the determinant of $A$, and their sum equals the trace of $A$ (the sum of its diagonal elements).

Eigenvalues in this form apply to square matrices. The singular value decomposition works for a matrix of any size $m \times n$. For the real matrices used in this tutorial:

Here $U$ ($m \times m$) and $V$ ($n \times n$) are orthogonal matrices, so $UU^{T} = I$ and $VV^{T} = I$. The matrix $S$ has the same size as $A$, and its diagonal holds the singular values $\sigma_1 \ge \sigma_2 \ge \dots \ge 0$ in descending order. Their squares are the eigenvalues of $A^{T}A$, with additional zero values as required by the matrix dimensions. The nonzero eigenvalues of $A^{T}A$ and $AA^{T}$ agree. These matrices are positive semidefinite, which makes their eigenvalues nonnegative. Complex matrices use the conjugate transpose in place of the ordinary transpose.

Watch online

The two sections of this tutorial are available on YouTube and are embedded here as a playlist.

Play playlistLoads from YouTube

What each video covers

Section 1: Eigenvalues and eigenvectors with eig

Eigenvalues and Eigenvectors (YouTube (opens in a new tab)). The video uses the matrix

With one output, eig(A) returns the three eigenvalues. The video then checks the two properties numerically: the determinant of $A$ is $-48$, the same as the product of the eigenvalues, and the sum of the eigenvalues is $7$, the same as the trace.

With two outputs, eig returns a matrix V whose columns are the eigenvectors and a diagonal matrix D with the eigenvalues. The function diag extracts the eigenvalues from D.

A = [1 2 3; 4 0 0; 5 0 6];
[V, D] = eig(A);
A*V - V*D

The video checks $Av = \lambda v$ for each eigenvector, and then for all of them at once with A*V - V*D, which is almost a zero matrix (the small values are rounding errors). If the eigenvectors form an invertible matrix $V$, then $AV = VD$ gives

The first form rebuilds the matrix from its eigenstructure. The second form is the diagonalization of the matrix.

A repeated eigenvalue does not by itself prevent diagonalization. The requirement is a full set of independent eigenvectors. SVD does not need that requirement and remains available for rectangular or rank deficient matrices.

Section 2: Singular Value Decomposition with svd

Singular Value Decomposition (YouTube (opens in a new tab)). The video first explains the shapes of $U$, $S$ and $V$, and why singular values are never negative. It then computes the SVD of a matrix that is not square:

The call [U, S, V] = svd(A) returns the three factors, and U*S*V' gives back $A$ up to rounding errors. The singular values, about $9.53$ and $0.51$, match the square roots of the eigenvalues of $A^{T}A$. Part of $U$ and $S$ is not needed for this rectangular matrix, and svd(A, 'econ') returns the smaller “economy size” factors.

A = [1 2; 3 4; 5 6];
[U, S, V] = svd(A, 'econ');
diag(S)

The rest of the video shows how singular values relate to the information in a matrix. If only the largest singular value is kept, the product of the reduced factors is already close to $A$. The final example applies this idea to the cameraman.tif image (256 singular values). The image is converted to double values, and a function rebuilds it from the first $k$ singular values:

As the number of singular values grows from 1 to 10, the cameraman slowly appears, and with 50 of the 256 singular values (about one fifth) the image is clearly visible, with some noise. This is the basic idea of image compression with SVD. The video also notes the close link between SVD and Principal Component Analysis (PCA).

Applications

• Diagonalizing a matrix and studying its structure with eigenvalues and eigenvectors.
• Low rank approximation and data compression with the largest singular values.
• Dimensionality reduction, as in PCA. See the PCA video course on Yarpiz.

Check the approximation

Compare A*V with V*D for the eigenvalue example and U*S*V' with A for SVD. In image compression, increase the number of retained singular values and inspect both the reconstruction error and the storage needed for the retained factors. The number of retained singular values alone is not the compression ratio.