Exercises — Lesson 11: Self-Adjoint Operators and the Spectral Theorem
Exercise 11.1. [Hand] Find the spectral decomposition \(Q \Lambda Q^\mathsf{T}\) of \(\left(\begin{smallmatrix}0&2\\2&3\end{smallmatrix}\right)\), and write the quadratic form \(q(x) = 4x_1x_2 + 3x_2^2\) as a combination of squares in the eigen-coordinates. Is the matrix PSD?
Solution. \(A = \begin{pmatrix} 0 & 2 \\ 2 & 3 \end{pmatrix}\).
\[ \begin{aligned} (-\lambda)(3 - \lambda) - 4 &= 0 \\[6pt] -3\lambda + \lambda^2 - 4 &= 0 \\[6pt] \lambda^2 - 3\lambda - 4 &= 0 \\[6pt] (\lambda - 4)(\lambda + 1) &= 0 \end{aligned} \]
\[ \begin{aligned} \lambda_1 &= 4 &&\Longleftrightarrow\quad v_1 = \begin{pmatrix} 1/2 \\ 1 \end{pmatrix} &&\Longrightarrow\quad \|v_1\| = \sqrt{\tfrac{1}{4} + 1} = \sqrt{\tfrac{5}{4}} = \tfrac{\sqrt{5}}{2} \\[6pt] \lambda_2 &= -1 &&\Longleftrightarrow\quad v_2 = \begin{pmatrix} -2 \\ 1 \end{pmatrix} &&\Longrightarrow\quad \|v_2\| = \sqrt{4 + 1} = \sqrt{5} \end{aligned} \]
Eigenvectors, by row reduction of \((A - \lambda I)v = 0\):
\[ \left[\begin{array}{cc|c} 1 & 2 & 0 \\ 2 & 4 & 0 \end{array}\right] \;\longrightarrow\; \left[\begin{array}{cc|c} 1 & 2 & 0 \\ 0 & 0 & 0 \end{array}\right] \qquad\Longrightarrow\qquad x_1 = -2x_2 \]
\[ \left[\begin{array}{cc|c} -4 & 2 & 0 \\ 2 & -1 & 0 \end{array}\right] \;\longrightarrow\; \left[\begin{array}{cc|c} 1 & -\tfrac{1}{2} & 0 \\ 0 & 0 & 0 \end{array}\right] \qquad\Longrightarrow\qquad x_1 = \tfrac{1}{2}x_2 \]
Normalizing the eigenvectors gives the orthonormal columns of \(Q\):
\[ Q = \begin{pmatrix} 1/\sqrt{5} & -2/\sqrt{5} \\ 2/\sqrt{5} & 1/\sqrt{5} \end{pmatrix}, \qquad Q^* = \begin{pmatrix} 1/\sqrt{5} & 2/\sqrt{5} \\ -2/\sqrt{5} & 1/\sqrt{5} \end{pmatrix}, \qquad \Lambda = \begin{pmatrix} 4 & 0 \\ 0 & -1 \end{pmatrix} \]
The matrix is not PSD, since it has a negative eigenvalue.
Writing \(q\) as a combination of squares in eigen-coordinates:
\[ q\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = 4x_1x_2 + 3x_2^2 \]
\[ \boxed{\; q\begin{pmatrix} y_1 \\ y_2 \end{pmatrix} = 4y_1^2 - y_2^2 \;} \qquad \text{where}\quad \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} = Q^* \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}, \quad x = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}, \quad y = \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} \]
Exercise 11.2. [Hand] The Hermitian matrix \(H = \left(\begin{smallmatrix}1&i\\-i&1\end{smallmatrix}\right)\) (note \(H^* = H\)): find its (real!) eigenvalues and a pair of orthogonal eigenvectors in \(\mathbb{C}^2\); verify orthogonality with the complex inner product.
Solution.
\[ H = \begin{pmatrix} 1 & i \\ -i & 1 \end{pmatrix}, \qquad H^* = \begin{pmatrix} 1 & i \\ -i & 1 \end{pmatrix} \]
\[ \begin{aligned} (1 - \lambda)^2 - 1 &= 0 \\[6pt] (1 - \lambda)^2 &= 1 \end{aligned} \qquad\qquad \begin{aligned} \lambda_1 &= 2 \\[6pt] \lambda_2 &= 0 \end{aligned} \]
Eigenvectors, by row reduction of \((H - \lambda I)v = 0\):
\[ \left[\begin{array}{cc|c} -1 & i & 0 \\ -i & -1 & 0 \end{array}\right] \;\longrightarrow\; \left[\begin{array}{cc|c} 1 & -i & 0 \\ 0 & 0 & 0 \end{array}\right] \qquad\Longrightarrow\qquad x_1 = i x_2 \]
\[ \left[\begin{array}{cc|c} 1 & i & 0 \\ -i & 1 & 0 \end{array}\right] \;\longrightarrow\; \left[\begin{array}{cc|c} 1 & i & 0 \\ 0 & 0 & 0 \end{array}\right] \qquad\Longrightarrow\qquad x_1 = -i x_2 \]
\[ \boxed{\; \begin{aligned} \lambda_1 &= 2 &&\Longleftrightarrow\quad v_1 = \begin{pmatrix} i \\ 1 \end{pmatrix} \\[6pt] \lambda_2 &= 0 &&\Longleftrightarrow\quad v_2 = \begin{pmatrix} -i \\ 1 \end{pmatrix} \end{aligned} \;} \]
Reader to verify \(\langle v_1, v_2 \rangle = 0\) with the complex inner product.
Exercise 11.3. [Proof, \(\star\)] Prove: a self-adjoint matrix with all eigenvalues in \(\{0,1\}\) is exactly an orthogonal projection \(P_U\) onto the span \(U\) of its \(\lambda = 1\) eigenvectors. (Use the spectral theorem to write \(A = \sum_j \lambda_j e_j e_j^*\), i.e. \(Ax = \sum_j \lambda_j \langle x, e_j \rangle e_j\), and compare with the projection formula.)
Proof. Let \(x \in \mathbb{F}^n\). Let \(A\) be such a matrix and \(Q \Lambda Q^*\) be its spectral decomposition, and \(e_1, \dots, e_n\) be an orthonormal basis. Thus
\[ A = \sum_{k=1}^{n} \lambda_k e_k e_k^* \qquad\text{and}\qquad Ax = \sum_{k=1}^{n} \lambda_k \langle x, e_k \rangle e_k. \]
Since \(\lambda_k \in \{0,1\}\) for all \(\lambda_k\), then
\[ Ax = \sum_{j} \langle x, e_j \rangle e_j, \]
where the \(e_j\) are all the eigenvectors whose corresponding \(\lambda_j\) is \(1\).
Note \(U = \ker(A - I) = \operatorname{span}\{e_k : \lambda_k = 1\}\). Thus
\[ P_U x = \sum_{\lambda_k = 1} \langle x, e_k \rangle e_k. \]
Therefore, \(Ax = P_U x\) for all \(x\).
\(\square\)
Exercise 11.4. [Proof] Show that for any (not necessarily self-adjoint) real \(A\), the matrix \(S = \tfrac12(A + A^\mathsf{T})\) is symmetric and \(x^\mathsf{T} A x = x^\mathsf{T} S x\) for all \(x\) — so quadratic forms lose nothing by assuming symmetry.
Proof. Note \(A\) must be a square \(n \times n\) matrix for the addition to be defined. Each entry \((S)_{i,j}\) is \(\tfrac{1}{2}\big( (A)_{i,j} + (A^\mathsf{T})_{i,j} \big)\), and each entry \((S)_{j,i} = \tfrac{1}{2}\big( (A)_{j,i} + (A^\mathsf{T})_{j,i} \big)\).
But note \((A)_{i,j} = (A^\mathsf{T})_{j,i}\) and \((A)_{j,i} = (A^\mathsf{T})_{i,j}\), so that means \((S)_{i,j} = (S)_{j,i}\). Thus \(S\) is a symmetric matrix.
\[ \begin{aligned} x^\mathsf{T} S x &= \langle x, Sx \rangle \\[6pt] &= \left\langle x, \tfrac{1}{2}Ax + \tfrac{1}{2}A^\mathsf{T}x \right\rangle \\[6pt] &= \left\langle x, \tfrac{1}{2}Ax \right\rangle + \left\langle x, \tfrac{1}{2}A^\mathsf{T}x \right\rangle \\[6pt] &= \tfrac{1}{2}\langle x, Ax \rangle + \tfrac{1}{2}\left\langle x, A^\mathsf{T}x \right\rangle \\[6pt] &= \tfrac{1}{2}\langle x, Ax \rangle + \tfrac{1}{2}\langle Ax, x \rangle \\[6pt] &= \tfrac{1}{2}\langle x, Ax \rangle + \tfrac{1}{2}\langle x, Ax \rangle \\[6pt] &= \langle x, Ax \rangle \\[6pt] &= x^\mathsf{T} A x \end{aligned} \]
\(\square\)