Week 1 — Linear Maps, Matrices, and Eigenstructure

Overview

Almost everything in the later weeks — least squares, the SVD, covariance matrices, Markov chains, convolution operators, the Hessian of a convex function — is a linear map wearing a costume. This week pins down the distinction between a linear map and the matrix that represents it in a chosen basis, then develops the eigenstructure of an operator: the directions it merely scales, the upper-triangular forms it can always be put into over \(\mathbb{C}\), and the conditions under which it is diagonalizable. Getting this right is what lets you read a matrix as a transformation rather than a grid of numbers.

Readings

Axler 3.C — Matrices. The matrix \(\mathcal{M}(T)\) of a linear map relative to chosen bases; matrix of a composition is the product; the space \(F^{m,n}\).
Axler 5.B — Eigenvectors and upper-triangular matrices. Existence of eigenvalues over \(\mathbb{C}\); every operator has an upper-triangular matrix with respect to some basis.
Axler 5.C — Eigenspaces and diagonal matrices. Conditions equivalent to diagonalizability; eigenspaces form a direct sum.

Key Concepts

Linear maps vs. matrices

A linear map \(T \in \mathcal{L}(V,W)\) satisfies \(T(\alpha u + \beta v) = \alpha T u + \beta T v\). Once bases \(v_1,\dots,v_n\) of \(V\) and \(w_1,\dots,w_m\) of \(W\) are fixed, \(T\) is encoded by the matrix \(\mathcal{M}(T) = A\) with entries \(A_{k,j}\) defined by

\[T v_j = \sum_{k=1}^m A_{k,j}\, w_k.\]

The \(j\)-th column of \(A\) is the coordinate vector of \(T v_j\). Composition corresponds to the product: \(\mathcal{M}(ST) = \mathcal{M}(S)\,\mathcal{M}(T)\). The matrix depends on the bases; the map does not. Change of basis conjugates the matrix: if \(A\) and \(A'\) represent the same operator in two bases related by the invertible \(P\), then \(A' = P^{-1} A P\).

Eigenvalues, eigenvectors, invariant subspaces

A subspace \(U \subseteq V\) is invariant under \(T\) if \(T U \subseteq U\). The one-dimensional invariant subspaces are spanned by eigenvectors: \(\lambda \in F\) is an eigenvalue if there is a nonzero \(v\) with

\[T v = \lambda v \quad\Longleftrightarrow\quad \ker(T - \lambda I) \neq \{0\}.\]

Eigenvectors for distinct eigenvalues are linearly independent. Over \(\mathbb{C}\), every operator on a nonzero finite-dimensional space has at least one eigenvalue — the fact that ultimately powers the spectral theorem in Week 2.

Upper-triangular matrices

\(T\) has an upper-triangular matrix with respect to a basis \(v_1,\dots,v_n\) iff each \(T v_j \in \operatorname{span}(v_1,\dots,v_j)\), equivalently each \(\operatorname{span}(v_1,\dots,v_j)\) is invariant. Over \(\mathbb{C}\) such a basis always exists. When the matrix is upper-triangular, its diagonal entries are exactly the eigenvalues, and \(T\) is invertible iff no diagonal entry is \(0\).

Diagonalizability

\(T\) is diagonalizable iff \(V\) has a basis of eigenvectors, iff \(V\) decomposes as a direct sum of eigenspaces:

\[V = E(\lambda_1, T) \oplus \cdots \oplus E(\lambda_k, T),\]

where \(E(\lambda, T) = \ker(T - \lambda I)\). A sufficient (not necessary) condition: if \(T\) has \(\dim V\) distinct eigenvalues, it is diagonalizable. Diagonalization is the statement \(A = P D P^{-1}\) with \(D\) diagonal; it makes powers \(A^n = P D^n P^{-1}\) and matrix functions trivial to compute — the engine behind Markov-chain steady states (Week 7) and linear dynamical systems.

Connections

Forward: Week 2 specializes to operators on inner product spaces, where the spectral theorem guarantees an orthonormal eigenbasis and the SVD handles the non-square, non-diagonalizable case. Week 4 turns “find the eigenvalues” into iterative numerical algorithms.
Across courses: Diagonalization and eigenstructure underlie PCA and covariance analysis (Course 1), stability of linear systems (Courses 2, 4), and the spectral view of graphs and embeddings (Course 3).