Week 5 — Real Analysis and Metric-Space Topology

Overview

This week supplies the rigor that the rest of the course quietly assumes. Every “the limit exists,” “the minimum is attained,” “the series converges,” and “we can swap the limit and the integral” is a theorem in real analysis or point-set topology, not a free lunch. Sequences, continuity, uniform convergence, and differentiation (Ross) give the one-dimensional machinery; metric spaces and compactness (Mendelson) generalize it to the abstract settings where optimization and ML theory actually live — where compactness is exactly what guarantees a minimizer exists. The payoff lands immediately in Weeks 6–9.

Readings

Ross Ch 1 — Introduction. The reals, the completeness axiom, supremum and infimum.
Ross Ch 2 — Sequences. Limits, limit theorems, monotone and Cauchy sequences, subsequences, \(\limsup/\liminf\).
Ross Ch 3 — Continuity. Limits of functions, continuity, uniform continuity, the intermediate and extreme value theorems.
Ross Ch 4 — Sequences and series of functions. Pointwise vs uniform convergence, power series.
Ross Ch 5 — Differentiation. Derivatives, the mean value theorem, Taylor’s theorem.
Mendelson Ch 2 — Metric spaces. Metrics, continuity, open balls and neighborhoods, limits, open and closed sets, subspaces.
Mendelson Ch 5 — Compactness. Compact sets, the Heine–Borel theorem, continuous images of compact sets.
MIR Ch 1–3 — Lebesgue measure, integration, and \(L^p\) spaces. Why Riemann integration breaks down on discontinuous and “infinite-spike” functions; outer measure, \(\sigma\)-algebras, and Lebesgue measure; the Lebesgue integral and its convergence theorems (monotone convergence, dominated convergence); \(L^p\) spaces and completeness. This is the rigorous foundation under the signals and Fourier analysis of Weeks 12–13.

Key Concepts

Completeness and suprema

The defining property of \(\mathbb{R}\) (versus \(\mathbb{Q}\)) is completeness: every nonempty set bounded above has a least upper bound, \(\sup\). This single axiom is what makes limits exist and underlies every convergence theorem below. The infimum is the dual notion.

Sequences, Cauchy, lim sup / lim inf

A sequence \((a_n)\) converges to \(L\) if for every \(\varepsilon>0\) there is \(N\) with \(|a_n - L|<\varepsilon\) for \(n\ge N\). Bounded monotone sequences converge. A sequence is Cauchy if its terms get arbitrarily close to each other; in \(\mathbb{R}\) (a complete space) Cauchy \(\Leftrightarrow\) convergent — the test for convergence that does not require knowing the limit. The

\[\limsup_{n} a_n = \lim_{n}\,\sup_{k\ge n} a_k, \qquad \liminf_{n} a_n = \lim_{n}\,\inf_{k\ge n} a_k\]

always exist (in the extended reals) and bracket all subsequential limits — the right tool when a sequence need not converge.

Continuity and its global theorems

\(f\) is continuous at \(c\) if \(\lim_{x\to c} f(x) = f(c)\); uniformly continuous if the same \(\delta\) works everywhere (a global, stronger condition). Two consequences on a closed interval are used constantly: the intermediate value theorem (a continuous function takes every value between any two it attains) and the extreme value theorem (a continuous function on a closed bounded interval attains a max and a min) — the latter is the existence guarantee behind optimization.

Uniform convergence

A sequence of functions \(f_n \to f\) pointwise if \(f_n(x)\to f(x)\) for each \(x\); uniformly if \(\sup_x |f_n(x)-f(x)|\to 0\). Uniform convergence is the strong notion: it preserves continuity, and it is what lets you exchange limits with integrals and derivatives. Power series converge uniformly on compact subsets inside their radius of convergence — the bridge to the complex power series of Week 11.

Differentiation: MVT and Taylor

The mean value theorem, \(f(b)-f(a) = f'(\xi)(b-a)\) for some \(\xi\in(a,b)\), is the workhorse that converts local derivative information into global bounds. Taylor’s theorem expands \(f\) in a polynomial plus a remainder, justifying the local quadratic models behind Newton’s method (Week 9) and the linearizations used throughout numerical analysis.

Metric spaces and topology

A metric space \((X,d)\) abstracts “distance.” Open balls \(B(x,r)\) generate the open sets; their complements are closed; continuity becomes “preimages of open sets are open.” This frees convergence and continuity from \(\mathbb{R}^n\) and lets the same theorems apply to function spaces, sequence spaces, and manifolds.

Compactness

A set is compact if every open cover has a finite subcover. In \(\mathbb{R}^n\), the Heine–Borel theorem says compact \(\Leftrightarrow\) closed and bounded. The two facts that matter downstream: a continuous image of a compact set is compact, and a continuous real function on a compact set attains its extrema (the general extreme value theorem). This is precisely why a continuous objective over a compact feasible set has a minimizer — the existence half of optimization (Weeks 8–9).

Lebesgue measure and integration (extension — MIR)

Riemann integration, the integral implicit above, is fragile: it cannot integrate the Dirichlet function \(\mathbf{1}_{\mathbb{Q}}\) (discontinuous everywhere), and the limit of Riemann-integrable functions need not be Riemann-integrable. Lebesgue integration (Axler MIR) repairs this. One assigns a measure — a countably additive size function on a \(\sigma\)-algebra of sets — extending length on \(\mathbb{R}\), then integrates measurable functions by slicing the range rather than the domain:

\[\int_{\mathbb{R}} f \, d\mu = \sup\left\{ \int_{\mathbb{R}} s \, d\mu : 0 \le s \le f,\; s \text{ simple} \right\}.\]

The payoff is the two convergence theorems that Riemann lacks — the monotone convergence theorem and the dominated convergence theorem — which give clean conditions to exchange limit and integral:

\[\lim_{n\to\infty} \int f_n \, d\mu = \int \lim_{n\to\infty} f_n \, d\mu.\]

Completing the picture, the \(L^p\) spaces \(\{f : \int |f|^p\,d\mu < \infty\}\) are complete normed spaces (Riesz–Fischer) — the Cauchy-sequence property of \(\mathbb{R}\) lifted to function spaces. These are the spaces where signals actually live and where the Fourier theory of Weeks 12–13 becomes rigorous.

Connections

Backward: builds the rigorous foundation under the calculus and limits used in Weeks 3–4 (numerical analysis) and assumed in Weeks 1–2.
Forward: Week 6’s limiting/convergence statements and Week 7’s laws of large numbers rest on this; Weeks 8–9 use continuity and compactness for existence of minimizers and the convergence analysis of algorithms; Week 11’s complex power series extend the uniform-convergence theory here.
Across courses: convergence and stability arguments for ML training and numerical methods (Courses 1, 3).