Course 1 — Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence
One self-contained booklet: Abridged Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence — 51 lessons in 7 parts, from vectors to rate–distortion theory
This course is a single self-contained booklet, worked cover to cover:
Fifty-one lessons in seven parts, dependency-ordered: the linear algebra core; eigenstructure, the spectral theorem, the DFT, and the SVD; the probability core; random vectors, limit theorems, and stochastic processes; signals and systems; the analysis behind the transforms; and convex optimization and information theory for machine learning, signals, and sensors. The booklet is sized against the undergraduate prerequisite sets for machine learning, the Fourier transform and its applications, and probability and statistical inference. The applied layer that used to follow — the DFT in practice, filter design and finite word lengths, multirate and sigma–delta conversion, spectrum estimation, Wiener/LMS/Kalman filtering, detection and communication, images, audio, and the electronics bench — now lives as theory sections inside the Course 3 — Embedded DSP labs that exercise it; Course 3 assumes this course as a mastered prerequisite.
Lesson structure. Every lesson carries two interleaved layers — Theory (precise definitions and theorems, with complete proofs where feasible) and Concrete practice (small 2-D/3-D examples, dice, coins, three-state chains, computed by hand) — followed by Exercises tagged [Proof] or [Hand], with a star on the most valuable proofs. Every lesson opens with a Needed for tag naming the courses and labs it serves, and closes with a Deeper reading pointer into the full textbooks.
Worked exercises. The booklet carries 218 exercises across its 51 lessons — 109 [Hand], 109 [Proof], numbered lesson.exercise. My solutions are worked by hand on paper, then typeset: one page per lesson, linked under each part below and collected in the booklet’s exercise set on the Books page.
Order of work. Parts I and III first (in that order); Part II next, with Part V right after it; Part IV any time after Part III (Parts IV and V are independent); Part VI at a relaxed pace, or interleaved with the Course 3 labs; Part VII any time after Parts I, III, and V.
Note on AI use: The booklet’s teaching text is AI-drafted and owner-reviewed, in the same arrangement as the Course 2 lessons pages. The mathematics I produce is mine: the exercises are worked by hand on paper, with AI used only to typeset finished solutions. The booklet’s own exercise set and the worked problem sets from the full textbooks — the same books its Deeper reading pointers name — both accumulate on the Books page.
Part I — The Linear Algebra Core
The linear algebra core: vectors and linear combinations; dot products, norms, and orthogonality; span, independence, basis, and dimension; matrices as linear maps; null space and rank; orthogonal projections and least squares; gradients.
Worked exercises: Lesson 1, Lesson 2, Lesson 3, Lesson 4, Lesson 5, Lesson 6, Lesson 7.
- Lesson 1 — Vectors and Linear Combinations
- Lesson 2 — Dot Products, Norms, and Orthogonality
- Lesson 3 — Span, Linear Independence, Basis, Dimension
- Lesson 4 — Matrices and Linear Maps
- Lesson 5 — Null Space, Rank, and Solving \(Ax=b\)
- Lesson 6 — Orthogonal Projections and Least Squares
- Lesson 7 — Gradients for Machine Learning
Part II — Eigenstructure, the Spectral Theorem, the DFT, and the SVD
Complex vector spaces and complex exponentials; eigenvalues, eigenvectors, and diagonalization; self-adjoint operators and the spectral theorem; unitary matrices and the discrete Fourier transform; the singular value decomposition; determinants and trace.
Worked exercises: Lesson 8, Lesson 9, Lesson 10, Lesson 11, Lesson 12, Lesson 13.
- Lesson 8 — Complex Vectors and Complex Exponentials
- Lesson 9 — Eigenvalues, Eigenvectors, and Diagonalization
- Lesson 10 — Self-Adjoint Operators and the Spectral Theorem
- Lesson 11 — Unitary Matrices and the Discrete Fourier Transform
- Lesson 12 — The Singular Value Decomposition
- Lesson 13 — Determinants and Trace
Part III — The Probability Core
Probability models; counting — permutations, combinations, partitions; conditioning and Bayes’ rule; discrete and continuous random variables; expectation and variance; joint distributions and conditioning; the normal distribution.
Worked exercises: Lesson 14.
- Lesson 14 — Probability Models, Counting, Conditioning, and Bayes’ Rule
- Lesson 15 — Discrete Random Variables and Expectation
- Lesson 16 — Multiple Random Variables and Conditioning
- Lesson 17 — Continuous Random Variables and the Normal
Part IV — Random Vectors, Limit Theorems, and Stochastic Processes
Derived distributions, covariance, and correlation; conditional expectation and least-mean-squares estimation; random vectors and Gaussian vectors; limit theorems; the Bernoulli and Poisson processes; Markov chains.
- Lesson 18 — Derived Distributions, Covariance, and Correlation
- Lesson 19 — Conditional Expectation and Least Mean Squares
- Lesson 20 — Random Vectors and Gaussian Vectors
- Lesson 21 — Limit Theorems
- Lesson 22 — The Bernoulli and Poisson Processes
- Lesson 23 — Markov Chains
Part V — The Signals-and-Systems Core
Signals and system properties; LTI systems and convolution; Fourier series; the continuous- and discrete-time Fourier transforms; frequency response and filtering; sampling and aliasing; Laplace and \(z\)-transforms.
- Lesson 24 — Signals and Systems
- Lesson 25 — LTI Systems and Convolution
- Lesson 26 — Fourier Series
- Lesson 27 — The Continuous-Time Fourier Transform
- Lesson 28 — The Discrete-Time Fourier Transform and Frequency Response
- Lesson 29 — Sampling
- Lesson 30 — Laplace and \(z\)-Transforms, a Working Minimum
Part VI — The Analysis Behind the Transforms
Sequences, series, and interchange of limits; metric spaces, continuity, and compactness; differentiation and the calculus theorems — the mean value theorem, L’Hôpital, Taylor — with the \(n\)-dimensional upgrade; the Riemann integral and the fundamental theorem of calculus; Lebesgue measure and integration and the swap theorems; Banach and Hilbert spaces, \(L^2\), and orthonormal bases; complex analysis — analytic functions, power series, and contour integrals, then zeros, poles, residues, and ROCs; the Fourier, Laplace, and \(z\)-transforms as operators, with the property tables proved; distributions and \(\delta\); dual spaces, Riesz representation, and functional derivatives; numerical linear algebra — conditioning, stability, factorizations, and iteration; transforms of distributions — moment generating and characteristic functions, the Chernoff bound, and the central limit theorem with proof.
- Lesson 31 — Limits Done Right: Sequences, Series, and When Swapping Is Legal
- Lesson 32 — Metric Spaces, Continuity, and Compactness
- Lesson 33 — Differentiation: The Calculus Theorems, Done Right
- Lesson 34 — The Riemann Integral and the Fundamental Theorem
- Lesson 35 — Integration Done Right: Lebesgue, Almost Everywhere, and the Swap Theorems
- Lesson 36 — Signal Space: Banach and Hilbert Spaces, \(L^2\), and Orthonormal Bases
- Lesson 37 — Complex Analysis I: Analytic Functions, Power Series, and Contour Integrals
- Lesson 38 — Complex Analysis II: Zeros, Poles, Residues, and Why ROCs Work
- Lesson 39 — The Fourier, Laplace, and \(z\)-Transforms as Operators: The Property Toolbox, Proved
- Lesson 40 — Distributions: Making \(\delta\) Honest
- Lesson 41 — Dual Spaces, Riesz Representation, and Functional Derivatives
- Lesson 42 — Numerical Linear Algebra: Conditioning, Stability, Factorizations, and Iteration
- Lesson 43 — Transforms of Distributions: Moment Generating Functions, Characteristic Functions, and the Central Limit Theorem
Part VII — Convex Optimization and Information Theory for ML, Signals, and Sensors
Convex sets, functions, and problems; regularized fitting and inverse problems; duality, KKT conditions, and certificates; gradient, Newton, and proximal algorithms; entropy, KL divergence, and mutual information; the asymptotic equipartition property and lossless compression; hypothesis testing, channels, and sensor information; rate–distortion, compression, and representation learning.
- Lesson 44 — Convex Models: Sets, Functions, and Problems
- Lesson 45 — Regularized Fitting and Inverse Problems
- Lesson 46 — Duality, KKT Conditions, and Certificates
- Lesson 47 — Algorithms: Gradient, Newton, and Proximal Steps
- Lesson 48 — Entropy, KL Divergence, and Mutual Information
- Lesson 49 — The Asymptotic Equipartition Property and Lossless Compression
- Lesson 50 — Testing, Channels, and Sensor Information
- Lesson 51 — Rate-Distortion, Compression, and Representation Learning