Course 1 — Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence

One self-contained booklet: Abridged Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence — 70 lessons in 9 parts, from vectors to deep learning and real-time rendering

This course is a single self-contained booklet, worked cover to cover:

→ Abridged Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence (PDF, 337 pages)

Seventy lessons in nine parts, dependency-ordered: the linear algebra core; eigenstructure, the spectral theorem, the DFT, and the SVD; the probability core; random vectors, limit theorems, and stochastic processes; signals and systems; the analysis behind the transforms; convex optimization and information theory for machine learning, signals, and sensors; neural networks and deep learning; and the mathematics of real-time rendering — the theory Course 4 applies. The booklet is sized against the undergraduate prerequisite sets for machine learning, the Fourier transform and its applications, and probability and statistical inference. The applied layer that used to follow — the DFT in practice, filter design and finite word lengths, multirate and sigma–delta conversion, spectrum estimation, Wiener/LMS/Kalman filtering, detection and communication, images, audio, and the electronics bench — now lives as theory sections inside the Course 3 — Embedded DSP labs that exercise it; Course 3 assumes this course as a mastered prerequisite.

Lesson structure. Every lesson carries two interleaved layers — Theory (precise definitions and theorems, with complete proofs where feasible) and Concrete practice (small 2-D/3-D examples, dice, coins, three-state chains, computed by hand) — followed by Exercises tagged [Proof] or [Hand], with a star on the most valuable proofs. Every lesson opens with a Needed for tag naming the courses and labs it serves, and closes with a Deeper reading pointer into the full textbooks.

Worked exercises. The booklet carries 294 exercises across its 70 lessons — 147 [Hand], 147 [Proof], numbered lesson.exercise. My solutions are worked by hand on paper, then typeset: one page per lesson, linked under each part below and collected in the booklet’s exercise set on the Books page.

Order of work. Parts I and III first (in that order); Part II next, with Part V right after it; Part IV any time after Part III (Parts IV and V are independent); Part VI at a relaxed pace, or interleaved with the Course 3 labs; Part VII any time after Parts I, III, and V; Part VIII after Parts I, III, and VII; Part IX any time after Parts I, II, IV, and VI (and Lesson 62), before or alongside Course 4.

Note on AI use: The booklet’s teaching text is AI-drafted and owner-reviewed. The mathematics I produce is mine: the exercises are worked by hand on paper, with AI used only to typeset finished solutions. The booklet’s own exercise set and the worked problem sets from the full textbooks — the same books its Deeper reading pointers name — both accumulate on the Books page.

Part I — The Linear Algebra Core

The linear algebra core: vectors and linear combinations; dot products, norms, and orthogonality; span, independence, basis, and dimension; matrices as linear maps; null space and rank; orthogonal projections and least squares; gradients.

Worked exercises: Lesson 1, Lesson 2, Lesson 3, Lesson 4, Lesson 5, Lesson 6, Lesson 7.

  • Lesson 1 — Vectors and Linear Combinations
  • Lesson 2 — Dot Products, Norms, and Orthogonality
  • Lesson 3 — Span, Linear Independence, Basis, Dimension
  • Lesson 4 — Matrices and Linear Maps
  • Lesson 5 — Null Space, Rank, and Solving \(Ax=b\)
  • Lesson 6 — Orthogonal Projections and Least Squares
  • Lesson 7 — Gradients for Machine Learning

Part II — Eigenstructure, the Spectral Theorem, the DFT, and the SVD

Complex vector spaces and complex exponentials; eigenvalues, eigenvectors, and diagonalization; self-adjoint operators and the spectral theorem; unitary matrices and the discrete Fourier transform; the singular value decomposition; determinants and trace.

Worked exercises: Lesson 8, Lesson 9, Lesson 10, Lesson 11, Lesson 12, Lesson 13.

  • Lesson 8 — Complex Vectors and Complex Exponentials
  • Lesson 9 — Eigenvalues, Eigenvectors, and Diagonalization
  • Lesson 10 — Self-Adjoint Operators and the Spectral Theorem
  • Lesson 11 — Unitary Matrices and the Discrete Fourier Transform
  • Lesson 12 — The Singular Value Decomposition
  • Lesson 13 — Determinants and Trace

Part III — The Probability Core

Probability models; counting — permutations, combinations, partitions; conditioning and Bayes’ rule; discrete and continuous random variables; functions of a random variable and the expected value rule; expectation and variance; joint distributions, functions of several random variables, and linearity of expectation; conditioning; the normal distribution.

Worked exercises: Lesson 14, Lesson 15.

  • Lesson 14 — Probability Models, Counting, Conditioning, and Bayes’ Rule
  • Lesson 15 — Discrete Random Variables and Expectation
  • Lesson 16 — Multiple Random Variables and Conditioning
  • Lesson 17 — Continuous Random Variables and the Normal

Part IV — Random Vectors, Limit Theorems, and Stochastic Processes

Derived distributions, covariance, and correlation; conditional expectation and least-mean-squares estimation; random vectors and Gaussian vectors; limit theorems; the Bernoulli and Poisson processes; Markov chains.

  • Lesson 18 — Derived Distributions, Covariance, and Correlation
  • Lesson 19 — Conditional Expectation and Least Mean Squares
  • Lesson 20 — Random Vectors and Gaussian Vectors
  • Lesson 21 — Limit Theorems
  • Lesson 22 — The Bernoulli and Poisson Processes
  • Lesson 23 — Markov Chains

Part V — The Signals-and-Systems Core

Signals and system properties; LTI systems and convolution; Fourier series; the continuous- and discrete-time Fourier transforms; frequency response and filtering; sampling and aliasing, and exact finite sampling of band-limited periodic signals by the DFT; Laplace and \(z\)-transforms.

  • Lesson 24 — Signals and Systems
  • Lesson 25 — LTI Systems and Convolution
  • Lesson 26 — Fourier Series
  • Lesson 27 — The Continuous-Time Fourier Transform
  • Lesson 28 — The Discrete-Time Fourier Transform and Frequency Response
  • Lesson 29 — Sampling
  • Lesson 30 — Laplace and \(z\)-Transforms, a Working Minimum

Part VI — The Analysis Behind the Transforms

Sequences, series, and interchange of limits; metric spaces, continuity, and compactness; differentiation and the calculus theorems — the mean value theorem, L’Hôpital, Taylor — with the \(n\)-dimensional upgrade; the Riemann integral and the fundamental theorem of calculus; Lebesgue measure and integration and the swap theorems; Banach and Hilbert spaces, \(L^2\), and orthonormal bases; complex analysis — analytic functions, power series, and contour integrals, then zeros, poles, residues, and ROCs; the Fourier, Laplace, and \(z\)-transforms as operators, with the property tables proved; distributions and \(\delta\), Heisenberg’s inequality, and the Hilbert transform, the analytic signal, and causality; dual spaces, Riesz representation, and functional derivatives; numerical linear algebra — conditioning, stability, factorizations, and iteration; transforms of distributions — moment generating and characteristic functions, the Chernoff bound, and the central limit theorem with proof.

  • Lesson 31 — Limits Done Right: Sequences, Series, and When Swapping Is Legal
  • Lesson 32 — Metric Spaces, Continuity, and Compactness
  • Lesson 33 — Differentiation: The Calculus Theorems, Done Right
  • Lesson 34 — The Riemann Integral and the Fundamental Theorem
  • Lesson 35 — Integration Done Right: Lebesgue, Almost Everywhere, and the Swap Theorems
  • Lesson 36 — Signal Space: Banach and Hilbert Spaces, \(L^2\), and Orthonormal Bases
  • Lesson 37 — Complex Analysis I: Analytic Functions, Power Series, and Contour Integrals
  • Lesson 38 — Complex Analysis II: Zeros, Poles, Residues, and Why ROCs Work
  • Lesson 39 — The Fourier, Laplace, and \(z\)-Transforms as Operators: The Property Toolbox, Proved
  • Lesson 40 — Distributions: Making \(\delta\) Honest
  • Lesson 41 — Dual Spaces, Riesz Representation, and Functional Derivatives
  • Lesson 42 — Numerical Linear Algebra: Conditioning, Stability, Factorizations, and Iteration
  • Lesson 43 — Transforms of Distributions: Moment Generating Functions, Characteristic Functions, and the Central Limit Theorem

Part VII — Convex Optimization and Information Theory for ML, Signals, and Sensors

Convex sets, functions, and problems; regularized fitting and inverse problems; duality, KKT conditions, and certificates; gradient, Newton, and proximal algorithms; entropy, KL divergence, and mutual information; the asymptotic equipartition property and lossless compression; hypothesis testing, channels, and sensor information; rate–distortion, compression, and representation learning.

  • Lesson 44 — Convex Models: Sets, Functions, and Problems
  • Lesson 45 — Regularized Fitting and Inverse Problems
  • Lesson 46 — Duality, KKT Conditions, and Certificates
  • Lesson 47 — Algorithms: Gradient, Newton, and Proximal Steps
  • Lesson 48 — Entropy, KL Divergence, and Mutual Information
  • Lesson 49 — The Asymptotic Equipartition Property and Lossless Compression
  • Lesson 50 — Testing, Channels, and Sensor Information
  • Lesson 51 — Rate-Distortion, Compression, and Representation Learning

Part VIII — Neural Networks and Deep Learning

Feedforward networks, activations, and what depth buys; logits, softmax, and the losses maximum likelihood forces; logistic regression and kernel methods; the matrix and tensor calculus behind the derivations — outer products, Kronecker products, traces, and index notation; Jacobians, the chain rule, and backpropagation; vanishing and exploding gradients, initialization, normalization, and residual connections; convolutional networks — equivariance, receptive fields, and parameter sharing; recurrence, gating, and attention; autoencoders, variational autoencoders, and adversarial networks; risk, the bias–variance decomposition, and cross-validation; Monte Carlo, Metropolis–Hastings, and Gibbs sampling.

  • Lesson 52 — Feedforward Networks: Layers, Activations, and What Depth Buys
  • Lesson 53 — Logits, Softmax, and the Losses That Come From Likelihood
  • Lesson 54 — Logistic Regression and Kernel Methods: The Two Classical Routes to Nonlinearity
  • Lesson 55 — The Matrix and Tensor Toolkit: Outer Products, Kronecker Products, and Index Notation
  • Lesson 56 — Jacobians, the Chain Rule, and Backpropagation
  • Lesson 57 — Training Dynamics: Vanishing Gradients, Initialization, and Normalization
  • Lesson 58 — Convolutional Networks: Weight Sharing, Equivariance, and Receptive Fields
  • Lesson 59 — Sequences: Recurrence, Gating, and Attention
  • Lesson 60 — Learned Representations: Autoencoders, Variational Autoencoders, and Adversarial Networks
  • Lesson 61 — Generalization: Risk, Validation, and Cross-Validation
  • Lesson 62 — Sampling: Monte Carlo, Markov Chains, and Gibbs

Part IX — The Mathematics of Real-Time Rendering

Points, vectors, and homogeneous coordinates — affine maps, the view matrix, perspective projection and its hyperbolic depth, reverse-Z; rotations — \(SO(3)\), Rodrigues’ formula, Euler angles and gimbal lock, quaternions and slerp; triangles, planes, and frusta — barycentric coordinates, edge functions, perspective-correct interpolation, frustum-plane extraction and culling, the geometry of shadow maps; radiometry and the rendering equation — the Lambertian \(1/\pi\), Blinn–Phong, Fresnel, and the Cook–Torrance microfacet model with GGX; integrating light — Monte Carlo shading, cosine and GGX importance sampling, the split sum, spherical harmonics, mipmapping as prefiltering; color — spectra as vectors, metamers, sRGB, high dynamic range, and tone mapping; Gaussian splatting — anisotropic Gaussians, their linearized projection, and alpha compositing; Fourier analysis in two dimensions — spatial frequency, the stretch theorem, sampling on a lattice and the dual lattice, when a texture lookup aliases, the 2-D DFT, and the projection–slice theorem.

  • Lesson 63 — Points, Vectors, and Homogeneous Coordinates: Affine Maps, Cameras, and Perspective Projection
  • Lesson 64 — Rotations and Orientation: \(SO(3)\), Axis–Angle, Euler Angles, and Quaternions
  • Lesson 65 — Triangles, Planes, and Frusta: Barycentric Coordinates, Rasterization, Interpolation, and Culling
  • Lesson 66 — Radiometry, the Rendering Equation, and Reflection Models: Lambert, Blinn–Phong, Fresnel, and Microfacets
  • Lesson 67 — Integrating Light: Monte Carlo Shading, Importance Sampling, Spherical Harmonics, and the Split Sum
  • Lesson 68 — Color and the Display: Spectra as Vectors, Linear Color, sRGB, High Dynamic Range, and Tone Mapping
  • Lesson 69 — Gaussian Splatting: Anisotropic Gaussians, Their Projection, and Alpha Compositing
  • Lesson 70 — Fourier Analysis in Two Dimensions: Spatial Frequency, the Stretch Theorem, Lattice Sampling, the 2-D DFT, and Projection–Slice