Course 1 — Mathematical & Theoretical Foundations

The 20-section theory prerequisite for Embedded DSP: mathematics, signal processing, and digital image and audio processing

This course is the mathematical backbone under Course 2 — Embedded DSP. Rather than build a system, it works through the highest-value sections of the core textbooks and orders them by dependency: linear algebra and the analysis that makes it rigorous, the numerical machinery that makes it computable, then probability and convex optimization, and then the complex analysis and distribution theory that put the Fourier transform (and therefore all of DSP) on solid ground. On that foundation it builds signal processing proper — digital filter design, the FFT, adaptive filtering, and the statistical signal processing of spectral estimation and Kalman filtering — and then carries the whole apparatus into two dimensions and into sound: the digital image processing of Gonzalez & Woods (spatial and frequency-domain filtering, restoration, compression, and segmentation) and the digital audio signal processing of Zölzer (quantization, sample-rate conversion, equalizers, reverberation, dynamic range control, and coding). It closes where both books do — on the convolutional neural networks that turn image and audio processing into learned signal processing, the mathematics behind Course 2’s on-device models.

It is a theory-first course, presented as a syllabus of themes and readings rather than lecture notes. For each section it names the topic and the exact chapters to read; I then work select problems from those chapters by hand, as I have been doing. The book-by-book worked solutions accumulate on the Books page.

Note on AI use: The section themes and syllabus summaries here are drafted with AI assistance for a consistent structure. The mathematics is mine: I work select problems from each section’s chapters by hand on paper, using AI only to typeset finished solutions to LaTeX/KaTeX. The goal is to learn the material, which only happens by producing the work myself.

Scope: Only the highest-value sections that support Course 2 are covered. Book-by-book worked solutions live in Books.

Book abbreviations:

Image and audio processing (Phases 5–6, Sections 15–20). The final six sections carry the whole foundation into two dimensions and into sound, using two applied books — Gonzalez & Woods for digital image processing and Zölzer for digital audio signal processing. Both are worked coursework, not reading-only companions: every chapter carries an exercise set I work by hand, tracked on the Books page. Each book closes on a convolutional-neural-network chapter, and those two chapters together form the learned-signal-processing capstone (Section 20) that leads into the on-device models of Course 2’s edge-ML module (Module 8) and its host-in-the-loop media pipeline (Module 9).

Phase 1 · Sections 1–4 — Linear Algebra, Numerical Computing, and Analysis

Section 1 — Vector Spaces, Linear Maps, and Eigenstructure

Theme: Vector spaces and finite-dimensionality; bases and dimension; linear maps and their matrices, invertibility, and duality; polynomials; eigenvalues, eigenvectors, and invariant subspaces.

Read: Axler 1–5 (vector spaces; finite-dimensional vector spaces, bases, dimension; linear maps, matrices, invertibility, products/quotients, duality; polynomials; eigenvalues, eigenvectors, and invariant subspaces).

Worked exercises: Axler Ch 1 — §1A, §1B, §1C; Ch 2 — §2A, §2B, §2C.

Section 2 — Inner Product Spaces, the Spectral Theorem, SVD, and Determinants

Theme: Inner product spaces, orthonormal bases and Gram–Schmidt, orthogonal projection and least squares; the spectral theorem for self-adjoint and normal operators; polar decomposition and the SVD; generalized eigenvectors and the Jordan form; determinants.

Read: Axler 6–9 (inner products, orthonormal bases, orthogonal complements and minimization; self-adjoint/normal operators, the spectral theorem, polar decomposition and the SVD; generalized eigenvectors, characteristic/minimal polynomials, the Jordan form; multilinear algebra and determinants).

Section 3 — Numerical Linear Algebra: QR, Least Squares, Conditioning, and Stability

Theme: Norms and orthogonal projectors; QR factorization via Gram–Schmidt and Householder; least squares; conditioning and numerical stability; floating-point arithmetic; the LU factorization with pivoting and Cholesky.

Read: T&B Lectures 1–15 (matrix-vector products, orthogonality, norms, SVD basics, projectors, QR, Gram–Schmidt, least squares, conditioning, stability, floating point), 20–23 (Gaussian elimination, pivoting, stability of LU, Cholesky).

Section 4 — Real Analysis

Theme: Completeness and suprema; sequences, the Cauchy criterion, and lim sup/inf; continuity and its global theorems (IVT, EVT); uniform convergence and power series; differentiation, the mean value and Taylor theorems; the Riemann integral and the Fundamental Theorem of Calculus.

Read: Ross 1–6 (completeness and sup/inf; sequences, Cauchy, lim sup/inf; continuity, IVT/EVT; uniform convergence and power series; differentiation, MVT, Taylor; the Riemann integral and the Fundamental Theorem of Calculus).

Worked exercises: Ross Ch 1 — §1, §2, §3, §4; Ch 2 — §8, §9, §10, §11, §12.

Phase 2 · Sections 5–8 — Probability and Convex Optimization

Section 5 — Probability Foundations and Random Variables

Theme: Probabilistic models, conditioning, Bayes’ rule, discrete and continuous random variables, expectation and variance, joint and conditional distributions, covariance, and transforms.

Read: BT 1–4 (sample space and probability; discrete random variables; general random variables; derived distributions, covariance, conditional expectation, transforms).

Section 6 — Limit Theorems, Markov Chains, and Statistical Inference

Theme: Limit theorems and the central limit theorem; Bernoulli and Poisson processes; discrete-time Markov chains and steady-state behavior; Bayesian and classical statistical inference.

Read: BT 5–9 (limit theorems; Bernoulli and Poisson processes; Markov chains; Bayesian inference; classical inference).

Section 7 — Convex Sets and Functions

Theme: Convex sets, cones, and the operations that preserve convexity; separating and supporting hyperplanes; convex functions, conjugate functions, quasiconvexity, and log-concavity.

Read: Boyd 2–3 (convex sets, hyperplanes, generalized inequalities; convex functions, operations preserving convexity, conjugate and quasiconvex functions, log-concavity).

Section 8 — Convex Problems, Duality, and Algorithms

Theme: Standard convex problem forms and their taxonomy (LP, QP, QCQP, SOCP, SDP), Lagrangian duality and the KKT optimality conditions, and the core unconstrained algorithms: gradient descent and Newton’s method with line search.

Read: Boyd 4, 5, 9 (convex/LP/QP/SOCP/SDP problems; Lagrange duality, optimality conditions, sensitivity; gradient descent, Newton’s method, line search). Skim Ch 5’s dual-derivation examples.

Phase 3 · Sections 9–10 — Complex Analysis, Distributions, and Fourier Transforms

Section 9 — Complex Analysis

Theme: Complex numbers and the complex plane; analytic functions and the Cauchy–Riemann equations; line integrals, Cauchy’s theorem, and the Cauchy integral formula; power series and Laurent series; poles and residues.

Read: B&N 1–7 (complex numbers, functions of \(z\), analytic functions, line integrals and Cauchy’s theorem, the Cauchy integral formula, power series, and Laurent series/poles/residues).

Section 10 — Distribution Theory and the Fourier Transform of Distributions

Theme: Test functions and distributions; the Dirac delta and Heaviside step; distributional derivatives; convolution; the Fourier transform of tempered distributions; and the Dirac comb and the sampling theorem, the distributional foundation of sampling and the FFT machinery of Course 2.

Read: Strichartz 1–5 (distributions, the Dirac delta and Heaviside step, distributional derivatives, convolution, and the Fourier transform of tempered distributions).

Phase 4 · Sections 11–14 — Digital and Statistical Signal Processing

The first three phases build the mathematics; this phase is where it becomes signal processing. Sections 11–12 take the transform theory of Phase 3 into the discrete, finite-wordlength world an embedded processor actually lives in — filter design and realization, the FFT, adaptive filters, and tone detection. Sections 13–14 then treat the signal as random rather than deterministic: how to estimate a spectrum from a finite noisy record and bound the error on that estimate, and how to recursively estimate the hidden state of a system from noisy measurements. These are the results Course 2 applies on the bench in Module 6, and all four books carry problem sets I work by hand. Lyons runs alongside Kuo through the first three sections as the intuition-first companion — the same material with the emphasis on what actually happens when you run it on hardware.

Section 11 — Real-Time DSP Fundamentals and Digital Filter Design

Theme: Real-time DSP system structure and the analog interface; digital signals and systems; sampling and quantization; fixed-point representation, quantization noise, and overflow; FIR filter design (windowing, linear phase) and implementation structures; IIR filter design from analog prototypes and its realizations, where the poles of Section 9 decide stability.

Read: Kuo 1–4 (real-time DSP systems and the analog interface; DSP fundamentals — digital signals and systems, random variables, fixed-point representation and quantization effects, overflow; design and implementation of FIR filters, including interpolation and decimation; design and implementation of IIR filters). Lyons 1, 2, 5, 6, 12 (discrete sequences and LTI systems; periodic sampling and aliasing, including bandpass sampling; FIR filters — convolution, windowed design, linear phase; IIR filters — the z-transform, poles and zeros, structures, and the pitfalls of building them; digital data formats and their effects — fixed-point binary formats, precision and dynamic range, finite word length).

Section 12 — Frequency Analysis, Adaptive Filtering, and Signal Detection

Theme: The Fourier series and transform specialized to the DFT, the FFT that computes it, and its implementation considerations (windowing, leakage, resolution) on a fixed-point machine; adaptive filtering — random processes, the LMS algorithm, and its performance and convergence; signal generation and detection — sine and noise generators and tone detection (DTMF/Goertzel).

Read: Kuo 5–7 (frequency analysis and the discrete Fourier transform — Fourier series, DFT, FFT, implementation considerations; adaptive filtering — random processes, adaptive filters, performance analysis; digital signal generation and detection — sine-wave and noise generators, DTMF generation and detection). Lyons 3, 4, 10, 13 (the DFT — leakage, windows, scalloping loss, resolution and zero-padding, processing gain; the FFT — the radix-2 algorithm, bit reversal, butterfly structures, and hints on using FFTs in practice; sample-rate conversion — decimation, interpolation, polyphase and CIC filters; DSP tricks — fast FIR filtering, A/D converter testing, spectral peak location).

Section 13 — Random Data: Spectral Density, Coherence, and Statistical Errors

Theme: Classification of deterministic and random data; sampling distributions, confidence intervals, and hypothesis tests; stationary random processes, autocorrelation, and spectral density functions; single-input/output relationships, frequency-response and coherence functions; and the random and bias errors, degrees-of-freedom, and record-length requirements that bound any spectrum estimated from a finite record.

Read: B&P 1, 3–6, 8, 11 (basic descriptions and classification of random data; probability fundamentals; statistical principles — sampling distributions, confidence intervals, hypothesis tests, regression; stationary random processes and spectral density functions; single-input/output relationships and coherence; statistical errors in basic estimates and record-length requirements; data-analysis procedures — FFT-based correlation and autospectral estimation). Lyons 11 (signal averaging — coherent vs. incoherent averaging, averaging multiple FFTs, and the SNR improvement that averaging buys; the practitioner’s view of the same variance reduction B&P derives).

Section 14 — Optimal Estimation: The Kalman Filter

Theme: Linear dynamic systems in state-space form and observability; the least-mean-square estimate and the orthogonality principle; random-process models and the propagation of mean and covariance; the Kalman filter as the recursive minimum-mean-square-error estimator — the discrete and Kalman–Bucy forms, the matrix Riccati equation, and the Kalman–Wiener relationship; and the computer-roundoff behavior and square-root/UD implementations that keep it stable on a microcontroller.

Read: G&A 1–5, 7 (introduction and optimal-estimation methods; linear dynamic systems and observability; probability and expectancy, the LMSE and the matrix trace; random processes, shaping filters, mean and covariance propagation, the orthogonality principle; linear optimal filters and predictors — the Kalman and Kalman–Bucy filters, optimal predictors, the matrix Riccati equation, Kalman–Wiener relationships; implementation methods — computer roundoff, effects of roundoff on Kalman filters, square-root and UD factorization).

Phase 5 · Sections 15–17 — Digital Image Processing

The mathematics of the first four phases was, almost entirely, one-dimensional. This phase carries it into two dimensions, working the highest-value chapters of Gonzalez & Woods. Nearly every result here is a 2-D restatement of something already proven: spatial filtering is 2-D convolution (Section 10), frequency-domain filtering is the 2-D DFT (Phases 3–4), and image restoration is the Wiener estimation of Section 14 applied to a degraded image. What is new is the domain — pixels, intensity, and the eye — and the algorithms that only make sense there: histogram processing, segmentation, and the compression that leans directly on the information theory of Section 15’s neighbour, Cover & Thomas. This is the theory behind the image and video labs of Course 2 Module 9. Every chapter carries a problem set, worked by hand.

Section 15 — Digital Image Fundamentals and Spatial Filtering

Theme: Image sensing and acquisition; sampling and quantization in two dimensions; basic relationships between pixels; intensity transformations and histogram processing; and spatial filtering — correlation and convolution over a neighbourhood, smoothing (lowpass) filters, and sharpening (highpass) filters built from the gradient and the Laplacian. Spatial filtering is the 2-D form of the convolution of Section 10.

Read: G&W 2–3 (digital image fundamentals — visual perception, image sensing, sampling and quantization, basic relationships between pixels; intensity transformations and spatial filtering — histogram processing, the fundamentals of spatial filtering, smoothing/lowpass and sharpening/highpass filters, the gradient and the Laplacian).

Section 16 — Frequency-Domain Filtering and Image Restoration

Theme: The 2-D discrete Fourier transform and its properties; filtering in the frequency domain — lowpass, highpass, and notch filters and the convolution theorem; image degradation and noise models; and image restoration by inverse and Wiener filtering, the 2-D application of the minimum-mean-square-error estimation of Section 14.

Read: G&W 4–5 (filtering in the frequency domain — the 2-D DFT and its properties, the basics of frequency-domain filtering, lowpass/highpass/selective filters, the convolution theorem; image restoration and reconstruction — noise models, restoration by spatial filtering, the degradation function, inverse filtering, and the Wiener filter).

Section 17 — Compression, Segmentation, and Feature Extraction

Theme: Image compression — coding redundancy and the entropy coders (Huffman, arithmetic, run-length) that reach the source-coding bound of Cover & Thomas; segmentation — edge detection (the gradient, Marr–Hildreth, Canny), thresholding, region growing, clustering (K-means, superpixels), and the morphological watershed; and feature extraction — boundary and region descriptors, principal components, and the Scale-Invariant Feature Transform (SIFT).

Read: G&W 8, 10–11 (image compression — coding redundancy, Huffman/arithmetic/run-length coding, and the fidelity criteria; image segmentation — point/line/edge detection, thresholding, region-based segmentation, clustering and superpixels, watersheds; feature extraction — boundary and region feature descriptors, principal components, and the SIFT). Ch. 7 (wavelet and other image transforms) and Ch. 9 (morphological image processing) as needed for the transform-based compression and the watershed.

Phase 6 · Sections 18–20 — Digital Audio Processing and Learned Signal Processing

The closing phase turns from images to sound, and then closes the whole course on the point where both media meet: the convolutional neural network. Two sections work the highest-value chapters of Zölzer — the quantization, sample-rate conversion, and converter theory that sit under any audio interface, then the effects, reverberation, dynamic-range, and coding algorithms that are audio processing. Much of it is the one-dimensional counterpart of Phase 5’s image work and an extension of the signal processing of Phase 4: Zölzer’s quantization and noise-shaping refine Section 11, his sample-rate conversion refines the multirate material of Section 12, and his audio coding leans, like image compression, on Cover & Thomas. The final section is the capstone — the deep convolutional networks that end both books, the learned signal processing that underpins Course 2’s edge-ML module (Module 8). Every chapter carries exercises, worked by hand.

Section 18 — Digital Audio Fundamentals: Quantization, Sample-Rate Conversion, and AD/DA

Theme: Continuous- and discrete-time signals and transforms in review; quantization — the classical error model, the quantization theorem, dither, noise shaping, and fixed/floating-point number representation; sample-rate conversion — up/downsampling with anti-imaging and antialiasing filtering, synchronous and asynchronous conversion, and polynomial/Lagrange/spline interpolation; and AD/DA conversion — Nyquist sampling, oversampling, and delta-sigma modulation.

Read: Zölzer 1–4 (introduction — continuous/discrete-time signals and transforms, the DFT, FIR/IIR and adaptive filters; quantization — the classical model and quantization theorem, dither, noise shaping, fixed- and floating-point representation; sampling rate conversion — up/downsampling, synchronous and asynchronous conversion, polynomial/Lagrange/spline interpolation; AD/DA conversion — Nyquist sampling, oversampling, and delta-sigma modulation). Ch. 5 (audio processing systems — DSPs and digital audio interfaces) as reference.

Section 19 — Audio Effects, Room Simulation, Dynamic Range, and Coding

Theme: Equalizers — recursive/parametric filter design, fast convolution (overlap-add/overlap-save), and complementary filter banks; room simulation — impulse responses, early reflections, and Schroeder/feedback-delay-network reverberation; dynamic range control — the static curve, level measurement and time constants, and limiters, compressors, expanders, and multiband processing; audio coding — lossless and lossy coding, psychoacoustics and masking, and ISO-MPEG; nonlinear/virtual-analog processing.

Read: Zölzer 6–10 (equalizers — recursive/parametric audio filters, fast convolution, complementary filter banks, delay-based effects; room simulation — room acoustics and impulse responses, early reflections, Schroeder and feedback-based reverberation; dynamic range control — static curve, level measurement, limiter/compressor/expander/noise-gate, multiband DRC; audio coding — lossless and lossy coding, psychoacoustics and masking, ISO-MPEG1 audio coding; nonlinear processing — virtual-analog modelling).

Section 20 — Learned Signal Processing: CNNs for Images and Audio

Theme: Pattern classes and prototype/minimum-distance classification; the feedforward neural network and the backpropagation algorithm (gradient descent, Sections 7–8, applied to a composed nonlinear function); and the deep convolutional network, where the learnable kernel is the descendant of the convolution and filter theory of Sections 10, 15, and 19. Gonzalez & Woods develops it for images (with a first-principles derivation of backpropagation), Zölzer for audio — the theory behind the on-device classifier, denoiser, and image/video/audio models of Course 2 Module 8 and Module 9.

Read: G&W 12 (image pattern classification — patterns and pattern classes, prototype/minimum-distance matching, the Bayes classifier, feedforward neural networks and the derivation of backpropagation, and deep convolutional neural networks) and Zölzer 11 (machine learning for audio — unsupervised and supervised learning, the feedforward artificial neural network, and the convolutional neural network). The two CNN chapters are the same idea in two media; read them together.

How this course supports Course 2

Every section here exists to underwrite a piece of Course 2 — Embedded DSP, where the mathematics is applied on the bench, never re-taught:

  • Fourier & distribution theory (Sections 9–10) → sampling. The Dirac comb and the distributional Fourier transform are the sampling theorem, the foundation of Course 2’s data acquisition and aliasing labs (Module 5).
  • Complex analysis (Section 9) → filter stability. Poles, zeros, and the region of convergence are the language of the z-transform and of IIR-filter stability (Module 6).
  • Probability (Sections 5–6) → noise and PSD. Random variables, expectation, and the limit theorems underpin the noise-floor and power-spectral-density characterization (Module 6) and the on-device ML signal processing (Module 8).
  • Linear algebra (Sections 1–3) → least squares and filter design. The SVD, least squares, and numerical stability drive filter design, system identification, and the fixed-point/floating-point reasoning of real-time firmware.
  • Convex optimization (Sections 7–8) → edge-ML training. Convexity, duality, and the gradient/Newton algorithms are the engine under the losses and optimizers that fit the on-device ML models (Module 8).
  • Digital signal processing (Sections 11–12) → the real-time DSP module. Filter design and realization, fixed-point and overflow behavior, the FFT, adaptive filtering, and tone detection are precisely the FIR, IIR-biquad, spectrum-analyzer, and Goertzel labs of Module 6 — worked on paper here before they are written in C there.
  • Statistical signal processing (Sections 13–14) → measurement and estimation. The random and bias errors, degrees of freedom, and confidence intervals of Section 13 are what make the noise-floor/PSD lab (6.4) and the vibration analysis (Module 8) measurements rather than guesses; the Kalman filter of Section 14 is the theory the real-time state-estimation lab (6.6) implements.
  • Digital image processing (Sections 15–17) → the image and video labs. 2-D spatial and frequency-domain filtering, restoration, and compression are precisely the 2-D separable blur, the Sobel edge detector, and the video-frame pipeline of Module 9 (labs 9.4 and 9.5) — worked as image mathematics here before they are streamed to the target there.
  • Digital audio processing (Sections 18–19) → the audio labs. Quantization and noise shaping, sample-rate conversion, parametric equalizers, fast convolution, and STFT-based effects are the theory behind the streamed-audio-filter and STFT-effects labs of Module 9 (labs 9.1–9.3), and the converter theory underpins the mixed-signal acquisition of Modules 3 and 5.
  • Learned signal processing (Section 20) → the edge-ML models. The convolutional networks that close both image and audio books are exactly the on-device classifier, denoiser, and image/video/audio models that the edge-ML module (Module 8) and the media pipeline (Module 9) are built from — the learnable convolution kernel being the direct descendant of this course’s convolution and filter theory.