Course 2 — Programming Foundations: Scientific Python, Modern Embedded C & Embedded Rust

Language and library fundamentals for the algorithm-first workflow: prototype and verify in Python (NumPy, SciPy, PyTorch), then implement in C17/18 or Rust on bare metal, under an RTOS, or on embedded Linux — the programming prerequisite for Course 3

This is the programming course beside Course 3 — Embedded DSP: the three languages and their libraries that every lab on this site is written in, taught to the level where the language stops being the problem. The working rhythm the course installs is the one Course 3 assumes — an algorithm is written first in Python, against NumPy and SciPy (or PyTorch, when it is learned), where it is cheap to be wrong and the reference arrays come from; then it is implemented in C17/18 or Rust on the tier where it will run, and checked against the Python version. Three tracks, one course:

From Module 4 on, every concept is taught once and then shown in both C and Rust, on the runtime tier where it lives; the Python modules come first because that is where the algorithms start.

Three runtime tiers, one rule. The rule is that the compiled code uses only libraries and constructs that exist on the target — no host-only conveniences smuggled into firmware. The tiers are the ones the site’s hardware actually has:

Tier Hardware C Rust Python
Bare metal STM32 NUCLEO-L476RG — Cortex-M4F @ 80 MHz, ARMv7E-M Thumb-2 (not ARM64) -std=gnu17, arm-none-eabi-gcc + newlib-nano, CMSIS/HAL thumbv7em-none-eabihf, #![no_std], cortex-m-rt, the stm32l4 PAC, embassy-stm32 as HAL, embedded-hal 1.0, heapless, defmt — (the reference implementation runs on the host)
RTOS the same STM32 FreeRTOS (native API and CMSIS-RTOS2) RTIC 2 and Embassy (embassy-executor, embassy-time, embassy-sync) —
Embedded Linux Jetson Orin Nano (JetPack 6, aarch64) and Raspberry Pi 5 glibc + POSIX — pthread, timerfd, epoll, mmap, SCHED_FIFO; libgpiod, i2c-dev, spidev std, nix, gpiod, linux-embedded-hal — the same embedded-hal driver crate that ran on the STM32 NumPy/SciPy, CuPy and onnxruntime/TensorRT on the Jetson; onnxruntime on the Pi
Host Apple Silicon Mac; the RTX 4090 Linux desktop for training clang + sanitizers cargo + Clippy + Miri the full stack — Jupyter, NumPy/SciPy/Matplotlib, scikit-learn, PyTorch (MPS on the Mac, CUDA on the desktop)

All three compiled tiers are reasoned about from the Mac: C is compiled with clang under AddressSanitizer/UndefinedBehaviorSanitizer for the host experiments and cross-compiled to thumbv7em-none-eabihf for the Cortex-M forensics; Rust is built with cargo for the host, checked for every target, tested under Miri, and — for the bare-metal tier — run in QEMU (lm3s6965evb, semihosting) so that interrupts, panics, and no_std startup are exercised without a board. Flashing the NUCLEO with probe-rs and building on the Jetson over SSH are the optional last rungs of each exercise, not prerequisites for any of them. Python runs on the Mac throughout, with the Jetson and the desktop GPU as optional rungs for inference and training.

Note on AI use: As across this site, the module themes, reading map, and exercise statements are drafted with AI assistance for consistent structure. The per-module lesson pages go further: they are AI-drafted teaching text, generated from the module’s references and reviewed by me, so that the books need not be read to work the exercises. The engineering substance remains mine: every line of exercise code in any of the three languages, every notebook, every build, every disassembly annotation, every measurement, and every reconciliation note is written, run, and debugged by me on my own machines.

Prerequisites: general programming fluency (this is not a first course in any of the three languages’ syntax); K&R 2nd edition read cover to cover for the C track — pointers, arrays, structs, functions, and the preprocessor are never retaught; a working knowledge of Python at the level of writing scripts and classes. No prior NumPy, Rust, or PyTorch is assumed: Module 1 teaches the ndarray model from its memory layout up, Module 5 teaches Rust from the Rust Book for a reader who already knows C, and Module 3 teaches PyTorch from tensors up. No Course 1 dependency — modules link Course 1 lessons where a library function is a theorem: Course 3 Lab 6.3 theory (the DFT in practice), Course 3 Lab 6.1 theory (filters and fixed point), Course 3 Lab 6.4 theory (spectrum estimation), Lesson 42 (floating point, conditioning), and Lessons 44–51 (the optimization and information theory behind training).

Book and source abbreviations (all cited inline; none carries a problem set, so none gets a Books-page entry):

Workflow: the code lives beside the Course 3 lab work in the companion repo diivanand/diiv_website_custom_courses, under a top-level course2/ folder with three trees organized by tier: python/ (scripts, notebooks, and pytest tests, run through the repo’s root uv project), c/ (CMake projects host, mcu, linux), and rust/ (one cargo workspace with crates host, mcu, qemu, linux), plus one mN/notes.md per module recording predicted vs. observed. Exercises are per-tier files — python/src/ex-1-3.py, c/host/src/ex-5-3/, rust/mcu/src/bin/ex-8-2.rs — and the build systems are pre-built so that no time goes into configuration. Predicted-vs-observed tables on the exercise pages leave the observed cells as “…” — I fill them at the machine, same convention as Course 3’s bench labs. Setup for the boards themselves is Course 3’s: STM32 with CubeMX + CMake + CLion and the Jetson Orin Nano essentials; the Python workbench conventions are Course 3’s MATLAB → Python map.

Order of work. Module 0 first. Then Part I (Python) straight through, since every later reference implementation depends on it. Parts II–V are dependency-ordered for the C and Rust tracks; a reader who only needs one of the two languages can skip the other’s half of each module, but the capstone needs both.

Part I · Modules 0–3 — Toolchains and Scientific Python

Module 0 — Toolchains, Targets, and the Availability Matrix

Theme: Hosted vs. freestanding C and the standard’s portability taxonomy (implementation-defined, unspecified, undefined, locale-specific); newlib-nano vs. glibc; core vs. alloc vs. std and what #![no_std] removes; the Mac toolchain for all three tiers and all three languages — clang and the optional arm-none-eabi-gcc, rustup targets, cargo-binutils, probe-rs, QEMU, uv and Jupyter; CMake for C, cargo for Rust, the root uv project for Python; cortex-m-rt, memory.x, #[entry], panic handlers, and the first no_std binary run in QEMU; reading sizes and disassembly; warnings and Clippy as a contract; the availability matrix — heap, threads, atomics width, floating point, printf, files, time, signals — across tiers and languages, established by experiment.

Lessons: Module 0 lessons — the three tiers as three C libraries and three Rust crate sets, the toolchain and workspace tour for all three languages, the first bare-metal binary end to end, and the availability matrix.

Reference (optional deep-dives): Seacord 1 (Getting Started — the portability taxonomy); Rust Book 1, 14 (Getting Started; Cargo); Embedded Rust Book Introduction and Getting Started (QEMU, hardware, memory.x); Embedonomicon The smallest #![no_std] program and Memory layout; the uv documentation.

Practice: Module 0 exercises — bring-up on every tier and every language, hello world in QEMU, first size and disassembly readings, the availability matrix filled by experiment, deliberate build failures read and explained, the Python environment and device probe.

Module 1 — Scientific Python Core: NumPy, SciPy, Matplotlib, and the Notebook Workflow

Theme: The uv project and Jupyter workflow; the ndarray model — dtype, shape, strides, C and Fortran order, views vs. copies — as the C memory model it wraps; indexing (basic, slicing, fancy, boolean), broadcasting, vectorization and ufuncs, reductions over axes, reshape/transpose/stack; index notation — einsum, tensordot, matmul vs. dot, contraction order — and the DSP/ML settings it covers (covariance and Gram matrices, mel projections, pairwise distances, bilinear forms, batched products); np.linalg, np.fft, np.random.Generator; dtype pitfalls — silent integer wraparound, float32 vs. float64, finfo/iinfo; structured arrays and frombuffer for binary captures and serial data; np.testing.assert_allclose as the arbiter convention; the SciPy tour — scipy.signal (filter design, lfilter/sosfilt/filtfilt, freqz, welch, stft, resample_poly, correlate, find_peaks, hilbert, windows), scipy.fft, scipy.linalg (Toeplitz solvers, factorizations), scipy.optimize, scipy.interpolate, scipy.integrate, scipy.io; Matplotlib’s object-oriented API for spectra, spectrograms, and figures worth keeping; pandas for bench CSV exports.

Lessons: Module 1 lessons — the numeric stack from the memory model up, each library function placed against the Course 1 lesson it implements and the C array it will later become.

Reference (optional deep-dives): NumPy user guide (absolute basics; fundamentals — indexing, broadcasting, copies and views, structured arrays); SciPy tutorials (signal, fft, linalg, optimize); Matplotlib quick start; pandas “10 minutes”.

Practice: Module 1 exercises — the ndarray memory-model probe, the vectorization ladder, FIR and IIR filters by hand vs. scipy.signal, PSD estimation by hand vs. welch, a Goertzel detector against its reference, the figure set, and eight contractions written three ways (loops, @/broadcasting, einsum) — each exercise saving the reference arrays a later C or Rust module is checked against.

Module 2 — Processing, Statistics, and Classical Machine Learning Libraries; Bridging Python to C and Rust

Theme: Audio (soundfile, sounddevice, librosa — load, resample, STFT, mel, MFCC); images and video (Pillow, OpenCV, scipy.ndimage); statistics — scipy.stats distributions and tests, bootstrapping with Generator, histograms, estimators and error bars; scikit-learn — train_test_split, scaling, PCA, pipelines, logistic regression, SVMs, nearest neighbors, random forests, cross-validation, the metrics behind ROC and confusion matrices; Numba and the CuPy drop-in on the Jetson; bridging — ndarray memory ↔︎ C pointers with ctypes and cffi, calling a C kernel from the notebook as a shared library, exposing Rust to NumPy with PyO3 and maturin, and choosing assert_allclose tolerances from the arithmetic (float64 reference vs. float32 device vs. Q15); pyserial for host-in-the-loop verification.

Lessons: Module 2 lessons — the processing and statistics libraries in the roles the labs use them in, and the three bridges that connect a Python reference to a C or Rust implementation.

Reference (optional deep-dives): librosa tutorial; OpenCV-Python tutorials; scikit-learn getting started and user guide; scipy.stats tutorial; NumPy’s C-API and ctypes guide; PyO3 user guide; Numba five-minute guide.

Practice: Module 2 exercises — an audio pipeline librosa vs. by hand, image convolution and edges three ways, a noise-floor estimate with confidence intervals, features + PCA + a classifier with ROC and confusion matrix, a Numba/CuPy speed-up table, a C kernel called through ctypes and a Rust kernel through PyO3, both verified with assert_allclose.

Module 3 — PyTorch: Tensors, Autograd, Training, and Deployment to the Edge

Theme: Tensors vs. ndarrays (zero-copy interop, dtype and device, MPS on the Mac and CUDA on the desktop and the Jetson); autograd — requires_grad, backward, no_grad, the graph; nn.Module, the common layers, losses, optimizers, schedulers; Dataset/DataLoader; the training and evaluation loops written by hand; metrics, reproducibility, TensorBoard; torch.fft and torchaudio transforms, torchvision models and transfer learning; quantization (dynamic, static, QAT overview) and pruning; export — torch.export/TorchScript → ONNX → onnxruntime on the Mac → TensorRT on the Jetson; profiling with torch.profiler, mixed precision, torch.compile; saving and loading state_dicts.

Lessons: Module 3 lessons — PyTorch from tensors to a deployed edge model, following the UvA notebook arc, with the training loop written in full and the export chain to the boards.

Reference (optional deep-dives): the UvA notebooks Introduction to PyTorch; PyTorch Learn the Basics, quantization, and ONNX export documentation; ONNX Runtime Python quick start.

Practice: Module 3 exercises — tensor/ndarray interop and a device probe, autograd by hand vs. backward, the XOR training loop from scratch, a 1-D CNN keyword classifier and a small learned denoiser (the Course 3 Module 8 precursors), a quantization ladder, ONNX export with onnxruntime parity, and a profiler reading.

Part II · Modules 4–5 — The C Subset and the Rust Core

Module 4 — The Modern C Subset: C99 → C17/18 for Embedded Work

Theme: The standards timeline and dialect pinning (-std=gnu17, the warning set); objects, storage duration, scope, and linkage; the fixed-width integer toolbox and LP64 vs. ILP32; integer promotions and the usual arithmetic conversions; modern initialization — designated initializers, compound literals, {0}; static inline, restrict, flexible array members, why VLAs are banned; the compile-time toolkit — _Static_assert, _Alignas/_Alignof, anonymous structs and unions, _Generic, _Noreturn; expressions, evaluation order, and sequencing; control-flow idioms including single-exit cleanup; preprocessor hygiene; what <stdatomic.h> and <threads.h> mean on each tier.

Lessons: Module 4 lessons — the working subset of modern C, feature by feature, each with its standard of origin, its embedded use, its trap, and a pointer to the Rust construct that replaces it.

Reference (optional deep-dives): MC 1–38; Seacord 2 (Objects, Functions, and Types), 3 (Arithmetic Types), 4 (Expressions and Operators), 5 (Control Flow), 9 (Preprocessor). Material beyond C17/18 is skipped.

Practice: Module 4 exercises — the same file through three compilers, modernizing a K&R-era module with a codegen diff as proof, width and promotion predict-verify across three ABIs, an expression safari, compound-literal lifetime pitfalls, compile-time layout contracts, preprocessor forensics, a _Generic type-safe clamp and restrict on a DSP kernel.

Module 5 — The Rust Core for Embedded Work

Theme: Rust for a C programmer: ownership, moves, and borrows as the rules C leaves to convention; references and slices; structs, enums, Option/Result, and exhaustive match; modules and crates; why Vec and String live in alloc; error handling without exceptions — Result propagation with ?, the panic policy (panic = "abort", #[panic_handler], no unwrap in firmware); generics, traits, and monomorphization; lifetimes as they appear in driver APIs; iterators and closures that compile to loops; patterns; trait objects vs. generics; const, static, const fn; arrays, slices, and const generics; integer overflow semantics (wrapping_*, checked_*, saturating_*); the no_std idioms that follow from all of it.

Lessons: Module 5 lessons — the Rust Book compressed for someone who knows C, with every concept paired against the C construct it replaces and the embedded constraint it serves.

Reference (optional deep-dives): Rust Book 3–10, 13, 18, 19; Embedded Rust Book Static Guarantees and Portability.

Practice: Module 5 exercises — the same small module written in C and Rust; a Result-based driver API; a no_std ring buffer with const generics run in QEMU; overflow semantics predict-verify; generics vs. trait objects counted in the disassembly; Clippy-clean and Miri-clean deliverables.

Part III · Modules 6–8 — Memory and the Machine

Module 6 — Memory Without a Heap

Theme: Storage durations and the firmware rule “no heap after init”; malloc/aligned_alloc/realloc/free, memory states, and why they are banned or fenced in firmware; static pools, arenas, ring buffers, and object pools in C; stack sizing with -fstack-usage and the map file; in Rust — where Box/Rc/Arc come from and why alloc is a choice, heapless collections and queues, StaticCell, MaybeUninit, #[repr(C)]/#[repr(align)], size_of/align_of, DMA buffer ownership and 'static; on Linux — mlockall, pre-faulting, and why real-time loops avoid the allocator too.

Lessons: Module 6 lessons — storage, layout, and allocation on each tier, and the static-allocation idioms that replace the heap in both languages.

Reference (optional deep-dives): Seacord 6 (Dynamically Allocated Memory); Rust Book 15 (Smart Pointers); heapless and static_cell crate docs; MC 25–27, 46.

Practice: Module 6 exercises — layout predict-verify tables in both languages, a fixed-block pool allocator in both, DMA buffer ownership, stack-usage prediction against .su files, a heapless queue in QEMU, what a heap costs, page faults in a real-time loop.

Module 7 — Undefined Behavior and unsafe

Theme: The firmware undefined-behavior list — signed overflow, oversized shifts, strict aliasing and effective types, out-of-bounds and uninitialized reads, sequence points, data races, restrict lies, misaligned access — and how the optimizer exploits each; memcpy as the blessed type pun; sanitizers, -fwrapv/-fno-strict-aliasing as crutches, static analysis, and the assertion discipline; in Rust — safety vs. soundness, the unsafe superpowers, raw pointers, read_volatile/write_volatile, transmute vs. from_ne_bytes, // SAFETY: contracts, Miri, #[repr(C)] at the FFI boundary, why static mut is banned and what replaces it.

Lessons: Module 7 lessons — the contracts underneath both languages: what the C compiler is allowed to assume, what the Rust compiler proves, and how unsafe re-introduces the C obligations explicitly.

Reference (optional deep-dives): Seacord 4, 11 (Debugging, Testing, and Analysis); Rust Book 20 (Unsafe Rust); MC 47–48; the Rustonomicon as the deep reference.

Practice: Module 7 exercises — a UB safari in C caught by optimizer and sanitizer, the same programs in Rust that refuse to compile or panic, a sound unsafe wrapper with contracts verified by Miri, aliasing and alignment forensics on both toolchains, register pointers three ways, static analysis on a real Course 3 module.

Module 8 — Talking to Hardware

Theme: volatile’s exact contract; memory-mapped I/O — access width, read-modify-write hazards, write-1-to-clear, set/reset registers; bit-fields vs. mask/position macros; the linker script, .data/.bss/.rodata, startup and the vector table, weak symbols, KEEP, and the map file; in Rust — the PAC’s register API and typestate fields, cortex-m-rt and #[exception], the PAC → HAL → driver-crate layering, embedded-hal 1.0 traits, typestate GPIO, singleton peripherals, embassy-stm32’s blocking API; on Linux — the device model, the GPIO character device (libgpiod in C, gpiod in Rust), i2c-dev and spidev (linux-embedded-hal), /dev/mem and mmap only for bring-up.

Lessons: Module 8 lessons — from a register address to a portable driver, in both languages, on both the microcontroller and the Linux board.

Reference (optional deep-dives): MC 39–44; Embedded Rust Book Peripherals (singletons, typestate, the borrow checker as a hardware guard) and Static Guarantees; Embedonomicon Memory layout and Exception handling; embedded-hal 1.0 and embassy-stm32 docs; the STM32L476 reference manual for device specifics.

Practice: Module 8 exercises — volatile proven in both toolchains, fake-register-block forensics in C and Rust, a typestate LED driver, blink three ways, one I²C sensor driver generic over embedded-hal and compiled for the STM32 and the Jetson, sections and weak symbols traced in the map file, a hard-fault frame read in both languages, gpiod and I²C on the Jetson.

Part IV · Modules 9–11 — Concurrency on Each Tier

Module 9 — Interrupts and Shared State

Theme: Data races defined; the C11 memory model in one page; volatile is not atomic; critical sections via PRIMASK/BASEPRI and the blind-re-enable bug; <stdatomic.h> widths that are lock-free on the Cortex-M4 and atomic_signal_fence; ISR ↔︎ main patterns — flag, single-producer/single-consumer ring, double buffer; in Rust — Send/Sync as the type-level statement of the same rules, critical_section::Mutex<RefCell<T>>, core::sync::atomic and Ordering, #[interrupt] handlers and static state, heapless::spsc across an interrupt, RTIC resources as the principled answer; NVIC priorities; on Linux the same ideas as signal handlers and sig_atomic_t.

Lessons: Module 9 lessons — sharing state with an interrupt correctly in C, and having the compiler check it in Rust.

Reference (optional deep-dives): MC 33–35, 45; Rust Book 16 (Fearless Concurrency); Embedded Rust Book Concurrency; critical-section and heapless docs.

Practice: Module 9 exercises — a shared counter three ways in C and the Rust versions where the wrong ones fail to compile, an SPSC ring across a timer interrupt in QEMU, the lock-free atomics table by prediction, a double-buffered DMA hand-off in both languages, fence forensics, a Linux signal-handler variant.

Module 10 — RTOS and Async: FreeRTOS, RTIC, and Embassy

Theme: Why an RTOS — tasks, the scheduler, queues, semaphores and mutexes, priority inheritance, task notifications, software timers, static allocation, stack-overflow hooks, configASSERT; C patterns — ISR → queue → task, FromISR APIs, deferred interrupt processing; RTIC 2 — the Stack Resource Policy, #[shared]/#[local], priorities, hardware and software tasks, monotonics; Embassy — the executor, async fn and .await, Timer, Channel/Signal/Mutex, select, interrupt executors; the Futures and Pin essentials as they matter for no_std; FreeRTOS vs. RTIC vs. Embassy compared on preemption model, memory, latency reasoning, and tooling — with Course 3 Lab 7.2’s pipeline as the running example.

Lessons: Module 10 lessons — three concurrency runtimes for one microcontroller, and how to choose.

Reference (optional deep-dives): FreeRTOS book (tasks, queues, software timers, interrupt management, resource management); RTIC book By example; Embassy book and embassy-executor/embassy-sync docs; Rust Book 17 (Fundamentals of Asynchronous Programming).

Practice: Module 10 exercises — a three-stage producer/consumer pipeline designed in FreeRTOS-C, RTIC, and Embassy; priority-inversion reasoning; stack sizing; a latency-measurement plan with observed cells left for the bench; timers three ways and a select-based timeout.

Module 11 — Embedded Linux Systems Programming

Theme: Processes, threads, and file descriptors; errno discipline; streams vs. descriptors, buffering, binary I/O, endianness; strings handled safely; timing with CLOCK_MONOTONIC, absolute clock_nanosleep, and timerfd; event loops with epoll, signalfd, eventfd; real-time knobs — SCHED_FIFO, mlockall, CPU affinity, priority-inheritance mutexes; pthreads with C11 atomics; in Rust — std::thread, channels, Arc<Mutex<T>>, std::io/std::fs/std::time, nix and libc for what std lacks; serial and sockets; Jetson specifics — nvpmodel, jetson_clocks, cyclictest.

Lessons: Module 11 lessons — the POSIX embedded-Linux toolkit in C and the std Rust that maps onto it, with the real-time discipline both need.

Reference (optional deep-dives): Seacord 7 (Characters and Strings), 8 (Input/Output); Rust Book 12, 16, 21; nix and gpiod crate docs; Course 3’s Jetson setup essentials.

Practice: Module 11 exercises — a timerfd 1 kHz loop in C and Rust with a jitter table, a SCHED_FIFO pipeline with a priority-inversion reproduction, gpiod edge events through epoll, a serial/TCP transport in both languages, an epoll event loop, a safe-strings audit, a thread-pool shutdown both ways.

Part V · Module 12 — Engineering Practice and the Capstone

Module 12 — Program Structure, Testing, Interop, and the Quality Gate

Theme: Coupling and cohesion, opaque types, linkage and header discipline; assertions at every level; unit tests off-target with a host harness and fakes for registers; sanitizers and static analysis as the build’s quality gate; in Rust — crates, modules, visibility, cargo workspaces, features, and profiles (opt-level = "s", lto, codegen-units = 1, panic = "abort"), host tests for no_std crates, Clippy/rustfmt/docs, the binary-size ladder, defmt vs. log; interop — C from Rust (bindgen, extern "C", #[repr(C)]) and Rust from C (cbindgen, #[no_mangle]), a Rust driver inside a CubeMX C project and a C DSP kernel inside a Rust firmware, and where the safety boundary lies; the Python reference implementation as the test oracle for both.

Lessons: Module 12 lessons — structuring, testing, and shipping embedded code in either language, and mixing the two on purpose.

Reference (optional deep-dives): Seacord 10 (Program Structure), 11 (Debugging, Testing, and Analysis); Rust Book 7, 11, 14; Grenning (TDD for embedded C, dual-targeting, test doubles); Embedded Rust Book Interoperability; MC 49.

Practice: Module 12 exercises — an opaque-handle module in both languages, host-tested register fakes, the quality gate on a real module, the size and profile ladder, logging three ways, a Rust crate called from C and a C kernel called from Rust, and the capstone: one sensor driver and pipeline written three times — Python reference first, then C17 and Rust — and built for three tiers, closing on the note “My working subset, in three languages.”

Where this course leads

Module 12’s capstone is the course deliverable: the ADS1115 driver and pipeline from Course 3 Lab 3.4, prototyped in NumPy against recorded data, then written in C17 and in Rust generic over embedded-hal, built for the STM32 bare-metal, under an RTOS, and on the Jetson — with host tests that check both compiled versions against the Python reference, a size and warnings gate, and a one-page note recording the working subset of each language, the constructs banned on each tier and why, and what each language caught that the others did not. After that, Course 3’s labs are worked in the rhythm this course installs — Python first, then the tier’s language — and Course 4’s C++20 engine work inherits the same discipline about layout, aliasing, ownership, and undefined behavior with a richer toolbox.