Lab 8.5 — STM32 vs Pi 5 vs Jetson Benchmark

← Course 2 syllabus · Bonus Module 8 · Prev: « Lab 8.4 · Next: Lab 8.6 »

Goal

Run the same workload on all three platforms — the STM32 (CMSIS-DSP), the Raspberry Pi 5 (NumPy / onnxruntime), and the Jetson Orin Nano (TensorRT) — and measure latency, throughput, and power, then build a latency-vs-power Pareto view and reason about when each platform is the right choice for embedded DSP. This is the capstone of the bonus module and the payoff of the whole course: you’ll have real numbers, taken on your own bench, for the classical-microcontroller pipeline versus the two edge accelerators, and a defensible answer to “what should I run this on?” — the exact judgment a DSP/firmware engineer is paid for. It closes the loop back to the classical pipeline of Modules 5–7.

Equipment & parts

  • STM32 Nucleo-L476RG with a CMSIS-DSP build of the workload (FFT and/or the classifier).
  • Raspberry Pi 5 and Jetson Orin Nano with the Module 8 pipelines already deployed.
  • WANPTEK PSU as the measured supply for the boards you can power from it, so you can read supply current directly; and/or the Fluke 117 in series for a clean current reading.
  • A stopwatch source that is on the device: the STM32’s DWT cycle counter (CYCCNT), and time.perf_counter_ns() / CUDA events on the Pi and Jetson. Do not time over a serial print — you’ll measure the UART, not the DSP.

Wiring & bench setup

The only wiring in this lab is power-measurement plumbing: each board runs its workload standalone while you read its supply current, with the Fluke in series on the DC input.

flowchart LR
  PSU["WANPTEK PSU<br/>set to the board's input voltage"]
  FLK["Fluke 117<br/>series ammeter<br/>10 A jack"]
  BRD["Board under test<br/>running the workload loop"]
  PSU -- "+ output, red lead" --> FLK
  FLK -- "COM, black lead" --> BRD
  BRD -- "supply return −" --> PSU

flowchart LR
  PSU["WANPTEK PSU<br/>set to the board's input voltage"]
  FLK["Fluke 117<br/>series ammeter<br/>10 A jack"]
  BRD["Board under test<br/>running the workload loop"]
  PSU -- "+ output, red lead" --> FLK
  FLK -- "COM, black lead" --> BRD
  BRD -- "supply return −" --> PSU

From To Lead/jack
WANPTEK + output Fluke 10 A input jack red lead
Fluke COM board supply input + black lead
Board supply return (−) WANPTEK − output lead
  • Jetson: DC input at the voltage its PSU spec calls for; WANPTEK current limit set above the boot peak (per Safety).
  • Pi 5: wants 5.1 V / 5 A over USB-C — if the WANPTEK + series meter can’t hold that under load, use the official supply plus an inline USB-C power meter (the fallback Safety describes) and take \(V\cdot I\) from its readout.
  • STM32 Nucleo: stays USB-powered; read its 5 V current with an inline USB power meter — it’s sub-watt, so resolution matters more than method.
  • Quick reads come off the WANPTEK’s current display, trustworthy ones off the Fluke; you reconcile the two in Part C. Move the red lead back to VΩ when done (per Safety).

Safety & don’t-break-it

  • Measuring current means breaking the supply loop. To read board current with the Fluke, put it in series on the DC input (red lead in the A jack), exactly as in Lab 0.1 — never across the rail. The Pi 5 and Jetson draw amps; make sure it’s within the Fluke’s fused A range (the 10 A input), and move the lead back to VΩ when done.
  • Inrush and brown-out. The Pi 5 and Jetson have large input capacitance and real inrush; a current-limited WANPTEK set too low will brown them out and they’ll boot-loop. Set the current limit high enough for the platform’s peak (check the board’s PSU spec) but still bounded, and expect a current spike at boot.
  • Don’t power the Jetson/Pi from a sagging rail. Undervoltage on these boards causes silent throttling and corrupt filesystems. If the WANPTEK can’t hold the voltage under load, power the board from its proper supply and measure current with a clamp or an inline USB-C power meter instead.
  • Fix the clocks and thermals on the Pi and Jetson (jetson_clocks, a fan, a cool room) before timing — a thermal-throttled run makes the accelerator look worse than it is and ruins the comparison.
  • STM32 side: standard 3.3 V discipline; the Nucleo is USB-powered and low-risk.

Project & environment setup

Firmware — reuse the Module 6 project (firmware/m6-dsp/). Nothing new to configure; confirm the .ioc and build still have:

CubeMX page Setting
Clock Configuration 80 MHz HCLK per the setup essentials — this is the divisor that turns DWT CYCCNT counts into seconds
Connectivity → USART2 Asynchronous, 115200 8-N-1 — the cycle counts print here, outside the timed region
(Software, not CubeMX) CMSIS-DSP linked and the DWT cycle counter enabled, both per the setup essentials

Pi 5 — the ~/edge-venv from Labs 8.1–8.4 already has numpy (np.fft.rfft workload) and onnxruntime; scipy.signal.welch is there too if you benchmark a PSD variant. Jetson — CuPy / NVIDIA-wheel PyTorch / TensorRT exactly as set up in Labs 8.1–8.2; the GPU rows of the FFT/PSD workload use cupyx.scipy.fft / cupyx.scipy.signal, CuPy’s drop-in mirrors of the SciPy API (same calls, on-device cupy arrays). Run jetson_clocks before timing (per Safety).

Scripts (you write them): labs/lab-8-5/edge/bench_pi.py and labs/lab-8-5/edge/bench_jetson.py (warm-up, ≥1000 timed runs, throughput window, CSV writer), plus the STM32 timing loop in firmware/m6-dsp/ — each emits one row per platform/precision variant. Pull the three CSVs back to the Mac and draw the Pareto plot in the course venv (labs/lab-8-5/host/pareto.py — numpy + matplotlib, ~20 lines, you write it): mkdir -p labs/lab-8-5/host labs/lab-8-5/captures.

Keep this lab’s reconciliation in labs/lab-8-5/host/analysis.ipynb — the notebook convention — and export final figures next to it.

Where results go (one results CSV per platform, same columns — variant,latency_us_mean,p95,p99,throughput,P_idle_W,P_load_W,E_inf_J — so the comparison table has a data trail):

Artifact Path
STM32 raw serial log of cycle counts labs/lab-8-5/captures/stm32-serial.log
STM32 results labs/lab-8-5/edge/results-stm32.csv
Pi 5 results labs/lab-8-5/edge/results-pi5.csv
Jetson results (incl. batch sweep rows) labs/lab-8-5/edge/results-jetson.csv
Pareto scatter (power vs latency, frontier drawn) labs/lab-8-5/host/pareto.png
Bench note: power reconciliation + the one-page recommendation labs/lab-8-5/notes.md

Background

Pick one workload, run it everywhere. Two good choices:

  1. A 1024-point real FFT — the pure-DSP kernel, directly comparable to Lab 6.3. CMSIS-DSP arm_rfft_fast_f32 on the STM32, np.fft.rfft on the Pi, cupyx/torch on the Jetson.
  2. The keyword classifier from Lab 8.2 — a realistic learned-DSP workload (the STM32 runs only the feature stage or a tiny quantized net; the accelerators run the full model).

Latency. The per-operation wall time, measured on-device after warm-up. Report the distribution, not just the mean:

\[ \text{latency: } \ \bar{t}, \ p_{95}, \ p_{99} \ \text{ over } N \ge 1000 \text{ runs}. \]

For real time, the relevant number is the tail (\(p_{99}\)), because one late frame is a dropped frame.

Throughput. Operations completed per second under sustained load,

\[ \text{throughput} \;=\; \frac{N_{\text{ops}}}{T_{\text{total}}} \quad [\text{inferences/s or FFTs/s}]. \]

Throughput ≠ \(1/\text{latency}\) once the platform pipelines or batches (the Jetson’s batched path from Lab 8.1 has high throughput but higher single-item latency).

Power and energy per inference. Estimate board power from the DC input:

\[ P \;=\; V_{\text{in}} \cdot I_{\text{in}}, \]

with \(V_{\text{in}}\) the supply voltage and \(I_{\text{in}}\) the current read on the WANPTEK display or the Fluke in series. Measure idle power and loaded power (running the workload in a tight loop); the workload’s marginal power is \(P_{\text{load}} - P_{\text{idle}}\). The figure of merit that fairly compares a 0.5 W microcontroller to a 15 W GPU is energy per inference:

\[ E_{\text{inf}} \;=\; \frac{P_{\text{load}}}{\text{throughput}} \;=\; P_{\text{load}} \cdot \bar{t}_{\text{eff}} \quad [\text{J/inference}]. \]

A platform can be slower yet more energy-efficient, or faster but far hungrier — the Pareto point depends on whether your product is latency-bound or battery-bound.

The Pareto view. Plot each platform (and each precision variant) as a point in (power, latency). The Pareto frontier is the set of points not dominated on both axes — those are the only rational choices; everything behind the frontier is strictly worse. The STM32, Pi 5, and Jetson typically land at three different corners: low power/high latency, balanced, and low latency/high power.

Procedure

Part A — Build the identical workload on all three.

  1. Fix the exact workload (say, 1024-pt f32 rFFT) and the exact input. STM32: CMSIS-DSP with the DWT cycle counter around the transform, converted to time via the core clock (80 MHz). Pi 5: perf_counter_ns around np.fft.rfft. Jetson: CUDA events (or perf_counter_ns with torch.cuda.synchronize()), timing both single-shot and batched.
  2. Warm up each (discard the first runs — cold caches, cuFFT plan, JIT), then time ≥1000 runs and record \(\bar t\), \(p_{95}\), \(p_{99}\).

Part B — Throughput.

  1. Run each platform flat-out for a fixed wall-clock window (e.g. 10 s) and count completed operations → throughput. On the Jetson, sweep batch size and record the throughput/latency curve.

Part C — Power.

  1. For each board: at the bench, set the supply (per Wiring & bench setup) and read idle current (booted, doing nothing), then loaded current (tight workload loop). Compute \(P_{\text{idle}}\), \(P_{\text{load}}\). Use the WANPTEK’s current display for a quick read and the Fluke in series for a trustworthy one; reconcile the two.
// STM32 timing — DWT cycle counter (illustrative)
CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;
DWT->CYCCNT = 0; DWT->CTRL |= DWT_CTRL_CYCCNTENA_Msk;
uint32_t t0 = DWT->CYCCNT;
arm_rfft_fast_f32(&fft, in, out, 0);
uint32_t cyc = DWT->CYCCNT - t0;      // /80e6 -> seconds

Part D — Energy per inference and the Pareto plot.

  1. Combine: \(E_{\text{inf}} = P_{\text{load}} / \text{throughput}\) for each platform/precision. Tabulate everything.
  2. Plot (power, latency) for every platform and precision variant; mark the Pareto frontier. Annotate each point with its energy/inference.

Deliverable & expected results

  • A full benchmark table (below) and a latency-vs-power Pareto scatter with the frontier drawn.
  • A one-page recommendation: which platform for a battery sensor node, which for a low-latency perception loop, which for a fixed-function tone detector — with the numbers backing each call.
Platform / precision Latency \(\bar t\) (p99) Throughput Load power Energy/inf
STM32 (CMSIS-DSP, f32) … (…) … … …
Pi 5 (NumPy / onnx int8) … (…) … … …
Jetson (TensorRT fp16, single) … (…) … … …
Jetson (TensorRT int8, batched) … (…) … … …

Order-of-magnitude expectations to reconcile against: the STM32 has the lowest power (sub-watt) and, for a small FFT, a competitive latency because there’s no OS jitter and no host↔︎device copy — but its throughput ceiling is low. The Pi 5 has no copy overhead and a fast CPU, so at tiny model sizes it is often within striking distance of the Jetson on latency while drawing a few watts. The Jetson wins throughput and large-model latency decisively but pays the highest idle and load power and a real copy/launch overhead that makes it a poor choice for a single tiny FFT.

Analysis & reconciliation

Explain each platform’s position on the Pareto plot from first principles: the STM32 is compute-bound on a slow-but-deterministic core with no memory-hierarchy surprises (predict its FFT cycles from the Lyons \(O(N\log N)\) operation count and the M4F’s cycles-per-FLOP); the Pi 5 is a fast OoO CPU with cache effects and OS scheduling jitter (hence a wider \(p_{99}\) tail); the Jetson only shines when the workload is big enough or batched enough to hide the fixed host↔︎device and launch cost — exactly the single-frame-loses / batched-wins result you already saw in Lab 8.1. Reconcile the two power readings (WANPTEK display vs Fluke in series) and attribute any gap to the display’s coarser resolution and the meter’s burden voltage. Then state the engineering conclusion plainly: for a fixed-frequency tone detector the STM32 + Goertzel (Lab 6.5) is unbeatable on energy and determinism; for a latency-critical, large learned model the Jetson is the only option; the Pi 5 is the pragmatic middle when you want learned DSP without a GPU’s power budget. This is the whole course’s thesis in one plot — the classical embedded pipeline of Modules 5–7 is not obsolete, it’s a point on the Pareto frontier, and picking the right point for the problem is the real skill.

Going further

  • Add fixed-point vs floating-point STM32 variants (CMSIS-DSP q15/q31) and show the latency/accuracy trade Kuo describes.
  • Add a duty-cycled measurement: real sensor nodes sleep between inferences, so integrate energy over a realistic duty cycle — the STM32’s deep-sleep current can flip the energy verdict entirely.
  • Put a proper inline USB-C power meter or a shunt+scope on the Pi/Jetson for a time-resolved power trace (see the boot inrush and the per-inference power spikes).
  • Extend the Pareto plot to a third axis (accuracy, from Lab 8.2) — the real deployment decision is a 3-way latency/power/accuracy trade.