Lab 6.4 — Bandwidth, Overdraw & the Optimization Ladder

← Course 4 syllabus · Module 6 · Prev: « Lab 6.3 · Next: Lab 7.1 »

Goal

The module’s synthesis. Labs 6.1–6.3 built three instruments — CPU truth, Metal-GPU truth, Vulkan-GPU truth — and each closed by nominating a worst offender. This lab spends the evidence, as a disciplined optimization ladder run on the worst-performing Module 5 scene per platform: hypothesis → single change → measure → keep-or-revert, every rung logged with before/after captures. It is the course’s predicted-vs-measured ethos turned into a loop — the same loop Lab 1.5 ran on CUDA kernels, now wearing render passes. Along the way the standard toolbox gets taught properly: overdraw (visualization, front-to-back sorting, why 5.5’s blending is the worst case), bandwidth reduction (compressed textures — BCn everywhere, ASTC where the device reports it; format narrowing continued from 3.4; vertex-attribute quantization), depth and early-z discipline (the depth pre-pass tradeoff, alpha-test breaking early-z), and state batching on 4.4’s sort keys. And one tiebreaker governs every keep-or-revert call: performance per watt. On the battery-powered, thermally constrained Mac laptop, a rung that trades milliseconds for joules is not obviously a win; on the mains-powered Linux desktop (RTX 4090) the power budget is large, but logging it is what makes the efficiency contrast a finding rather than a vibe — powermetrics and nvidia-smi sit beside the frame timer on every rung. One honesty note for the bandwidth rungs: the 4090’s raw bandwidth is enormous, so judge them by counter deltas and fractions of peak (from Nsight / the device query), not by expecting a visibly struggling frame.

Recommended reading

• Apple, “Optimizing GPU performance” and the TBDR/bandwidth-oriented WWDC sessions revisited from Lab 6.2 — this time reading for the remedies, not the diagnosis.
• KTX-Software / toktx documentation (Khronos) — the KTX 2 container, encoding options, and mip generation; whatever compressonator-class CLI you choose for direct BCn paths, read its format matrix first.
• Vulkan specification — the format sections on BC and ASTC compressed formats and the corresponding feature/format-property queries; Metal’s pixel-format documentation for the same on the Apple side. The rule of the day: feature-query, don’t assume.
• C&S — the optimization-oriented material (bandwidth, batching, GPU-driven ideas as performance tools) at title level; MbT — the performance/optimization chapters likewise (5th-ed. numbering — confirm both against the copies in hand).
• man powermetrics (Mac) and the nvidia-smi documentation — nvidia-smi dmon and the --query-gpu field list (Linux) — the exact fields being logged, so the perf-per-watt matrix cites fields, not vibes.
• Course 1 §3 — every keep-or-revert decision is a comparison of two noisy measurements; the decision rule must respect the noise floor established in 6.1.

Prerequisites

• All of 6.1, 6.2, 6.3: the zone taxonomy, benchmark baseline, limiter tables, timestamp HUD, and three-way table — plus each lab’s nominated worst pass/scene per platform.
• The Module 5 scenes at their pinned benchmark cameras; 4.4’s sort-key path switchable so batching order can be A/B’d.
• Asset toolchain access: the source (uncompressed) textures for the scenes, so compressed variants can be generated rather than faked.

Project & environment setup

• Asset pipeline addition: a tools/compress_textures script wrapping the chosen CLI (toktx or compressonator-class), emitting KTX 2 next to each source texture; loader support for compressed formats in both backends, gated on the device’s format-feature query with an uncompressed fallback.

toktx --genmipmap --t2 --encode <mode-per-your-toolchain> out.ktx2 in.png
# Power logging on every rung, time-aligned with the run window:
sudo powermetrics -i 1000 -s gpu_power,thermal   # Mac — check field names in man page
nvidia-smi --query-gpu=timestamp,power.draw,temperature.gpu,clocks.gr --format=csv -l 1 > rung.log   # Linux (4090)

• The ladder is a protocol, fixed before the first rung: pinned camera, pinned duration (e.g. 60 s), profiling build preset, thermal warm-up first, HUD dump + power log per run, one change per rung, revert if the keep-rule fails. Write the protocol at the top of notes.md before touching code.

Where results go:

Artifact	Path
The ladder log (per-rung hypothesis/evidence/verdict), derivations, final matrix	labs/lab-6-4/notes.md
Before/after captures per rung (Tracy, .gputrace, .rdc), overdraw screenshots	labs/lab-6-4/captures/
HUD dumps, power logs, texture size ledger, quality-diff images	labs/lab-6-4/benchmarks/

Background

• Bandwidth arithmetic is hand-derivable, which is why it anchors the predictions. Color-write traffic alone is

  \[B \;=\; b \times O \times W \times H \times f\]

  for \(b\) bytes per pixel, average overdraw \(O\) (fragments shaded per pixel), resolution \(W\times H\), and refresh \(f\). Worked once for scale: an RGBA16F target (\(b=8\)) at \(2560\times1440\), overdraw \(3\), \(60\) Hz is \(8\cdot3\cdot2560\cdot1440\cdot60 \approx 5.3\) GB/s — for one attachment’s writes, before texture reads, depth traffic, or blending’s read-modify-write doubling the touch count. Redo this arithmetic with your scene’s real \(b\), measured \(O\), and native resolution; the deltas predicted below fall out of the same formula.

• Overdraw is the \(O\) term, and it is measurable: a visualization pass that additively counts fragments per pixel turns it into a heat map, and the counters from 6.2/6.3 corroborate it. Opaque geometry drawn front-to-back lets early-z kill occluded fragments; blended geometry must draw back-to-front and every layer pays full freight — which is why the splat scene is the worst case by construction.

• Compressed textures attack the read side. BC7 stores a \(4\times4\) block in 16 bytes — 1 byte/pixel against RGBA8’s 4 — a 4:1 memory and bandwidth reduction at equal resolution, before the cache-hit-rate improvement that smaller footprints buy. ASTC offers finer rate control where supported. Support differs per GPU and per format — the loader queries, never assumes — and quality is judged with eyes and a difference image, not asserted.

• Vertex quantization is the same idea at the other end of the pipe: positions as 16-bit normalized ranges over a mesh’s bounds, normals/tangents packed small, against full-float baselines — bytes per vertex drops by an amount you can compute in one line, and the quality question (does the terrain crack? do normals band?) is empirical.

• Depth pre-pass buys early-z rejection for the main pass at the price of a full extra geometry pass — a trade whose sign depends on the scene: geometry-heavy scenes with cheap shaders lose; shading-heavy scenes with real overdraw can win; and on the TBDR GPU, hidden-surface removal already does related work, so the answer may differ per platform. That is a question for measurement, not doctrine. Related: alpha-test/discard breaks early-z for the draws that use it — order and separate them deliberately.

• The keep-rule, stated up front: a rung is kept only if it improves frame time or power at no frame-time cost, by more than the 6.1 noise floor, on the platform being laddered — with perf-per-watt as the tiebreaker when milliseconds and joules disagree.

Tasks

1. Overdraw visualization. Add an overdraw-count visualization mode (both backends) and capture heat maps for all three scenes; extract an average and worst-case \(O\) per scene and reconcile against the 6.2/6.3 counter evidence. This measured \(O\) feeds every bandwidth prediction below.
2. Compressed-texture pipeline. Run the scenes’ textures through the compression toolchain (BCn baseline; ASTC variants where a device reports support); wire loader support in both backends behind format queries. Record the texture-memory ledger (before/after bytes, per scene) and a visual quality diff for the worst-affected material.
3. Vertex quantization. Quantize the heaviest vertex streams (terrain is the natural target) with a computed bytes-per-vertex ledger and a quality diff (wireframe overlay and silhouette comparison). Keep the full-float path switchable for the A/B.
4. Depth pre-pass A/B. Implement a depth-only pre-pass and A/B it on two scenes where the outcomes may plausibly differ — the city scene (5.2) and the terrain (5.3) are the suggested pair. Which wins where, and does the verdict flip between the TBDR Mac and the immediate-mode 4090? Frame the expectation as a question in the log before running it.
5. The ladder. On the worst scene per platform (as nominated by 6.2/6.3): run at least five rungs, drawing on the toolbox above plus 4.4’s sort-key batching and anything the 6.1 lock evidence justified. Each rung in the log: limiter-based hypothesis (cite the capture), the single change, before/after frame time and power, verdict per the keep-rule. Reverted rungs stay in the log — a rung that loses is a finding, not a failure.
6. The matrix. Close the module with a scene × platform matrix: frame time, average power over the pinned run, and perf-per-watt (frames per joule), before Module 6 versus after the kept rungs — the quantitative summary Lab 7.2’s report format inherits.

Deliverable & expected results

• The overdraw heat maps and measured \(O\); the compressed-texture and quantization ledgers with quality diffs; the depth pre-pass A/B verdicts; the full ladder log (≥5 rungs per platform, evidence attached); the final matrix.

Quantity	Predicted	Measured
Texture memory, BC7 vs. RGBA8 (per scene ledger)	≈ 4:1 reduction on converted textures — derivable from 16 bytes per 4×4 block	…
Bandwidth saving for the laddered scene’s dominant pass	fill in from the arithmetic: recompute \(B = b\,O\,W\,H\,f\) with the post-rung \(b\) and \(O\), predict the delta before measuring	…
Depth pre-pass verdict, per scene per platform	a question, not a prediction — depends on measured overdraw and shader cost; expected to differ between the two scenes	…
Power delta from kept bandwidth rungs	direction: down — bandwidth is power; DRAM traffic is among the most energy-expensive things a frame does. Magnitude from the logs	…
Frame-time gain from front-to-back sorting (opaque)	qualitative: bounded above by the measured overdraw; near-zero where \(O\approx1\)	…

Profiling & performance

The whole module was this section; this lab’s evidence contract: every rung leaves a pair of artifacts in captures/ (before/after — Tracy trace plus the platform’s GPU capture) and a pair of power logs in benchmarks/, named rung-<k>-<change>-{before,after}.*. The matrix cites files; the ladder log cites limiters. A rung without its captures did not happen.

Analysis & reconciliation

Three reconciliations close the module. Arithmetic vs. counters: set the derived bandwidth deltas against the measured memory-traffic counters from 6.2/6.3 — where the formula overpredicts, find the term that was wrong (\(O\)? \(b\)? caching the formula ignores?). Ladder vs. limiter theory: for each kept rung, did the limiter move the way the hypothesis said — and for each reverted rung, write the one sentence naming why the hypothesis failed. Time vs. energy: does the perf-per-watt ranking match the frame-time ranking across the matrix, and where they diverge, which would you ship on battery? End with the module’s transferable statement — the same one Lab 1.5 closed on: ceiling, limiter, hypothesis, single change, re-measure — because that loop, not any tool’s UI, is what Lab 7.1 assumes as reflex.

Going further

• Try a visibility-buffer-flavored experiment: how far can the G-buffer’s \(b\) shrink before reconstruction cost eats the bandwidth win? (The 3.4 format-narrowing question, taken to its limit.)
• Evaluate Basis Universal / UASTC transcoding as the ship-one-asset answer to the BCn-vs-ASTC split the loader currently handles with two files.
• Run one kept rung’s A/B at several resolutions and refresh rates and check the bandwidth formula’s scaling predictions across the sweep — the closest this course gets to a controlled experiment on \(W\,H\,f\).