Lab 3.4 — HDR, Tone Mapping & Bloom

← Course 4 syllabus · Module 3 · Prev: « Lab 3.3 · Next: Lab 4.1 »

Goal

Give the lighting just built somewhere to live: a post-processing stack. Render the scene into an RGBA16F offscreen target — because physically based lighting produces radiance far beyond \([0,1]\), and clamping it at the framebuffer destroys exactly the information Labs 3.1–3.3 worked to compute — then map HDR to the display with tone-mapping operators (Reinhard, exposure-based, and an ACES-fitted curve, compared A/B/C), and add bloom as a bright pass plus separable Gaussian blur.

Bloom is this course’s flagship example of theory cashing out: it is Course 1 §16’s 2-D convolution, applied — the separability argument that turns \(O(k^2)\) into \(O(2k)\) per pixel is proved there and benchmarked here. Along the way: the offscreen-pass machinery (the first render-to-texture of the course), the fullscreen-triangle idiom, and the production-shaped downsample/upsample pyramid variant of bloom. The module ends with the frame a real engine starts from — geometry → HDR light buffer → post chain → display — which is exactly the structure Module 4’s render graph will formalize.

Prerequisites

  • Lab 3.3 done on both tracks — PBR + IBL is the HDR signal source; without it there is nothing worth tone mapping.
  • Lab 3.1’s linear/sRGB discipline — this lab moves the sRGB encode from “end of the lighting shader” to “end of the post chain”, and being able to say why is part of the deliverable.
  • Offscreen rendering concepts from the platform notes: on Vulkan, dynamic rendering targeting a non-swapchain image you created and own; on Metal, a render pass whose color attachment is your own texture.

Project & environment setup

  • Shaders: the scene shaders unchanged; new post-chain stages fullscreen.vert + tonemap.frag, brightpass.frag, blur.frag (GLSL) and a Post.metal counterpart set. A compute variant of the blur is a scheduled comparison, not the default path.
  • Engine plumbing — fair-game scaffolding, and the real new machinery of this lab: an offscreen RGBA16F color target + depth, sized with the swapchain and recreated on resize; on Vulkan an image barrier to sampled-layout between the scene pass and each post pass, on Metal a texture handed from one render pass descriptor to the next encoder’s binding. The swapchain/drawable format stays sRGB — the final pass alone writes it.
  • No new assets: Lab 3.3’s environment-lit scene is the input signal — the brighter its highlights, the better the bloom demo.
  • Half-float sanity check before building anything (§3): RGBA16F stores IEEE half — max finite value 65504, ~11 bits of significand. Plenty of range for radiance; limited precision for accumulation. Note in notes.md where each fact will matter in this chain.

Where results go:

Artifact Path
Notes, operator A/B/C verdicts, blur benchmark table labs/lab-3-4/notes.md
Screenshots, GPU captures of the post stack labs/lab-3-4/captures/
Naive-vs-separable timings labs/lab-3-4/benchmarks/

Background

Why HDR

A sunlit sphere from Lab 3.3 can carry radiance hundreds of times the display’s white. Clamped into an 8-bit \([0,1]\) target, every such pixel collapses to identical white — highlight shape, Fresnel gradients, and bloom’s input all destroyed before post-processing ever sees them. The fix: keep the frame in linear float until a deliberate tone-mapping step compresses it for display, and only then encode sRGB.

Operators

Reinhard is the gentlest possible statement —

\[ L_d \;=\; \frac{L}{1 + L}, \]

mapping \([0,\infty)\) into \([0,1)\): nothing ever clips, and highlights famously desaturate toward flat gray. Exposure-based mapping adds the photographic control this lab wires to a key:

\[ L_d \;=\; 1 - e^{-E\,L}, \]

with \(E\) the exposure. The ACES-fitted curve (Narkowicz’s and Hill’s fitted approximations are the standard citations) adds the filmic shoulder and toe — deeper contrast, saturated highlight rolloff — and is the closest thing real-time rendering has to a default. The comparison is empirical: same scene, same exposure, three screenshots.

One ordering fact governs the whole stack: tone mapping is nonlinear, so bloom is extracted and blurred in linear HDR, composited, and tone mapping comes after. Swapping that order is one of the scheduled failure images.

Bloom as convolution

Bloom approximates veiling glare: convolve the bright part of the image with a wide kernel. The Gaussian factors —

\[ G(x, y) \;=\; \frac{1}{2\pi\sigma^2}\, e^{-\frac{x^2 + y^2}{2\sigma^2}} \;=\; g(x)\,g(y), \qquad g(t) = \frac{1}{\sqrt{2\pi}\,\sigma}\, e^{-\frac{t^2}{2\sigma^2}}, \]

because the exponent separates — precisely §16’s separability condition (the kernel matrix has rank 1). Consequence: a \(k \times k\) convolution, \(k^2\) taps per pixel, becomes a horizontal \(k\)-tap pass followed by a vertical \(k\)-tap pass —

\[ O(k^2) \;\longrightarrow\; O(2k) \quad \text{per pixel:} \qquad \frac{k^2}{2k} = \frac{k}{2}\times \text{ fewer taps}, \]

so a 33-tap blur predicts a ~16× tap-count win. That prediction is the lab’s headline benchmark — and the measured gap between tap counts and wall time is the reconciliation.

The production-shaped variant: rather than one wide blur at full resolution, bright-pass into a mip pyramid — progressively downsample, then upsample-and-accumulate back up — getting an effectively enormous kernel from small filters at low resolution. The whole pyramid touches a geometric series of pixels:

\[ \frac{1}{4} + \frac{1}{16} + \frac{1}{64} + \cdots \;=\; \frac{1}{3} \]

extra, relative to the full-resolution image — wide-radius bloom at a bargain.

The fullscreen triangle

Post passes draw one triangle whose three vertices (generated from the vertex index — no vertex buffer at all) land at clip coordinates covering the whole screen, not a two-triangle quad. The quad’s interior diagonal makes the rasterizer’s \(2{\times}2\) pixel quads straddle two triangles along it — redundant helper-lane shading down the entire seam — and costs a second triangle’s setup for nothing. A small win, but a universal idiom: reason it through once here and use it for every post pass forever.

Tasks

Vulkan (C++20)

  1. The HDR pass. Scene renders into the RGBA16F offscreen target via dynamic rendering; a minimal tonemap.frag pass (start as a passthrough) draws the fullscreen triangle to the swapchain. First checkpoint: image identical to Lab 3.3’s, with the sRGB encode now living in the post pass. Then produce the “clamped LDR intermediate” failure image once — render the scene to an RGBA8 intermediate instead — and caption what died.
  2. Exposure + operators. Reinhard, exposure, ACES-fitted behind a runtime toggle; exposure on a key. A/B/C screenshots of the same high-contrast viewpoint (sun-facing spheres from 3.3 are ideal) at fixed exposure.
  3. Bright pass + separable blur. Threshold with a soft knee (a hard threshold flickers as pixels cross it — note why) into a half-resolution target; then the two-pass separable Gaussian, ping-ponging between two targets. Also implement the naive 2-D single-pass variant of the same kernel behind a toggle — it exists purely to be measured.
  4. The benchmark. Naive vs. separable at kernel widths ~9 / 17 / 33 at fixed resolution: frame-time table into benchmarks/, set against the \(k/2\) prediction.
  5. The pyramid. Progressive downsample/upsample chain (4–6 levels) with accumulation on the way up; composite into the tone-map pass with a strength control. Compare its look against the single wide Gaussian — the pyramid’s long soft tails are what sell it.
  6. Order postmortem. Once, deliberately: tone map before bloom extraction, screenshot the sickly result, and caption the nonlinearity argument in notes.md.

Metal (Swift)

  1. HDR pass to an RGBA16F texture via a render pass descriptor targeting it; fullscreen-triangle tone-map pass to the drawable. Same passthrough-first checkpoint, same LDR-intermediate failure image.
  2. Operators + exposure mirrored; matched A/B/C screenshots — operator character must be identical across APIs, because it is pure math.
  3. Bright pass + separable blur with ping-pong textures; naive 2-D variant behind a toggle. Note what Metal’s automatic hazard tracking is silently doing between passes that Vulkan made you spell as barriers — one sentence per pass in notes.md, feeding Module 4.
  4. The benchmark repeated on the Apple GPU — keep the table side by side with the Vulkan numbers from the Linux desktop (RTX 4090); the ratio between the two GPUs is as interesting as the ratio between the two algorithms.
  5. The pyramid mirrored; matched composite screenshots.
  6. Compute-vs-fragment question. Implement the horizontal blur once as a compute kernel using threadgroup memory to share fetched texels between neighboring pixels — does it beat the fragment version, on this GPU, at this kernel width? On TBDR hardware the answer is less obvious than CUDA intuition suggests — measure, don’t assume. One-paragraph verdict in notes.md.

Deliverable & expected results

  • Both apps: PBR scene → HDR buffer → bright pass → blur pyramid → bloom composite → tone map → sRGB swapchain, with operator/exposure/bloom-strength controls live; visually matched across APIs.
  • notes.md: the A/B/C operator verdicts, both failure-image captions, the blur benchmark table, and the compute-vs-fragment paragraph.
Quantity Predicted Measured
Separable vs. naive blur cost, \(k = 33\) taps ratio \(\tfrac{k^2}{2k} = 16.5\times\); achieved speedup lower (two passes’ fixed cost, bandwidth of the intermediate) — record both …
RGBA16F vs. RGBA8 target bandwidth 8 B/px vs. 4 B/px — 2× the traffic per read/write of the scene target …
Pyramid cost vs. one full-res blur pass downsample chain touches \(\sum 4^{-i} \approx \tfrac{1}{3}\) extra pixels; upsample similar — cheap next to any wide full-res kernel …
Reinhard vs. ACES on bright highlights Reinhard: desaturated, flat shoulder; ACES: saturated rolloff, deeper contrast — qualitative A/B …
Tone-map-before-bloom failure halos change color and weight; bloom reads as pasted-on — qualitative …

Profiling & performance

This lab is where per-pass GPU cost becomes the daily unit of thought. On the Metal side: Xcode GPU capture of one full frame — read the GPU timeline pass by pass (scene, bright pass, each blur/pyramid level, composite, tone map) and tabulate the costs; open the shader profiler on the blur to see the tap loop itself. On the Vulkan side: RenderDoc on the Linux box, the same tabulation from the event browser’s durations, and the texture viewer to step through the pyramid levels visually — watching the bright regions spread level by level is the best intuition for what the pyramid does.

Wrap each pass in a named Tracy GPU zone now — the labels persist into Module 4’s render graph, and Lab 6.3’s timestamp queries will put hard numbers behind them. Check the RGBA16F bandwidth prediction against the capture’s memory-traffic counters while you’re there.

Analysis & reconciliation

Reconcile the blur benchmark first: the tap-count model predicts \(k/2\); the measured speedup will undershoot it — attribute the gap (fixed per-pass overhead, texture-cache behavior at large radii, the intermediate target’s write bandwidth) and state which term dominates at which \(k\). Reconcile the pyramid’s measured cost against the geometric-series model the same way.

For the operators, write a verdict paragraph: which one ships in your engine and why — grounded in the A/B/C evidence, not blog consensus. Then close the module with one page: trace a single bright pixel from BRDF evaluation (3.3) through the HDR buffer, bright pass, pyramid, composite, tone map, and sRGB encode — every transformation it undergoes, and the Course 1 section that owns each one. That page is the module’s real exam.

Going further

  • Auto-exposure: average log-luminance of the frame (a parallel reduction — Module 1’s pattern, now inside a graphics API) driving \(E\), with temporal adaptation so the eye “adjusts” over a second or two.
  • Firefly suppression: a single very-bright pixel becomes a flickering blob under the pyramid — implement the standard weighted-average (Karis-average) downsample for the first level and show the before/after under camera motion.
  • Dirt masks and anamorphic stretch on the bloom composite — cheap, shippable flourishes that fall out of the existing chain.
  • Dithering before the final 8-bit quantization to kill banding in dark gradients — §8’s noise shaping in one line of theory.
  • Compare RGBA16F against the smaller B10G11R11 HDR format: measure the bandwidth win, then hunt for the precision artifacts §3 predicts — the blue channel has the fewest bits.
  • Rebuild the whole post chain as compute passes and revisit the Task 6 verdict at pyramid scale.
  • On a Mac with an EDR-capable display, investigate presenting HDR directly (CAMetalLayer’s extended-dynamic-range path) instead of tone mapping to SDR — what does the tone-mapping stage become when the display itself has headroom?