Lab 7.1 — Capstone Build: A Streaming 3D World Viewer

← Course 4 syllabus · Module 7 · Prev: « Lab 6.4 · Next: Lab 7.2 »

Goal

Ship the engine. Everything built since Lab 0.1 converges into one running artifact: a streaming 3D world viewer — a continuous terrain landscape (Lab 5.3’s chunked-LOD quadtree, streamed on Lab 4.3’s job system) populated with instanced PBR scene content (Lab 2.5’s glTF assets shaded by Lab 3.3, drawn through Lab 4.4’s culled scene and Lab 5.4’s GPU-driven path), with at least one Gaussian-splat vignette (Lab 5.5) embedded in the world, cascaded sun shadows (Lab 5.1), the deferred pipeline with its TBDR-aware variant (Lab 5.2), the HDR post stack (Lab 3.4), and a fly camera that goes from orbit altitude down to ground level — all from one C++20 engine core through both backends (Vulkan on Mac/MoltenVK and native on the Linux desktop (RTX 4090); Metal via metal-cpp), with the in-engine HUD (Lab 6.3) always on.

No new technique is introduced here — deliberately. The skill this lab builds is the one the whole course has been circling: integration under a budget — making systems that each worked alone share a frame, on three platform configurations, without the validation layers or the profiler catching you cheating. It is also the lab where “it works” stops being a claim about code and becomes a claim about evidence: every milestone below closes with an artifact, and the flight path recorded in M5 becomes the benchmark instrument Lab 7.2 runs to measure what you shipped.

Recommended reading

This is a synthesis pass — the reading is mostly your own notes.md from Modules 4–6. Beyond that, sparingly:

• C&S — revisit the render-graph and GPU-driven-rendering chapters (title-level reference; confirm against the copy in hand), now reading them as an integrator: where do they put the seams between systems, and where did you?
• vkguide.dev — the GPU-driven-rendering and engine-architecture writeups, same purpose: a second opinion on how the pieces compose, read after your own composition exists on paper.
• Apple WWDC sessions on Metal performance fundamentals and memoryless render targets — the checklist for the TBDR-aware deferred variant your Metal backend must keep intact under integration.
• Kerbl et al. 2023, 3D Gaussian Splatting for Real-Time Radiance Field Rendering — reread the rendering section before M4; the compositing question this lab forces (splats vs. the rasterized depth buffer) is one the paper’s standalone renderer never had to answer.
• MbT — the tile-memory/deferred and performance chapters (title-level reference) if the Metal variant fights back.
• Lengyel — the view-frustum and projection chapters (title-level reference; confirm against the copy in hand), only if the cascade-fitting or culling math needs a reference while debugging M2/M3 — the theory itself is Course 1 §1, mastered.

Prerequisites

Effectively: Modules 0–6 complete. The specific load-bearing items, worth verifying before starting rather than discovering mid-integration:

• Lab 5.3’s terrain streaming on Lab 4.3’s job system, with reverse-Z already the engine’s depth convention.
• Lab 5.4’s culling + indirect-draw path running on both backends, and Lab 4.2’s render graph owning every pass and barrier.
• A trained/downloaded splat scene from Lab 5.5 and the glTF asset set from Lab 2.5 in assets/.
• The HUD (Lab 6.3) rendering timestamp/counter stats in both backends; Tracy wired since Lab 0.1.
• Linux desktop toolchain healthy (NVIDIA driver + Vulkan SDK current; native Vulkan build of a Module 5 lab runs; RenderDoc and nvidia-smi usable per Lab 6.3 / Lab 6.4).
• Both backends validation-clean / API-diagnostics-clean at their last respective labs — you cannot keep clean what didn’t start clean.

Project & environment setup

• One new executable target: world_viewer, in the course-4 workspace of the labs repo (diiv_website_custom_courses/course4/), composing existing engine/* components — the intent is that this lab adds glue and content, not engine subsystems. New engine code should be the exception and each instance justified in notes.md.
• Three build configurations must all work — the matrix Lab 7.2 measures. Keep the CMakePresets.json entries for all three current from day one:

Configuration	Backend	Where it runs	Primary tools
Mac/Metal	metal-cpp (Module 4)	Apple Silicon Mac	Xcode GPU capture, Instruments, Tracy
Mac/MoltenVK	Vulkan (Module 0)	same Mac, same engine core	validation layers, Tracy
Linux/Vulkan	Vulkan, native	Linux desktop (RTX 4090)	RenderDoc, nvidia-smi, Tracy

• A --flythrough <spline-file> mode stub can be scaffolded now (argument parsing only; the playback itself is task M5’s deliverable):

// world_viewer --flythrough labs/lab-7-1/flight.spline [--benchmark]
// (parsing scaffold only — playback and stats are yours to build)

Where results go:

Artifact	Path
Budget table, milestone log, integration postmortems, design decisions	labs/lab-7-1/notes.md
Screenshots (per milestone, per platform), parity pairs, Tracy/RenderDoc/Xcode captures	labs/lab-7-1/captures/
HUD stat dumps, flight-path spline file	labs/lab-7-1/benchmarks/

Background

What “engine-shaped” means at this scale. Every lab so far had one system on stage and the rest as scenery. Here there is no scenery: terrain streaming, GPU culling, shadow cascades, the deferred pass, splats, and post all want the same frame, the same bandwidth, the same job workers, and the same transient memory. Three disciplines separate an engine from a pile of features:

• Budgets set in advance. Before integrating anything, write the budget table below — triangle counts, draw counts, texture residency, splat count, and per-pass millisecond targets per platform — and treat it the way Lab 6.4 taught: as a hypothesis you defend or revise with evidence, never as a wish. A 60 fps frame is ≈16.6 ms; on the 4090 the temptation runs the other way — a higher target (120 fps ≈ 8.3 ms) is a legitimate choice for some content — choose, in writing, before measuring. The targets are yours to set; the point is that they exist before the first composite frame renders.

Budget line	Target — Mac/Metal	Target — Mac/MoltenVK	Target — Linux/4090	Measured
Visible triangles (ground level, worst view)	…	…	…	…
Draws / indirect-draw count after culling	…	…	…	…
Resident texture memory (MB)	…	…	…	…
Splat count in the vignette	…	…	…	…
Shadow pass (ms)	…	…	…	…
G-buffer + lighting (ms)	…	…	…	…
Splat pass (ms)	…	…	…	…
Post chain (ms)	…	…	…	…
Whole frame p95 (ms)	…	…	…	…

Two of these rows can be derived, not guessed, and deriving them is part of setting them. The frame’s time budget partitions: at a 60 fps target the per-pass targets must satisfy \(\sum_i t_i \le 16.6\,\text{ms}\) with headroom deliberately reserved — decide how much slack you hold back for integration surprises and write it down. And a first-order bandwidth estimate for the deferred core, per frame at resolution \(W \times H\), \[B \approx W \cdot H \cdot \big(b_{\text{G-buffer}} + b_{\text{depth}} + b_{\text{light}} + b_{\text{post}}\big) \cdot f\] puts a floor under the per-pass millisecond targets once divided by each platform’s measured bandwidth from Lab 6.4 — and predicts on paper how much the Metal path’s memoryless G-buffer should save. Show both derivations in notes.md.

• Integration failure modes. Systems that worked alone fight over the frame in predictable ways. Expect at least these five:
  • The terrain streamer’s uploads contend with the queue and transfer bandwidth the GPU-driven culling pass and the shadow passes also want — a hitch that only appears while moving.
  • The shadow cascades multiply the vertex and draw load that the culling budget was set without; a scene that fit the budget in the camera view blows it fourfold under the light.
  • The splat sort lands on the same job workers as chunk generation and cascade fitting; three systems each “own” the worker pool they were built against.
  • The post chain’s HDR intermediates blow the transient-memory footprint the render graph was sized for — and the TBDR variant’s memoryless savings quietly disappear if a new pass forces a G-buffer store.
  • The two backends disagree about who owns an image layout or a hazard the moment a new pass is spliced in; what synchronization validation catches on Vulkan surfaces as a subtle artifact on Metal, or vice versa.
  The render graph and the job system exist precisely so these fights surface as visible, nameable contention rather than mystery hitches — which is why the HUD and Tracy stay on for the entire build (see Profiling & performance).
• Validation as a ratchet. The rule since Lab 0.3 — validation clean at every commit, synchronization validation included, Metal API validation and shader validation on the Metal side — is hardest and most valuable exactly now. Every milestone below ends “validation clean” because an integration bug hidden today is a Lab 7.2 measurement artifact tomorrow.

The frame, as the render graph should express it:

flowchart LR
  subgraph CPU["CPU — job system (Lab 4.3)"]
    STREAM["Terrain streamer<br/>(Lab 5.3 chunks)"]
    SCENE["Scene update<br/>+ cascade fitting (5.1)"]
    REC["Command recording<br/>(render graph, Lab 4.2)"]
  end
  subgraph GPU["GPU — one graph, two backends"]
    CULL["GPU culling +<br/>indirect args (5.4)"]
    SHADOW["Cascaded shadow<br/>passes (5.1)"]
    GBUF["G-buffer + deferred lighting<br/>TBDR variant on Metal (5.2)"]
    SPLAT["Splat vignette<br/>sort + blend (5.5)"]
    POST["HDR post chain (3.4)<br/>+ HUD (6.3)"]
  end
  STREAM --> SCENE --> REC
  REC --> CULL --> SHADOW --> GBUF --> SPLAT --> POST

flowchart LR
  subgraph CPU["CPU — job system (Lab 4.3)"]
    STREAM["Terrain streamer<br/>(Lab 5.3 chunks)"]
    SCENE["Scene update<br/>+ cascade fitting (5.1)"]
    REC["Command recording<br/>(render graph, Lab 4.2)"]
  end
  subgraph GPU["GPU — one graph, two backends"]
    CULL["GPU culling +<br/>indirect args (5.4)"]
    SHADOW["Cascaded shadow<br/>passes (5.1)"]
    GBUF["G-buffer + deferred lighting<br/>TBDR variant on Metal (5.2)"]
    SPLAT["Splat vignette<br/>sort + blend (5.5)"]
    POST["HDR post chain (3.4)<br/>+ HUD (6.3)"]
  end
  STREAM --> SCENE --> REC
  REC --> CULL --> SHADOW --> GBUF --> SPLAT --> POST

Tasks

Work as integration milestones, in order — each has an acceptance criterion you can check, and each ends validation-clean with a screenshot and a HUD capture in captures/. Log every integration fight (what broke, which tool named it, what changed) in notes.md as you go; those postmortems are Lab 7.2 raw material.

0. M0 — The contract. Before integrating anything: freeze the budget table’s Target columns in notes.md (dated — revisions later are allowed but logged), inventory the assets (glTF set, environment map, heightmap source, splat scene) into assets/ with provenance noted, stand up the world_viewer target building on all three configurations, and sketch the render graph on paper — the mermaid diagram above, but yours, with the resources labeled. Accept when: all three configurations build and open a window on the Lab 5-era scene, and the budget table has no empty Target cells.
1. M1 — World base. Terrain + sky + fly camera, orbit altitude to ground, reverse-Z throughout, streaming on the job system. Accept when: 60 fps sustained at altitude on the Mac, chunk residency and LOD stats on the HUD, no streaming hitches visible in a Tracy capture of a full descent, validation clean.
2. M2 — Population. The glTF asset set placed across the terrain as an instanced scene through the GPU-driven path — culling compute, indirect draws, Lab 4.4’s CPU scene feeding Lab 5.4’s GPU side. Accept when: instance/draw counts before-and-after culling are live on the HUD, agree with a RenderDoc/Xcode capture, and the draw count stays within budget in the worst ground-level view.
3. M3 — Light & shadow. The sun, PBR shading, and cascaded shadow maps over the deferred pipeline — with the memoryless/tile-memory TBDR variant active on Metal and the classic attachment path on Vulkan. Accept when: cascade boundaries are stable during the descent (log your fitting choices), the Metal G-buffer is confirmed memoryless in an Xcode capture, and shadow + G-buffer pass times are on the HUD and within budget.
4. M4 — The splat vignette. The Gaussian-splat scene embedded at a chosen site in the world, sorted and blended correctly against the rasterized scene. This contains the genuinely hard problem of the lab: compositing splats with rasterized depth. Alpha-blended splats don’t write depth the way triangles do, yet terrain and instanced geometry must correctly occlude and be occluded by the vignette; where you place the splat pass, what it reads or writes of the depth buffer, and how sorting interacts with the deferred pipeline is a design decision — document it in notes.md as one: the options you considered, the one you chose, and the artifacts it does and doesn’t exhibit. Accept when: walking the camera around and through the vignette shows correct occlusion in both directions, on both backends, and the splat pass time is within budget.
5. M5 — Post & polish. The full HDR chain — exposure, tone mapping, bloom — over the composite, plus the deterministic camera spline: a recorded flight from orbit to ground past the vignette, replayable bit-identically (fixed timestep for playback). This spline is Lab 7.2’s benchmark instrument — build it as one (stored in benchmarks/, versioned, boring). Accept when: exposure behaves through the sky-to-ground brightness range, and a Tracy capture of a full spline run shows clean pacing — no recurring hitch you can’t name.
6. M6 — Second-backend parity. Whichever backend led, bring the other to parity: same scene, same spline, pixel-level screenshot pairs at fixed spline stations, a HUD-vs-HUD parity table (pass times, counts) in notes.md. Accept when: screenshot pairs differ only where documented and explained (blend-order differences at the splat pass are the expected suspect — say so with evidence, not hand-waving), and both backends run the full spline validation clean.
7. M7 — Linux port (RTX 4090). The native-Vulkan build on the Linux desktop: feature queries re-run and diffed against MoltenVK’s (record what each platform lacks or adds — expect desktop NVIDIA to expose much more, multiDrawIndirect and deep descriptor indexing included), budgets applied from the Linux/4090 column, content scaled by the budget table not by improvisation. Accept when: the full spline runs end-to-end on the 4090 at your chosen target, validation clean, with an nvidia-smi log captured alongside.

Deliverable & expected results

• world_viewer running the same world through both backends and all three platform configurations, HUD on, validation clean.
• The flight-path spline in benchmarks/, replayable deterministically — the contract with Lab 7.2. Choose the path so it passes through three stations worth naming now, because Lab 7.2 benchmarks at exactly these:

Station	Where	What it stresses
S1 — Altitude	orbit height, whole landscape in view	LOD selection, cascade coverage, streaming breadth
S2 — Ground, worst view	ground level, densest instanced content in frame	culling + indirect draws, shadow density, overdraw
S3 — The vignette	inside/around the splat scene	sort + blend cost, depth compositing, bandwidth

• Per-milestone screenshots and the M6 parity pairs in captures/; the completed budget table (targets owner-set, Measured still “…” until Lab 7.2’s disciplined runs) and all integration postmortems in notes.md.

Where numbers aren’t derivable in advance, the predicted-vs-measured table is the milestone acceptance ledger:

Check	Predicted / acceptance criterion	Measured
M1 sustained fps at altitude (Mac)	60, no streaming hitches in Tracy	…
M2 draw count, worst ground view	within owner-set budget; HUD agrees with capture	…
M3 Metal G-buffer	memoryless in Xcode capture; pass times within budget	…
M4 splat/depth compositing	correct occlusion both directions, both backends	…
M5 spline replay	deterministic; pacing clean in Tracy	…
M6 parity pairs	differences enumerated and explained, none mysterious	…
M7 Linux (4090) spline run	end-to-end at chosen target, validation clean	…

Profiling & performance

One rule, kept for the whole lab: the HUD stays on and Tracy stays attached while you integrate — not as a final check but as the ambient instrument, so the moment two systems start fighting over the frame you see which pass grew rather than discovering a slow frame later. Save a Tracy capture at every milestone boundary (captures/mX-<platform>.tracy); these before/after pairs are the integration story Lab 7.2 tells. Deep-dive tools (Lab 6.2’s Xcode capture, Lab 6.3’s RenderDoc) come out on demand when a milestone’s numbers miss budget.

Formal benchmarking waits for Lab 7.2 — resist tuning past the budget here. The discipline is asymmetric on purpose: investigate every budget miss now, while the cause is one milestone old, but optimize only to budget; the ladder beyond it belongs to the report, where each rung gets before/after evidence instead of a hurried fix.

Analysis & reconciliation

In notes.md, three reconciliations.

Budgets vs. first contact. For each budget line that a milestone blew, say whether the target was wrong or the implementation was — and which evidence decided it. Reconcile the Background section’s paper bandwidth estimate against what the counters actually showed at station S2; a factor-of-two miss usually means a resource you forgot is being read or stored.

The integration ledger. List every fight between systems that worked alone — who contended for what, which tool named it, what changed — and reconcile against the failure modes predicted in Background. The ones you didn’t predict are the interesting entries; each becomes a paragraph in Lab 7.2’s retrospective.

The splat decision. Write the M4 design decision up properly — alternatives considered, the choice, the residual artifacts it accepts — leaning on Course 1 §1 for the projective geometry of the depth comparison and Course 1 §3 for why reverse-Z changes the floating-point terms of that comparison.

Close with the sentence Lab 7.2 will open with: what this engine is, in one paragraph, and what you expect the three platforms to reveal about it.

Going further

• A day/night cycle — the sun angle sweeping the cascades and the exposure range, a stress test of both M3 and M5 decisions.
• A second streaming ring: stream the instanced scene (not just terrain chunks) by distance, retiring the assumption that all instances are resident.
• Add the Raspberry Pi 5 (V3DV) as a fourth configuration and see which budget line breaks first — the syllabus’s optional third port target, at maximal stress.
• A photo mode: freeze the frame, free the camera, and dump a super-resolution capture — a small feature that touches an unreasonable number of systems, which is the point.
• A minimal in-viewer debug console (toggle passes, freeze culling, visualize cascades) — the Lab 6.3 HUD grown one notch toward the tooling real engines carry.