Lab 6.3 — Vulkan Performance: RenderDoc & Timestamp Queries

← Course 4 syllabus · Module 6 · Prev: « Lab 6.2 · Next: Lab 6.4 »

Goal

The portable-API side of Lab 6.2’s skills. Two instruments carry it. RenderDoc on the Linux desktop (RTX 4090): frame capture, the event browser, pipeline-state inspection, pixel history, and the texture viewer used as a debugger — the tool the whole Vulkan world shares. And the engine’s own VkQueryPool timestamps: GPU timings around every render-graph pass, with the timestampPeriod conversion and the synchronization caveats done right, plus pipeline statistics queries feeding the HUD — because on the portable API, the always-on numbers are ones you build yourself. Nsight Graphics gets a nod as the vendor-deep alternative on NVIDIA hardware. Then the MoltenVK reality check: RenderDoc does not run on macOS — the path for the Mac’s Vulkan build is Xcode capturing the translated Metal work, plus MoltenVK’s own performance logging. That machinery makes the lab’s centerpiece possible: the three-way comparison — the same scene, per pass, on Mac-Metal, Mac-MoltenVK, and Linux-4090-Vulkan — with a translation-cost analysis. After this lab, every pass in the engine has a number on every platform, and Lab 6.4 can finally spend those numbers.

Prerequisites

  • Lab 6.2 — the Metal-native numbers that anchor the three-way table, and the HUD overlay format the Vulkan HUD must match.
  • The Module 5 scenes running on the Linux box (native Vulkan) and on the Mac under MoltenVK, at the same pinned cameras as 6.1/6.2; Lab 5.4’s GPU culling in place — the pipeline-stats sanity check audits it.
  • RenderDoc installed on the Linux desktop; the render graph’s passes already labeled with VK_EXT_debug_utils names (if any pass is unlabeled, fix that first — it is what the event browser displays).
  • 6.1’s run-header discipline, extended with the 4090’s clock settings (nvidia-smi) — a three-way table whose columns ran at unrecorded clocks is not a table.

Project & environment setup

  • RenderDoc drives the Linux workflow — launch the engine through qrenderdoc (or renderdoccmd capture over SSH for headless-style runs), trigger captures from the overlay hotkey, save .rdc files per scene:
renderdoccmd capture --opt-hook-children ./engine_app --scene city
# Mac / MoltenVK side: no RenderDoc — use Xcode GPU capture on the translated
# Metal work, plus MoltenVK's performance logging:
MVK_CONFIG_PERFORMANCE_TRACKING=1 MVK_CONFIG_PERFORMANCE_LOGGING_FRAME_COUNT=300 \
  ./engine_app --scene city   # confirm exact variable names in the MoltenVK guide
  • New engine piece: engine/vulkan/ gains a GPU timer — a query-pool ring buffer, two timestamps per render-graph pass, results read back N frames later, never a same-frame stall — and a pipeline-stats query wrapper, both feeding the same HUD overlay Lab 6.2 built for Metal and both dumping JSON to benchmarks/.
  • Feature checks before code, on both Vulkan targets: timestampComputeAndGraphics and per-queue-family timestampValidBits; pipelineStatisticsQuery in the device features; and the actual timestampPeriod value. Record all of it, including what MoltenVK does and doesn’t expose. Query, don’t assume — the HUD must degrade gracefully (feature missing → row absent, not garbage).

Where results go:

Artifact Path
Notes, feature-query record, three-way table, translation-cost investigation, sanity-check log labs/lab-6-3/notes.md
RenderDoc .rdc per scene, Xcode captures of the MoltenVK build labs/lab-6-3/captures/
Timestamp-HUD dumps, pipeline-stats dumps, MoltenVK perf logs, power logs labs/lab-6-3/benchmarks/

Background

  • Timestamp arithmetic. A timestamp query returns ticks; the device’s timestampPeriod (nanoseconds per tick) converts: \(\Delta t = (q_1 - q_0)\cdot\text{timestampPeriod}\) ns, with timestampValidBits bounding wraparound. Simple — the caveats are the curriculum.
  • The sync caveats, done right. vkCmdWriteTimestamp writes when the specified pipeline stage completes, so a top-of-pipe/bottom-of-pipe pair brackets more than the pass’s own unique work on a GPU that overlaps passes; two passes running concurrently make naive per-pass sums exceed the whole frame, which is a feature of the data, not a bug in the timer. Results must be read only after availability — fence-guarded, N frames late, with the pool region reset before reuse. And on tile-based implementations (MoltenVK over the Apple GPU included) timestamps can be coarse or quantized toward render-pass granularity — expect the Mac columns to be blunter instruments than the 4090’s, and say so in the table’s caption.
  • Pipeline statistics count what the pipeline did: input-assembly primitives, vertex/fragment invocations, clipping output. Their power is cross-checking claims — Lab 5.4 asserts its compute culling shrinks the draw stream; primitives-in versus primitives-rasterized is the audit, and fragment invocations per pixel is a first overdraw estimate a lab ahead of 6.4’s proper one.
  • RenderDoc is a replay debugger. It serializes the frame and replays it instrumented — perfect for what happened: full pipeline state per event, resource contents at any point, and pixel history (“why is this pixel this color?”) answering with every event that touched the pixel and each one’s pass/fail reason. But its timings come from replaying events, serialized and repeated, on a machine also running the debugger UI. Treat RenderDoc durations as relative attribution, cross-checked against the in-engine HUD’s live numbers — when they disagree, the disagreement is information about overlap or replay artifacts.
  • The MoltenVK layer translates Vulkan to Metal at runtime: descriptor sets to Metal’s argument model, SPIR-V to MSL (via SPIRV-Cross), render passes and barriers to Metal’s encoder and hazard model. Translation has costs — some at pipeline-creation time, some per-frame — and MoltenVK’s performance-tracking log itemizes its own internal activity per API call class. The Mac therefore yields two datapoints for the same engine code: what it costs natively (the 4090) and translated (Mac), with 6.2’s Metal-native numbers as the control arm.
  • Three-way comparison logic. Mac-Metal vs. Mac-MoltenVK isolates translation cost — same GPU, same scene, different API path. Mac-MoltenVK vs. Linux-4090-Vulkan isolates hardware architecture — same API code, TBDR unified-memory GPU vs. immediate-mode discrete GPU. Neither comparison is clean — clocks, drivers, thermal regimes, and even timestamp granularity differ — which is why every row of the table carries reasoning, not just a ratio.

Tasks

  1. RenderDoc workflow, per scene. On the Linux box, capture each Module 5 scene at its pinned camera. For each:
    • Walk the event browser against the render graph — pass names, order, and any events you didn’t know you were emitting.
    • Inspect full pipeline state for one draw per pass: formats, blend state, depth state, vertex layout — verified against intent, not assumed.
    • Run pixel history on one interesting pixel (a splat-blended pixel in 5.5 is ideal) and narrate its story in notes.md.
    • Use the texture viewer to step through the frame’s attachments; log one surprise per scene — something the capture showed that the code didn’t advertise.
  2. Timestamp HUD. Implement the query-pool timer across every render-graph pass, with correct availability handling and timestampPeriod conversion, displayed in the 6.2 HUD format and dumped to JSON. Validate against RenderDoc’s per-pass timings on the Linux box: record agreement per pass, and explain any pass where they diverge beyond noise — overlap? replay serialization? granularity?
  3. Pipeline-stats sanity checks. Wire pipeline statistics (where the feature exists) and audit:
    • Primitives submitted vs. rasterized per scene — a culling-and-clipping reality check.
    • The 5.4 claim specifically: does the indirect path’s invocation count match the visible-set size the CPU-side HUD reports? Any mismatch is a finding, not an annoyance.
  4. The three-way table. For each scene: per-pass GPU ms on Mac-Metal (from 6.2’s captures/HUD), Mac-MoltenVK (this lab’s HUD under translation), and Linux-4090-Vulkan (this lab’s HUD natively), plus whole-frame time and time-aligned power (powermetrics / nvidia-smi). This table is the lab’s centerpiece and a standing artifact the capstone report reuses — build it to be rebuilt.
  5. Translation-cost investigation. From the table, pick the pass with the largest Mac-native vs. Mac-MoltenVK gap and explain it with evidence: the Xcode capture of the translated work (what Metal objects did MoltenVK emit for this pass — more encoders? extra copies? different store actions?), the MoltenVK performance log, and, if shader-side, the SPIRV-Cross-generated MSL set against your hand-written MSL for the equivalent 6.2 pass. Write the explanation as claim → evidence → confidence.

Deliverable & expected results

  • Three annotated .rdc captures; the timestamp HUD live on both Vulkan targets with its validation note; the pipeline-stats audit; the three-way scene × pass table; the translation-cost writeup.
Quantity Predicted Measured
Timestamp HUD vs. RenderDoc per-pass timings (4090) same attribution within noise; divergences explained by overlap or replay artifacts …
Sum of per-pass timestamps vs. whole-frame GPU time plausibly exceeds the frame where passes overlap — if it does, that’s the overlap caveat made visible, not an error …
Primitives rasterized vs. submitted with 5.4 culling on consistent with the culling HUD’s visible-set claim — this is an audit, not a prediction …
MoltenVK query-feature exposure unknown until queried — record exactly which of timestamps / pipeline stats survive translation …
MoltenVK overhead, per pass type hypothesis to verify, not asserted: overhead concentrates where the API models mismatch (descriptor-heavy, pipeline-switch-heavy passes) rather than in raw draw throughput — the table decides …
4090 vs. M-series per-pass ratios unknown — no honest prior across different GPUs, drivers, and clocks; reason about architecture (TBDR vs. immediate-mode, unified vs. discrete memory paths) after measuring …

Profiling & performance

This lab is its own section; the evidence contract: captures/ holds one .rdc per scene (Linux) and one Xcode capture of the MoltenVK build per scene (Mac), named <scene>-<platform>-<api>.<ext>; benchmarks/ holds the HUD JSON dumps behind every three-way-table cell, the pipeline-stats dumps, the MoltenVK performance logs, and the power logs. No table cell without a file behind it.

Analysis & reconciliation

Reconcile the three-way table in prose: which passes are expensive everywhere (algorithmic cost), which only under translation (API-model cost), and which only on one GPU (architecture cost)? Set the 4090 columns against what Lab 6.2’s limiter analysis found for the same passes on the Apple GPU — do the two tools tell one story per pass, told in two counter vocabularies, and where they can’t be compared, why not? Grade the timestamp HUD as an instrument in §3 terms: resolution (from timestampPeriod and any observed quantization), noise floor across 300 frames at a pinned camera, and systematic bias vs. RenderDoc. Then the architecture paragraph: using Course 3 Module 2’s vocabulary, argue — from evidence, post hoc — what the measured 4090-vs-Mac ratios say about where each architecture spends its effort. Close by nominating, per platform, the single worst pass — the input Lab 6.4’s ladder starts from.

Going further

  • Run one scene under Nsight Graphics on the 4090 and compare its GPU trace against RenderDoc’s view of the same frame — what does the vendor tool see that the portable one cannot?
  • Probe for VK_KHR_performance_query — if the NVIDIA driver exposes it, hardware counters arrive in-engine the way MTLCounterSampleBuffer did on Metal, and the 6.2/6.3 HUDs converge further.
  • Capture the Raspberry Pi 5 (V3DV) as a fourth column for one scene — a second tiler, and a test of how portable the render graph’s performance assumptions really are.
  • Script renderdoccmd into the repo’s tooling so a scene capture is one make target — captures you can take lazily are captures you actually take.