Lab 2.5 — Meshes & Blending: Loading Real Assets
← Course 4 syllabus · Module 2 · Prev: « Lab 2.4 · Next: Lab 3.1 »
Goal
Graduate from procedural quads to real assets: load glTF files through tinygltf into the engine’s own mesh and material structures — indices, attribute streams, per-primitive materials, textures — and establish the asset path that every later module streams through. Then do transparency honestly, which is rarer than it sounds: blend states on both APIs, the over operator and why it makes alpha blending order-dependent, premultiplied alpha, back-to-front sorting with depth writes off, alpha-tested cutouts as the cheap alternative, and — deliberately produced on screen — the failure modes (intersecting transparents, sort-order popping) that no per-object sort can fix and that motivate order-independent techniques (Going further, mention only). This lab closes Module 2: from here the pipeline, memory, transforms, textures, and now real scenes are all in hand, and Module 3 starts making them look like something.
Recommended reading
- tinygltf README + the glTF 2.0 quick-reference poster (Khronos) — the scene/node/mesh/primitive/accessor/bufferView object model. Thirty minutes with the poster saves hours in the debugger; glTF is JSON + binary blobs, and the accessor indirection is the only genuinely confusing part.
- vkguide.dev — the mesh-loading and glTF sections, for the engine-shaped version: what to copy out into your own structures and what to leave in the file.
- MbT — the model-loading/asset chapters and the blending/transparency material (title-level; 5th-ed. numbering — confirm against the copy in hand). MbT loads via Model I/O — note where that diverges from the tinygltf path, since this course loads identically on both APIs on purpose.
- Lengyel — the transparency/blending discussion in the shading-related chapters, topic-level: the over operator and sorting, from the mathematics side.
- C&S — the asset-pipeline/resource chapters, topic-level: where loading sits in a production engine (offline processing, not per-run parsing) — the “why” behind this lab’s Going further.
Prerequisites
Project & environment setup
- Vulkan:
labs/lab-2-5/→vk_scene; the loader lives inengine/core/(API-agnostic: it produces CPU-side structures; each backend uploads them with its own Lab 2.2 machinery). - Metal:
metal-swift/MetalScene/consuming the same asset. (Bridging the C++ loader into Swift is real work — a small C wrapper or simply parsing with a Swift glTF path is acceptable for this lab; note the choice, since Module 4’s metal-cpp backend dissolves the problem.) - Target structures (a spec — the code is yours):
Mesh= vertex/index buffers + a list ofPrimitives (index offset/count, material index);Material= base-color texture + factor, alpha mode (OPAQUE/MASK/BLEND) + cutoff;SceneObject= mesh + model matrix. Nothing more — lighting-relevant material fields wait for Module 3. - Assets into
assets/gltf/: from the Khronos glTF-Sample-Assets repository, one simple model to bring the loader up (e.g. the boxed/embedded-texture starters) and one multi-material scene for the real test; plus at least one object withBLENDmaterials — add your own transparent-material objects to the scene if the chosen asset has none. Record asset names and sources innotes.md;assets/stays out of git.
Where results go:
| Artifact | Path |
|---|---|
| Notes, asset stats table, sorting postmortem | labs/lab-2-5/notes.md |
| Wrong-then-fixed screenshot series, captures | labs/lab-2-5/captures/ |
Background
glTF in one paragraph. A glTF file is a JSON scene graph over binary buffers: a mesh holds primitives; each primitive names its attribute accessors (position, normal, uv — typed views over bufferViews over buffers) and a material; nodes give transforms. tinygltf hands you all of it parsed — the lab’s work is transcription with judgment: walk the primitives, copy attributes into Lab 2.2’s interleaved vertex layout, respect per-primitive materials (one draw per primitive, distinct texture bindings), and apply node transforms into model matrices. Two conventions to check, not assume: glTF is right-handed, Y-up, meters, and its base-color textures are sRGB — both slot into decisions Labs 2.3/2.4 already made, which is why this lab comes last.
The over operator. Blending composites a source fragment over the destination pixel (Porter–Duff “over”, from the 1984 Compositing Digital Images paper): with straight alpha,
\[ C_o = \alpha_s C_s + (1 - \alpha_s)\, C_d, \]
and with premultiplied alpha (\(C'_s = \alpha_s C_s\) stored in the texture/output):
\[ C_o = C'_s + (1 - \alpha_s)\, C_d . \]
Two facts drive everything in this lab. First, over is not commutative: red-over-blue ≠ blue-over-red, so transparent surfaces must be drawn back to front — the hardware blends in submission order, and only you know the depth order. Second, the depth buffer makes it worse if you let it: a near transparent surface drawn early writes depth and the depth test then discards the far surfaces that should have shown through it — hence the standard regime: draw all opaques (depth test + write on), then transparents sorted far-to-near with depth test on, write off. Premultiplied alpha is the associativity fix for filtering and compositing chains (and the correct form for textures with alpha — Lab 2.4’s mip generation of an alpha edge is subtly wrong without it); adopt it as the course convention.
The cheap alternative: glTF’s MASK mode — alpha test, a per-fragment discard against a cutoff. No blending, no sorting, depth writes stay on; the price is hard edges. Foliage and fences ship this way, and your scene should too where it can.
The honest limit. Per-object sorting compares one depth per object. Two intersecting transparent objects — or one large transparent object folding behind and in front of another — have no correct object order at all: some pixels need A-over-B, others B-over-A, simultaneously. You will build this case and watch it fail. Per-triangle sorting only shrinks the problem (triangles intersect too, and the sort cost explodes); the real fixes are order-independent transparency techniques — named in Going further, built much later.
Tasks
Vulkan (C++20)
- The loader. tinygltf → the target structures above (structure specified; code owner-written, per the course rule). Start with the simple model: geometry on screen via the existing pipeline, textures via Lab 2.4’s path. Then the multi-material scene: one draw per primitive, correct texture per draw. Print a load-time stats table — meshes, primitives, vertices, triangles, textures, materials by alpha mode — into
notes.md. - Blend state. Pipelines now come in three flavors from the Lab 2.1 builder: opaque (blending off), blended (over operator via the color-blend attachment state, straight and premultiplied variants — you’ll flip between them), and masked. Note where Vulkan puts blend state (baked in the pipeline) for the comparison table.
- The transparent scene. Opaque scene + several transparent objects at staggered depths (tinted glass panes are ideal — simple quads with
BLENDmaterials you add yourself). First wrong on purpose: draw transparents in arbitrary order with depth writes still on; screenshot the vanished/haloed geometry. Then fix in two moves — depth write off for the blended pipeline, then a per-object back-to-front sort by view-space depth each frame — screenshotting after each move so the series incaptures/tells the story: broken → half-fixed → correct. - Cutouts. Give one object a
MASKmaterial (a fence/foliage-style texture with binary-ish alpha) and confirm it needs no sorting and survives depth-writes-on; note the edge quality vs. the blended version of the same texture. - Break the sort. Two intersecting transparent panes. Orbit the camera and screenshot the popping/incorrect region — the failure no object sort fixes. One paragraph in
notes.mdon why (the per-pixel order argument from Background).
Metal (Swift)
- Same asset, same structures. Load the identical glTF (per the setup note on bridging), upload through the Metal path, and reproduce the full scene — the pixel-match discipline of Lab 2.3 applies to the opaque portion (transparents match only when both sorts agree — make them).
- Blend parity. Blending configured on the
MTLRenderPipelineDescriptor’s color attachment (source/destination factors and operation — mirror both straight and premultiplied variants); depth write off via a secondMTLDepthStencilState(note the asymmetry: Metal splits blend state (pipeline) from depth-write state (encoder-bindable) differently than Vulkan). Same sort, same wrong-then-fixed screenshot series.
Deliverable & expected results
vk_sceneandMetalScenerendering the same glTF scene with correct opaques, sorted transparents, and a cutout object; the broken → fixed screenshot series and intersecting-panes failure incaptures/; stats table and postmortems innotes.md.
| Quantity | Predicted | Measured |
|---|---|---|
| Draw calls for the scene | one per primitive — count from the asset’s stats table (“from the asset”) | … |
| Triangle count | from the asset’s accessors (“from the asset”) | … |
| Unsorted + depth-write-on symptom | far transparents vanish behind near ones’ depth; order-dependent tint | … |
| Sorted, write-off result | stable, order-correct layering from any camera angle — except… | … |
| Intersecting panes | per-pixel order error no sort fixes; pops as camera crosses plane | … |
| Blending bandwidth direction | up — blending is a read-modify-write against the framebuffer per covered fragment, vs. write-only opaque; direction, not a number | … |
| Per-frame sort cost (Tracy zone) | trivial at object counts this small — µs | … |
Profiling & performance
Capture the blending pass: in Xcode GPU capture, isolate the transparent draws in the encoder timeline and check the frame’s bandwidth counters against an all-opaque frame — the read-modify-write cost made visible (and a first look at how Apple’s TBDR keeps that traffic in tile memory — one sentence in notes.md, full treatment in Lab 5.2). Use the capture’s overdraw/transparency visualization to see stacked transparent layers as overdraw heat. On the Linux desktop (RTX 4090), RenderDoc’s event browser + pixel history on a doubly-covered pixel shows both blend events with inputs and outputs — the over operator, evaluated before your eyes. Full overdraw treatment — measuring it, budgeting it, killing it — is Lab 6.4; here you only learn to see it.
Analysis & reconciliation
Reconcile the stats table against the source file (the glTF JSON is readable — count primitives yourself for one mesh). Verify the over operator by hand at one pixel: pick two overlapping panes with known \(\alpha\) and colors, compute \(C_o\) for both orders, and check against the captured pixel values (RenderDoc pixel history or Xcode’s) — the non-commutativity as arithmetic, not slogan. Explain the premultiplied variant’s advantage in one paragraph (associativity; correct filtered edges) and state which convention your pipelines now assume. Close the module with the Vulkan↔︎Metal state-location table finished: blending, depth-write, winding, viewport — every piece of state, its home in each API, one line each.
Going further
- Order-independent transparency, by name only: depth peeling, per-pixel linked lists, weighted blended OIT — read one survey-level description and note in
notes.mdwhich failure from Task 5 each addresses; building one is far-future work, not this module’s. - Offline asset processing: per C&S, real engines parse glTF once into an engine-native binary. Sketch (prose only) what your
Mesh/Materialstructs would serialize to — Module 4’s asset work starts there. - Sort transparents near-to-front deliberately and describe what you see — the cheapest way to internalize why the order matters is to run it backwards.
- Try
alphaToCoveragewith MSAA on the cutout object (both APIs expose it) — the anti-aliased middle ground between test and blend.