Skip to content

Frame-side frustum culling in the G-buffer pass - #249

Merged
BenjaFriend merged 7 commits into
mainfrom
feature/frustum-culling
Nov 17, 2026
Merged

BenjaFriend merged 7 commits into
mainfrom
feature/frustum-culling

Conversation

@BenjaFriend

@BenjaFriend BenjaFriend commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Feature/Issue

Every mesh in the render snapshot was drawn every frame, including meshes outside the camera's view. This adds frustum culling on the frame (main thread) side. The publish path between the game thread and the main thread is unchanged: TickPacket and RenderSnapshot are as before.

Implementation/Solution

  • Where: in the G-buffer pass, OffscreenSubpass::Draw, after the frame's camera is final. It uses the same projection × view matrix as the shader, Y flip included. A mesh outside the view skips its draw and its uniform buffer write. It keeps its proxy slot and GPU resources. Each drawable mesh is recorded once as an indirect draw (vkCmdDrawIndexedIndirect), and the CPU writes its instance count every frame: 1 in view, 0 culled. So a moving camera never makes the command buffers record again; they record only when the draw list changes, as before.
  • Why not in PublishSnapshotSystem: the camera is only finalized in VulkanApp::Update, after the frame systems run. Culling there would use the previous frame's camera and could hide a mesh that is on screen.
  • Bounds: Model computes a local bounding box from its vertices at load (Model::GetLocalBounds). The box is placed with the mesh's drawn (blended) matrix.
  • Test: Frustum (Graphics/inc/Frustum.h) takes six planes from the view-projection matrix, using Vulkan depth 0 to w. A mesh is culled only when its box is entirely outside one plane. A mesh with no bounds is never culled. MeshCuller (Graphics/inc/MeshCuller.h) applies the test to the proxy table.
  • Not culled: lights. Deferred lighting is a full-screen pass, and there is no shadow pass.
  • Setting: --frustum-culling on|off or [Engine] FrustumCulling (default on), through RunSettings and RunControls. The GPU Options window has a checkbox and a "Last frame: N meshes drawn, M outside the view" line. With --null-render the flag logs that it does nothing.
  • Vulkan validation fix: [Vulkan] EnableValidationLayers=true asked for VK_LAYER_LUNARG_standard_validation, which Khronos replaced in 2019, so the engine could not start with validation on. It now asks for VK_LAYER_KHRONOS_validation. If the layer is not installed, it logs a warning and runs without validation instead of failing to create the instance.
  • Docs: AGENTS.md, Docs/threading-model.md, Docs/time-and-ticks.md, Docs/agent-guide.md and Docs/architecture-overview.md are updated.

Dependencies (if any?)

Screenshots/GIF (if applicable)

Level_1 and Levels/PhysicsTest.json, rendered with --offscreen --screenshot at 640x360 at ticks 1, 30 and 60, with culling on and off. All six pairs are byte-identical PNGs.

A temporary log line (not committed) showed that culling was really skipping meshes:

  • Level_1: 165 drawn and 591 culled of 756 meshes, every frame.
  • PhysicsTest: 1 to 2 of 8 culled.

Tests

Vulkan validation: Level_1 (camera-turn replay) and PhysicsTest, culling on and off, with EnableValidationLayers=true: no validation errors. To check that the layer catches mistakes on the new path, a deliberately misaligned indirect offset produced VUID-vkCmdDrawIndexedIndirect-offset-02710; it was then reverted. The only messages are performance warnings about unused vertex attributes in a pipeline this PR doesn't touch.

  • Have you reviewed your own code for quality?
  • Did you the Sandbox game project and get the expected results? Sandbox --null-render --unattended --frames 120 --seed 1 exits 0, and the offscreen screenshots above match with culling on and off.
  • Did you run the FlingTest suite and ensure that there are no regressions? CTest passes 683 of 683. TickLimiter ticks are never early (Core, untouched here) fails now and then when the suite runs 8 tests at once, and passes alone.

New: GraphicsTests "[culling]", 8 test cases, 102,405 assertions:

  • The planes agree with a direct clip-space test on random points, with and without the Y flip.
  • A box inside, outside each of the six planes, straddling, behind the camera, and with a turned camera.
  • Turned and scaled boxes.
  • 400 random cameras and boxes: no culled box has a point inside the view.
  • The full path from snapshot to proxy table to culler: an off-screen mesh is skipped and a straddling one is drawn. Moving the mesh or the camera, turning culling off and destroying a mesh each update the result.
  • Aabb::Add: a box built point by point equals Aabb::Of, and a NaN point never loses a point in view.

The tests can fail: a wrong near plane (OpenGL depth) and a flipped sign in the box test were each caught.

Known limits:

  • The blend and matrix work in PublishSnapshotSystem still runs for every mesh. Removing it would need the camera finalized before the frame systems run: Cull meshes before the render snapshot's blend and matrix work #250.

  • Bounds come from CPU vertex data. Anything that moves vertices on the GPU would need larger bounds; nothing does that today.

🤖 Generated with Claude Code

BenjaFriend and others added 5 commits November 15, 2026 16:00
Agents can now see a run without opening a window.

--dump-state F [--dump-state-ticks L] writes the simulation state as JSON
on the game thread right after each tick: every entity and singleton of
the types the state hash covers (rollback state) plus the saved ones,
with every field, the tick and its state hash. Works with --null-render.
The same run writes the same bytes in every exec mode.

--offscreen renders with Vulkan into images the renderer owns: no
window, no surface, no GLFW, no window system extension, with the
virtual clock and no frame cap of --null-render. --screenshot F
[--screenshot-ticks L] saves the frame that first shows each tick as a
PNG, offscreen or windowed. --resolution WxH sets the frame or window
size. A capture that was asked for and not made fails the run (code 1).

Also fixes two crashes on software Vulkan (llvmpipe), which agent and
CI machines use. Both were also in windowed play:
- The skybox pipeline declared mesh vertex input but drew with no
  vertex buffer (GraphicsPipeline::m_HasVertexInput).
- llvmpipe's uniform inlining miscompiles the two light loops of
  deferred.frag; the loop bounds now depend on a value it cannot inline.

The surface and swap chain Vulkan commands are optional in the loader
table and checked only when there is a window.

Tests: LaunchOptions and RunSettings cases, World state dump tests, the
CTests "Sandbox: headless state dump" (direct and threaded dumps are
byte-identical) and, with FLING_TEST_OFFSCREEN=ON, "Sandbox: offscreen
screenshots". Documented in Skills/testing.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fling::Screenshot::Request(path, callback) (Core/inc/Screenshot.h) asks
for the next frame the renderer draws to be saved as a PNG. It is safe
on any thread and needs no Graphics header, so the editor, frame
systems, tools and tests can all call it. The renderer serves every
waiting request at its next frame, before the present, and runs each
callback on the main thread. With no path the file goes to
Saved/Screenshots/Screenshot_<local time>_<n>.png. Under --null-render,
and at the end of a run, waiting requests fail with the reason, so a
caller always hears back.

--screenshot now goes through the same API, which replaces
VulkanApp::RequestCapture. FrameCapture reads the frame once and
writes one PNG per request.

The editor's GPU Options window has a Take screenshot button that saves
the next frame, editor UI included, and shows the path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before, every frame paid for screenshots even with none requested:
- the renderer locked the request queue each frame (and so did the
  --null-render loop);
- every swap chain image, and every offscreen image, was made with
  VK_IMAGE_USAGE_TRANSFER_SRC_BIT, which can turn off framebuffer
  compression on some GPUs.

Now:
- Screenshot::HasPending is one atomic load; TakePending and
  FailPending take no lock and allocate nothing when it is false.
- Swapchain::SetCopyable decides whether its images get the copy usage.
  It is off by default. A run with --screenshot is copyable from the
  start (RenderSurfaceDesc::CopyableFrames) and never rebuilds the swap
  chain. Otherwise a request makes the swap chain again with copyable
  images for its frame (VulkanApp::PrepareScreenshots), saves it, and
  makes it again without them after that frame's present.
- Offscreen images that are not copyable end the frame in
  COLOR_ATTACHMENT_OPTIMAL, as TRANSFER_SRC_OPTIMAL needs the usage.
  RecreateSwapchain works offscreen too.
- Engine::RequestScreenshots returns at once with no --screenshot tick
  pending.

Checked with the validation layer: --screenshot at launch, a runtime
request offscreen, no request, and the GPU Options button in a window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The renderer now skips the draw call and uniform buffer write of each mesh
whose world box is outside this frame's camera view. The test runs in
OffscreenSubpass after the camera is final, against exactly the projection
and view the shader uses (Vulkan clip volume, depth 0..1, y flip), with the
item's drawn (blended) world matrix and the model's local AABB, which Model
now computes at load. Conservative: a mesh is culled only when its world box
is strictly outside one plane; unknown bounds are never culled.

The publish path, TickPacket and RenderSnapshot are unchanged: the snapshot
still holds every mesh, so a culled mesh keeps its proxy slot and GPU
resources. A change in the visible set re-records the cached command buffers.

- Frustum and Aabb (Graphics/inc/Frustum.h): plane extraction and box test
- MeshCuller (Graphics/inc/MeshCuller.h): per-slot visibility and stats,
  no GPU data, so it is tested without a device
- --frustum-culling on|off, [Engine] FrustumCulling (default on),
  RunControls, and a GPU Options checkbox with drawn/culled counts
- GraphicsTests "[culling]": planes against the clip test, boxes inside,
  outside each plane, straddling, behind, turned camera, turned and scaled
  boxes, a randomized never-cull-a-visible-box check, and the snapshot to
  proxy to culler path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for the frame-side frustum culling:

- Each drawable mesh is recorded once as an indirect draw. The CPU writes its
  instance count (1 in view, 0 culled) every frame, so a change of the visible
  set no longer makes every offscreen command buffer record again. They record
  only when the draw list changes, as before culling.
- The culler counts a slot as drawable with the same test the recording uses
  (IsDrawable: drawn, a model and a descriptor set), so the GPU Options count
  matches the draws.
- The renderer keeps one copy of the culling switch, in the mesh pass.
  VulkanApp passes it on; Engine::Present sets it before every frame.
- Model bounds are built point by point (Aabb::Add), with no temporary vector
  of positions. A NaN never starts the box again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t when missing

[Vulkan] EnableValidationLayers=true asked for VK_LAYER_LUNARG_standard_validation,
which the Vulkan SDK replaced with VK_LAYER_KHRONOS_validation in 1.1.106
(2019). No current SDK has it, so vkCreateInstance failed and the engine could
not start with validation on.

- Ask for VK_LAYER_KHRONOS_validation.
- When the layer is not installed, log a warning and turn validation off,
  instead of asking for it anyway and failing to create the instance.

Checked with the layer installed: the debug messenger is set up and the
offscreen runs of Level_1 (look camera replay) and PhysicsTest, culling on
and off, report no validation errors. Without the layer: the warning, then
the run goes on without validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from feature/headless-capture to main November 17, 2026 12:41
Take main's async screenshot capture (FrameCapturer) over the earlier
copyable-on-request swap chain; keep frustum culling's VulkanApp API.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@BenjaFriend
BenjaFriend merged commit 82fe1a6 into main Nov 17, 2026
14 checks passed
@BenjaFriend
BenjaFriend deleted the feature/frustum-culling branch November 17, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant