Repository navigation
Headless state dumps, offscreen screenshots and a screenshot API - #248
Merged
Merged
Conversation
Agents can now see a run without opening a window. --dump-state F [--dump-state-ticks L] writes the simulation state as JSON on the game thread right after each tick: every entity and singleton of the types the state hash covers (rollback state) plus the saved ones, with every field, the tick and its state hash. Works with --null-render. The same run writes the same bytes in every exec mode. --offscreen renders with Vulkan into images the renderer owns: no window, no surface, no GLFW, no window system extension, with the virtual clock and no frame cap of --null-render. --screenshot F [--screenshot-ticks L] saves the frame that first shows each tick as a PNG, offscreen or windowed. --resolution WxH sets the frame or window size. A capture that was asked for and not made fails the run (code 1). Also fixes two crashes on software Vulkan (llvmpipe), which agent and CI machines use. Both were also in windowed play: - The skybox pipeline declared mesh vertex input but drew with no vertex buffer (GraphicsPipeline::m_HasVertexInput). - llvmpipe's uniform inlining miscompiles the two light loops of deferred.frag; the loop bounds now depend on a value it cannot inline. The surface and swap chain Vulkan commands are optional in the loader table and checked only when there is a window. Tests: LaunchOptions and RunSettings cases, World state dump tests, the CTests "Sandbox: headless state dump" (direct and threaded dumps are byte-identical) and, with FLING_TEST_OFFSCREEN=ON, "Sandbox: offscreen screenshots". Documented in Skills/testing.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fling::Screenshot::Request(path, callback) (Core/inc/Screenshot.h) asks for the next frame the renderer draws to be saved as a PNG. It is safe on any thread and needs no Graphics header, so the editor, frame systems, tools and tests can all call it. The renderer serves every waiting request at its next frame, before the present, and runs each callback on the main thread. With no path the file goes to Saved/Screenshots/Screenshot_<local time>_<n>.png. Under --null-render, and at the end of a run, waiting requests fail with the reason, so a caller always hears back. --screenshot now goes through the same API, which replaces VulkanApp::RequestCapture. FrameCapture reads the frame once and writes one PNG per request. The editor's GPU Options window has a Take screenshot button that saves the next frame, editor UI included, and shows the path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before, every frame paid for screenshots even with none requested: - the renderer locked the request queue each frame (and so did the --null-render loop); - every swap chain image, and every offscreen image, was made with VK_IMAGE_USAGE_TRANSFER_SRC_BIT, which can turn off framebuffer compression on some GPUs. Now: - Screenshot::HasPending is one atomic load; TakePending and FailPending take no lock and allocate nothing when it is false. - Swapchain::SetCopyable decides whether its images get the copy usage. It is off by default. A run with --screenshot is copyable from the start (RenderSurfaceDesc::CopyableFrames) and never rebuilds the swap chain. Otherwise a request makes the swap chain again with copyable images for its frame (VulkanApp::PrepareScreenshots), saves it, and makes it again without them after that frame's present. - Offscreen images that are not copyable end the frame in COLOR_ATTACHMENT_OPTIMAL, as TRANSFER_SRC_OPTIMAL needs the usage. RecreateSwapchain works offscreen too. - Engine::RequestScreenshots returns at once with no --screenshot tick pending. Checked with the validation layer: --screenshot at launch, a runtime request offscreen, no request, and the GPU Options button in a window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 tasks done
A capture is now a copy in the frame's own command buffer plus a writer thread, instead of a queue-idle copy and a PNG encode on the main thread. - FrameCapturer records vkCmdCopyImageToBuffer into the frame after the render pass, into a reused, mapped staging buffer. After the frame's fence the buffer goes to the "Screenshot Writer" thread, which converts, encodes the PNG once and writes it to every requested path. Callbacks run on the main thread on a later frame. The thread and buffers are made on the first capture only. - The swap chain images get TRANSFER_SRC once (when the surface allows it; always offscreen), so a capture never rebuilds the swap chain. - Offscreen frames skip the present semaphores, removing two empty vkQueueSubmit calls per frame. - deferred.frag's llvmpipe workaround runs only on CPU drivers, through a specialization constant, so GPUs keep the plain light loops. - FlingPaths::SaveFile replaces three copies of "make the folders, then write the file". - VulkanApp::FinishScreenshots waits for the writes at the end of a run, so --screenshot still checks every file before it exits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature/Issue
Agents (and scripts) had to open a real window to see the game. This PR lets them look at a run with no window:
--dump-state <file.json>(works with--null-render, no GPU).--offscreen --screenshot <file.png>renders with Vulkan into images the engine owns. No window, no surface, no GLFW, so it runs on a machine with no display. A software driver (llvmpipe) is enough.Fling::Screenshot::Request(path, callback)takes a screenshot from anywhere in the engine, plus a Take screenshot button in the editor's GPU Options window.Implementation/Solution
--dump-state F [--dump-state-ticks L](World::SaveStateToJson)tick,simHz,entityCountandstateHash.EntityGuidget one made from their handle, so the same run writes the same bytes in every exec mode.--offscreen,--resolution WxH,--screenshot F [--screenshot-ticks L]Swapchainhas an offscreen mode that owns its images. Acquire and present become semaphore-only submits, soVulkanApp::Updateruns the same code with or without a window.Instanceskips the GLFW/surface extensions offscreen.LogicalDeviceuses the graphics queue when there is no surface.VulkanFunctions.inl, and checked only when there is a window.--null-render(virtual clock, no frame cap), so they give the same ticks.--screenshotsaves the first frame that shows each tick. It works offscreen and windowed.Fling::Screenshot(Core/inc/Screenshot.h)Requestis safe on any thread and needs no Graphics header.Saved/Screenshots/Screenshot_<local time>_<n>.png.--null-render, and when a run ends, waiting requests fail with the reason.--screenshotand the GPU Options button both use it.Two crashes fixed on software Vulkan (llvmpipe)
Both were already on
main, and windowed play crashed the same way:GraphicsPipeline::m_HasVertexInput = falsefor the skybox.deferred.frag(a null pointer read in JIT code). The loop bounds now depend on a value it can't inline; the output is the same.Docs:
Skills/testing.md("Seeing the game without a window"),AGENTS.md,Docs/agent-guide.md,Plans/improve-ai-iteration.md.Dependencies (if any?)
stb_image_writeis already vendored inexternal/stb.Screenshots/GIF (if applicable)
Run on a cloud container with no display:
It rendered the lit Sandbox (spheres, point lights, skybox) at 1280x720 in about 10 s on llvmpipe. The editor button was also tested on a virtual X display: clicking it saved the frame with the editor UI and showed the path.
Tests
Sandboxgame project and get the expected results? Offscreen, windowed (Xvfb) and--null-render, in direct, inline and threaded exec modes; screenshots and state dumps are byte-identical across modes.FlingTestsuite and ensure that there are no regressions? 674/674 locally on Linux GCC Debug, withFLING_TEST_OFFSCREEN=ON.New tests:
ScreenshotTests,WorldStateDumpTests, and new LaunchOptions/RunSettings cases.Sandbox: headless state dump(direct and threaded dumps must match byte for byte) andSandbox: state dump of a tick that does not run.Sandbox: offscreen screenshots, only with-DFLING_TEST_OFFSCREEN=ONbecause it needs a Vulkan driver.Known:
TickLimiter ticks are never earlyfailed about 1 in 15 runs locally. It is a wall-clock test of code this PR doesn't touch.🤖 Generated with Claude Code