diff --git a/README.md b/README.md index 1498b6d..9663473 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Experimental source patches and tools for running the **macOS League of Legends client on x86-64 Linux through Darling**. -This is a development snapshot, not a finished launcher. Riot and League clients can launch and manually played ARAM games have run. Graphics, audio, input, startup crashes and memory reclamation have received compatibility fixes. On the Ryzen 5 5500 / RX 6600 PC, ARAM now averages about 120–127 FPS, with the slowest 10-second stretches around 103–110 (fights were 70–80 FPS before the 2026-10-13/14 work, versus 170–190 on Windows). The effect of the memory changes on client RAM remains unmeasured. A fresh checkout has not been validated end to end on another machine. +This is a development snapshot, not a finished launcher. Riot and League clients can launch and manually played ARAM games have run. Graphics, audio, input, startup crashes and memory reclamation have received compatibility fixes. On the Ryzen 5 5500 / RX 6600 PC, ARAM now averages about 118–135 FPS, with the slowest 10-second stretches around 106–119 (the higher figures with Hyprland's direct scanout off; fights were 70–80 FPS before the 2026-10-13/14 work, versus 170–190 on Windows). The effect of the memory changes on client RAM remains unmeasured. A fresh checkout has not been validated end to end on another machine. ## Included @@ -64,7 +64,7 @@ Start with standalone regression checks, then play a match manually. See [testin FPS counters, game presentation requests and physical display refresh are different measurements. Record average and p99 frame times, the active scene, graphics settings, charger/power state and whether the game stayed focused. See [current status](docs/STATUS.md). -On the Ryzen 5 5500 / RX 6600 test PC, ARAM on Hexgate went from 84 to 127 FPS average over 2026-10-13/14, with the slowest 10-second stretches around 110 ([status](docs/STATUS.md#game-fps-2026-10-1314)). The game-process speed switches and their opt-out files are listed in [setup](docs/SETUP.md#game-process-switches), and passive match capture is described in [testing](docs/TESTING.md#passive-match-capture). +On the Ryzen 5 5500 / RX 6600 test PC, ARAM on Hexgate went from 84 to 135 FPS average over 2026-10-13/14, with the slowest 10-second stretches around 119 in the best match (one match so far with Hyprland's direct scanout off; [status](docs/STATUS.md#game-fps-2026-10-1314)). The game-process speed switches and their opt-out files are listed in [setup](docs/SETUP.md#game-process-switches), and passive match capture is described in [testing](docs/TESTING.md#passive-match-capture). For existing installations, [incremental update instructions](docs/SETUP.md#updating-an-existing-runtime) explain the server and memory installers, user-owned backups and the full Darling restart needed to activate replacements. Perform activation only after manually exiting the game and both clients. diff --git a/docs/SETUP.md b/docs/SETUP.md index 4a3a863..ff53143 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -89,6 +89,18 @@ sh build/runtime/tools/install-riot-metal-pair.sh The installer copies `build/riot-Metal` and `build/riot-libindium.dylib` into the guest `RuntimeFrameworks/Metal.framework` and `RuntimeLibraries`. It backs up the pair it replaces in `build/metal-pair-backups`; a repeated install keeps the older backup. `sh build/runtime/tools/install-riot-metal-pair.sh --rollback` restores that backup. `tools/install-riot-gpu-runtime.sh` still builds and installs the whole GPU set, including these two. Activation needs the namespace restart below. +Indium also contains experimental hand-off changes. They are **off by default**: without the files below, libindium behaves like the build measured before them, and two matches with them on had lower FPS. Each one is a guest file, read once per game process when the game starts. + +| Guest file | Effect when present | +|---|---| +| `/tmp/metal-present-dispatch-wait` | a presenting commit waits only for its own present dispatch on the submit thread, instead of the whole submit queue | +| `/tmp/metal-deferred-blit` | blit encoders are recorded on the game thread and replayed on the submit thread (only while deferred render encoding is on) | +| `/tmp/metal-worker-lease` | the Vulkan command buffer is set up by the first thread that records into it (the submit thread) instead of when `MTLCommandBuffer` is created | +| `/tmp/metal-skip-cpu-devices` | software Vulkan devices (lavapipe) are skipped while a hardware device exists | +| `/tmp/metal-debug-waits` | diagnostic: counts how often and how long threads block at the hand-off waits and in `nextDrawable`, every 5 s in `/tmp/metal-trace.log`; the first check writes a `waits enabled` line | + +Create these files from inside the guest, for example `darling shell touch /tmp/metal-debug-waits`, or while Darling is shut down. The guest `/tmp` is `~/.darling/private/tmp` seen through Darling's overlay mount. A file created on the host in that directory while Darling runs may stay invisible to guest processes that already looked for it. The same applies to removing a file. The older `/tmp/metal-no-present-dispatch-wait`, `/tmp/metal-no-deferred-blit` and `/tmp/metal-no-skip-cpu-devices` files are ignored. + ### Game-process switches The kernel library decides these once when a process starts, and only for the League game process (`.../LeagueofLegends.app/Contents/MacOS/LeagueofLegends`). The Riot and League clients keep the server path. Each switch is a host file, so it takes effect at the next game start without reinstalling: diff --git a/docs/STATUS.md b/docs/STATUS.md index 62e01ad..a0f28f2 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -16,9 +16,17 @@ All matches were ARAM, played manually with unchanged game settings; FPS is the | 10-14 14:18 | Hexgate | + core isolation (since made opt-in) | 102.9 | 92.0 | | 10-14 14:42 | Bilgewater | local mutexes, condition variables and semaphores | 119.5 | 103.4 | | 10-14 16:26 | Hexgate | + Metal bind deduplication and cheaper blits | 126.7 | 109.6 | +| 10-14 20:07 | Hexgate | + Indium hand-off changes (since made opt-in) | 114.8 | 102.9 | +| 10-14 20:27 | Bilgewater | same | 107.2 | 97.2 | +| 10-14 23:19 | Hexgate | hand-off removed; + vertex-descriptor dirty masks | 118.5 | 106.0 | +| 10-14 23:48 | Hexgate | same, Hyprland `render:direct_scanout = 0` | 135.4 | 118.7 | The 13:04 and 16:26 Hexgate matches were captured with the passive sampler (`tools/riot-match-capture`). Per frame, the game's main thread waited 1.36 ms on darlingserver in the first and 0.10 ms in the second. It blocked 39 times per frame in the first and 14 in the second. darlingserver CPU fell from 82% to 61% of a core. GPU utilization stayed at 70–74%: the main thread is the limit and is now about 90% busy running code. Frame times within each 10-second window vary by about 1–1.4 ms (standard deviation) and never more than 2.7 ms, versus hitch windows of 77–168 ms in the matches of 10-13 morning. +The Indium hand-off round moved command-buffer setup, blits and the frame-end synchronisation onto Indium's submit thread. In both of its matches the main thread waited on futexes 0.77–0.79 ms per frame instead of about 0.5 ms, and FPS fell below every earlier match on the same map, so those changes are now opt-in (see [setup](SETUP.md#metal-and-indium-overlays)). + +The 23:48 match ran the same Darling build as 23:19, with Hyprland's direct scanout turned off to fix a cursor that stopped changing shape in fullscreen (see [troubleshooting](TROUBLESHOOTING.md#cursor-stops-changing-shape-in-fullscreen)). Over the first ten minutes of each match, average FPS rose from 120.6 to 136.3 and the slowest quarter from 107.3 to 118.8; main-thread CPU per frame fell from 7.1–7.7 ms to 6.5 ms. That match lasted 11 minutes, so one more match is needed before crediting the whole gain to the compositor setting. + What changed in the runtime (all in the patches): - **Process-local pthread mutexes and condition variables** (`psynch_mutexwait.c`, `bsdthread_register.c`): contended first-fit mutexes and their condition variables sleep on futexes inside the game instead of making two darlingserver round trips. On by default in the League game process only. diff --git a/docs/TESTING.md b/docs/TESTING.md index 05aae9a..c2d2d0c 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -226,6 +226,14 @@ sh tools/riot-metal-bind-model/run.sh The bind-tracking helpers are extracted verbatim from `MTLRenderCommandEncoder.mm`. The model drives two encoder states, one receiving every call and one only the forwarded calls, and requires them to be equal after every call. That covers directed scenarios, among them a freed wrapper whose address is reused by a different resource and depth-stencil retention, plus 119,401 random calls. Expected: `randomized calls=119401 forwarded=110294 dropped=9107 (7.6%)`, `BIND_DEDUP_MODEL failures=0` and `BLIT_CAST_MODEL failures=0`. +## Indium hand-off tests + +```sh +sh tools/run-deferred-handoff-tests.sh +``` + +Both tests compile the real Indium private headers with the host compiler. `test-deferred-handoff` checks four things. The submit thread runs tasks in post order. The present-dispatch token returns while a later task is still queued, and `drain()` waits for that task. The wait counters write nothing without `/tmp/metal-debug-waits` (checked in a forked child). With the flag, they write one `waits enabled` line and then count only waits that actually blocked. `test-deferred-encoder-model` checks the deferred blit and render hand-off: order, resource lifetime and the drop after commit. The test creates the host `/tmp/metal-debug-waits` for its run and removes it afterwards. In the runtime these mechanisms are opt-in (see [setup](SETUP.md#metal-and-indium-overlays)). Expected: `all host deferred-hand-off tests passed`. + ## Passive match capture These tools observe a manually played match from the host. They wait for the game process, stop when it exits, and never attach to it, write to it, or read its memory. diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 84eda54..dc16ed6 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -87,3 +87,24 @@ If FPS drops back towards the 10-13 morning numbers (75–85 on Hexgate): 3. Capture a match with `tools/riot-match-capture/sample-match.py` and compare it with an earlier one using `compare-matches.py` ([testing](TESTING.md#passive-match-capture)). A server wait above about 0.5 ms per frame means one of the local paths is off. A runqueue wait above a few percent on the main thread means the CPU is oversubscribed, for example by `/tmp/rclol-core-isolation` or background load. Compare matches on the same map: Bilgewater runs a few percent below Hexgate. + +On Hyprland, also check that direct scanout is off (`hyprctl getoption render:direct_scanout` prints `int: 0`). The first match with it off ran about 16 FPS faster than the previous one on the same build; see the next section. + +## Cursor stops changing shape in fullscreen + +On Hyprland the game's cursor can keep its first shape (for example, it never switches to the attack cursor) until you leave and re-enter fullscreen. Darling sends every change: `tools/riot-cursor-watch/cursor-watch.c` logs the X server's displayed cursor and saw about 1,500 changes in a 12-minute match. What freezes the shape on screen is Hyprland's direct scanout of the fullscreen Xwayland window. Turn it off in `hyprland.conf`: + +``` +render { + direct_scanout = 0 +} +``` + +`hyprctl keyword render:direct_scanout 0` applies it until Hyprland restarts. To check the X side yourself, build and run the watcher while playing: + +```sh +cc -O2 -o build/riot-cursor-watch tools/riot-cursor-watch/cursor-watch.c -lX11 -lXfixes +build/riot-cursor-watch +``` + +It prints one line per cursor change. It only listens for X server notifications and does not touch the game. diff --git a/patches/cocotron.patch b/patches/cocotron.patch index 49999b2..a5d98de 100644 --- a/patches/cocotron.patch +++ b/patches/cocotron.patch @@ -6822,12 +6822,17 @@ index d6f86b31..81a4ac10 100644 static void glDebugCallback(GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, const void* context) { diff --git a/QuartzCore/CAMetalLayer.mm b/QuartzCore/CAMetalLayer.mm -index b81e2e14..7b9891fb 100644 +index b81e2e14..63533444 100644 --- a/QuartzCore/CAMetalLayer.mm +++ b/QuartzCore/CAMetalLayer.mm -@@ -26,6 +26,74 @@ +@@ -25,7 +25,79 @@ + #import #import ++// Wait-site counters (flag file /tmp/metal-debug-waits, checked once at process start): the same summary the Indium ++// hand-off waits write, so the drawable hand-out and the command-buffer hand-offs appear in one histogram. ++#include ++ #include +#include +#include @@ -6900,7 +6905,7 @@ index b81e2e14..7b9891fb 100644 #import "CALayerInternal.h" -@@ -205,6 +273,65 @@ static void reportGLErrors(void) { +@@ -205,6 +277,65 @@ static void reportGLErrors(void) { @end @@ -6966,7 +6971,7 @@ index b81e2e14..7b9891fb 100644 @implementation CAMetalLayerInternal #if DARLING_METAL_ENABLED -@@ -363,6 +490,7 @@ static void reportGLErrors(void) { +@@ -363,12 +494,15 @@ static void reportGLErrors(void) { - (id)nextDrawable { std::shared_ptr drawable = nullptr; @@ -6974,7 +6979,15 @@ index b81e2e14..7b9891fb 100644 [_drawableCondition lock]; -@@ -380,6 +508,8 @@ static void reportGLErrors(void) { + NSDate* endTime = [NSDate dateWithTimeIntervalSinceNow: 1]; + + while (_usableDrawablesBitmap == 0) { ++ // No drawable is free: this thread really blocks here (the wait is what the pool starvation costs the game). ++ Indium::Waits::Block block("nextDrawable"); + if (_allowsNextDrawableTimeout) { + if (![_drawableCondition waitUntilDate: endTime]) { + break; +@@ -380,6 +514,8 @@ static void reportGLErrors(void) { if (_usableDrawablesBitmap == 0) { // timeout reached @@ -6983,7 +6996,7 @@ index b81e2e14..7b9891fb 100644 [_drawableCondition unlock]; return nil; } -@@ -400,6 +530,8 @@ static void reportGLErrors(void) { +@@ -400,6 +536,8 @@ static void reportGLErrors(void) { } _usableDrawablesBitmap &= ~(1u << drawableID); @@ -6992,7 +7005,7 @@ index b81e2e14..7b9891fb 100644 drawable = _drawables[drawableID]; -@@ -410,20 +542,36 @@ static void reportGLErrors(void) { +@@ -410,20 +548,36 @@ static void reportGLErrors(void) { - (void)queuePresent: (NSUInteger)drawableID { @@ -7030,7 +7043,7 @@ index b81e2e14..7b9891fb 100644 [_drawableCondition lock]; _usableDrawablesBitmap |= 1u << drawableID; [_drawableCondition signal]; -@@ -455,7 +603,7 @@ static void reportGLErrors(void) { +@@ -455,7 +609,7 @@ static void reportGLErrors(void) { } _usableDrawablesBitmap = 0; @@ -7039,7 +7052,7 @@ index b81e2e14..7b9891fb 100644 // can't create new drawables [_drawableCondition unlock]; return; -@@ -506,6 +654,10 @@ static void reportGLErrors(void) { +@@ -506,6 +660,10 @@ static void reportGLErrors(void) { glTextureParameteri(_tex, GL_TEXTURE_MIN_FILTER, GL_NEAREST); glTextureParameteri(_tex, GL_TEXTURE_WRAP_S, GL_REPEAT); glTextureParameteri(_tex, GL_TEXTURE_WRAP_T, GL_REPEAT); @@ -7050,7 +7063,7 @@ index b81e2e14..7b9891fb 100644 reportGLErrors(); } } -@@ -516,6 +668,10 @@ static void reportGLErrors(void) { +@@ -516,6 +674,10 @@ static void reportGLErrors(void) { - (void)prepareRender { std::shared_ptr drawable = nullptr; @@ -7061,7 +7074,7 @@ index b81e2e14..7b9891fb 100644 [_drawableCondition lock]; if (_queuedDrawableCount > 0) { -@@ -528,11 +684,62 @@ static void reportGLErrors(void) { +@@ -528,11 +690,62 @@ static void reportGLErrors(void) { if (drawable) { // wait for the drawable to be fully rendered before copying it to the render texture // (this is a GPU-side wait) @@ -7124,7 +7137,7 @@ index b81e2e14..7b9891fb 100644 } @synchronized(self) { -@@ -575,6 +782,11 @@ static void reportGLErrors(void) { +@@ -575,6 +788,11 @@ static void reportGLErrors(void) { { [super _setContext: context]; @@ -7136,7 +7149,7 @@ index b81e2e14..7b9891fb 100644 if (_tex == 0) { // create our textures now using the context's CGLContext CGLContextObj prev = CGLGetCurrentContext(); -@@ -589,10 +801,15 @@ static void reportGLErrors(void) { +@@ -589,10 +807,15 @@ static void reportGLErrors(void) { glTextureParameteri(_tex, GL_TEXTURE_MIN_FILTER, GL_NEAREST); glTextureParameteri(_tex, GL_TEXTURE_WRAP_S, GL_REPEAT); glTextureParameteri(_tex, GL_TEXTURE_WRAP_T, GL_REPEAT); diff --git a/patches/metal--deps--indium.patch b/patches/metal--deps--indium.patch index b8fcadc..8f0081c 100644 --- a/patches/metal--deps--indium.patch +++ b/patches/metal--deps--indium.patch @@ -37,10 +37,78 @@ index 70f2b28..2c2e579 100644 class Library { diff --git a/include/indium/render-command-encoder.hpp b/include/indium/render-command-encoder.hpp -index d17993c..c0b163b 100644 +index d17993c..0458265 100644 --- a/include/indium/render-command-encoder.hpp +++ b/include/indium/render-command-encoder.hpp -@@ -71,5 +71,7 @@ namespace Indium { +@@ -19,7 +19,16 @@ namespace Indium { + public: + virtual ~RenderCommandEncoder() = 0; + +- virtual void setRenderPipelineState(std::shared_ptr renderPipelineState) = 0; ++ // (darling-lol:) The resource parameters are passed by const reference rather than by ++ // value: a by-value std::shared_ptr parameter makes the *caller* copy the pointer (an ++ // atomic increment plus the temporary's increment/decrement pair) and hands the callee a ++ // private address to copy out of. Every implementation here copies the pointer once into ++ // the table or op record that has to keep the resource alive anyway, so the reference ++ // removes one atomic round trip and one store-to-load chain per forwarded bind. These ++ // signatures are part of the Metal<->Indium pair (both are built together and installed ++ // together, see patches/metal.patch and patches/metal--deps--indium.patch), and no other ++ // runtime module calls a render encoder. ++ virtual void setRenderPipelineState(const std::shared_ptr& renderPipelineState) = 0; + virtual void setFrontFacingWinding(Winding frontFaceWinding) = 0; + virtual void setCullMode(CullMode cullMode) = 0; + virtual void setDepthBias(float depthBias, float slopeScale, float clamp) = 0; +@@ -31,7 +40,7 @@ namespace Indium { + virtual void setScissorRects(const ScissorRect* scissorRects, size_t count) = 0; + virtual void setScissorRects(const std::vector& scissorRects) = 0; + virtual void setBlendColor(float red, float green, float blue, float alpha) = 0; +- virtual void setDepthStencilState(std::shared_ptr state) = 0; ++ virtual void setDepthStencilState(const std::shared_ptr& state) = 0; + virtual void setTriangleFillMode(TriangleFillMode triangleFillMode) = 0; + virtual void setStencilReferenceValue(uint32_t value) = 0; + virtual void setStencilReferenceValue(uint32_t front, uint32_t back) = 0; +@@ -41,35 +50,37 @@ namespace Indium { + virtual void drawPrimitives(PrimitiveType primitiveType, size_t vertexStart, size_t vertexCount, size_t instanceCount) = 0; + virtual void drawPrimitives(PrimitiveType primitiveType, size_t vertexStart, size_t vertexCount) = 0; + +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) = 0; +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount) = 0; +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset) = 0; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) = 0; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount) = 0; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset) = 0; + + virtual void setVertexBytes(const void* bytes, size_t length, size_t index) = 0; +- virtual void setVertexBuffer(std::shared_ptr buffer, size_t offset, size_t index) = 0; ++ virtual void setVertexBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) = 0; + virtual void setVertexBuffers(const std::vector>& buffers, const std::vector& offsets, Range range) = 0; + virtual void setVertexBufferOffset(size_t offset, size_t index) = 0; +- virtual void setVertexSamplerState(std::shared_ptr state, size_t index) = 0; +- virtual void setVertexSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) = 0; ++ virtual void setVertexSamplerState(const std::shared_ptr& state, size_t index) = 0; ++ virtual void setVertexSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) = 0; + virtual void setVertexSamplerStates(const std::vector>& states, Range range) = 0; + virtual void setVertexSamplerStates(const std::vector>& states, const std::vector& lodMinClamps, const std::vector& lodMaxClamps, Range range) = 0; +- virtual void setVertexTexture(std::shared_ptr texture, size_t index) = 0; ++ virtual void setVertexTexture(const std::shared_ptr& texture, size_t index) = 0; + virtual void setVertexTextures(const std::vector>& textures, Range range) = 0; + + virtual void setFragmentBytes(const void* bytes, size_t length, size_t index) = 0; +- virtual void setFragmentBuffer(std::shared_ptr buffer, size_t offset, size_t index) = 0; ++ virtual void setFragmentBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) = 0; + virtual void setFragmentBuffers(const std::vector>& buffers, const std::vector& offsets, Range range) = 0; + virtual void setFragmentBufferOffset(size_t offset, size_t index) = 0; +- virtual void setFragmentSamplerState(std::shared_ptr state, size_t index) = 0; +- virtual void setFragmentSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) = 0; ++ virtual void setFragmentSamplerState(const std::shared_ptr& state, size_t index) = 0; ++ virtual void setFragmentSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) = 0; + virtual void setFragmentSamplerStates(const std::vector>& states, Range range) = 0; + virtual void setFragmentSamplerStates(const std::vector>& states, const std::vector& lodMinClamps, const std::vector& lodMaxClamps, Range range) = 0; +- virtual void setFragmentTexture(std::shared_ptr texture, size_t index) = 0; ++ virtual void setFragmentTexture(const std::shared_ptr& texture, size_t index) = 0; + virtual void setFragmentTextures(std::vector>& textures, Range range) = 0; + + virtual void useResource(std::shared_ptr resource, ResourceUsage usage, RenderStages stages) = 0; virtual void useResources(const std::vector>& resources, ResourceUsage usage, RenderStages stages) = 0; virtual void useResource(std::shared_ptr resource, ResourceUsage usage) = 0; virtual void useResources(const std::vector>& resources, ResourceUsage usage) = 0; @@ -144,18 +212,33 @@ index a348b2b..9ad8975 100644 }; }; diff --git a/private-include/indium/command-buffer.private.hpp b/private-include/indium/command-buffer.private.hpp -index b7ffcce..4645377 100644 +index b7ffcce..5b1e84c 100644 --- a/private-include/indium/command-buffer.private.hpp +++ b/private-include/indium/command-buffer.private.hpp -@@ -12,6 +12,7 @@ +@@ -2,16 +2,22 @@ + + #include + ++#include + #include + #include ++#include + + #include + #include + #include ++#include + namespace Indium { class PrivateCommandQueue; class PrivateDevice; + class PrivateRenderCommandEncoder; ++ class PrivateBlitCommandEncoder; ++ class PrivateComputeCommandEncoder; class PrivateCommandBuffer: public CommandBuffer, public std::enable_shared_from_this { private: -@@ -25,7 +26,6 @@ namespace Indium { +@@ -25,7 +31,6 @@ namespace Indium { bool _committed = false; std::condition_variable _completedCondvar; bool _completed = false; @@ -163,21 +246,33 @@ index b7ffcce..4645377 100644 public: PrivateCommandBuffer(std::shared_ptr commandQueue); ~PrivateCommandBuffer(); -@@ -46,6 +46,13 @@ namespace Indium { +@@ -46,6 +51,25 @@ namespace Indium { virtual std::shared_ptr commandQueue() override; virtual std::shared_ptr device() override; + // creates the real render encoder immediately (what renderCommandEncoder() did before deferred encoding existed) + std::shared_ptr renderCommandEncoderNow(const RenderPassDescriptor& descriptor); ++ // same for blit/compute: the real encoder is created on the worker that replays the deferred op list ++ std::shared_ptr blitCommandEncoderNow(); ++ std::shared_ptr blitCommandEncoderNow(const BlitPassDescriptor& descriptor); ++ std::shared_ptr computeCommandEncoderNow(const ComputePassDescriptor& descriptor); ++ // leases the Vulkan command buffer (pool, allocation, begin) on the calling thread if that has not happened yet; ++ // only used with /tmp/metal-worker-lease (by default the constructor leases it, as in round 3) ++ void ensureRecordingStarted(); ++ void leaseRecording(); + void deferredBegin(); + void deferredEnd(); + // blocks until every deferred encoder of this command buffer has been replayed + void waitDeferred(); ++ // false while the commit has not started. A deferred replay may only be posted while the Vulkan command buffer ++ // is still open; a facade dropped after the commit (a Metal contract violation) then logs instead of recording ++ // into a closed command buffer. ++ bool replayAllowed() const { return !_committedAtomic.load(std::memory_order_acquire); } + void addScheduledHandlerLocked(std::function)> handler); void addCompletedHandlerLocked(std::function)> handler); -@@ -53,5 +60,13 @@ namespace Indium { +@@ -53,5 +77,15 @@ namespace Indium { INDIUM_PROPERTY_READONLY_OBJECT(PrivateDevice, p, P,rivateDevice); INDIUM_PROPERTY(VkCommandBuffer, c, C,ommandBuffer) = VK_NULL_HANDLE; @@ -189,6 +284,8 @@ index b7ffcce..4645377 100644 + unsigned _deferredPending = 0; + std::mutex _deferredMutex; + std::condition_variable _deferredCondvar; ++ // set at the start of the commit body (see replayAllowed()); appended last, like the members above ++ std::atomic _committedAtomic {false}; }; }; diff --git a/private-include/indium/command-encoder.private.hpp b/private-include/indium/command-encoder.private.hpp @@ -904,7 +1001,7 @@ index a43f532..6f2d7dd 100644 binding.stageFlags = functionTypeToVkShaderStageFlags(function->functionInfo().functionType); } diff --git a/private-include/indium/render-command-encoder.private.hpp b/private-include/indium/render-command-encoder.private.hpp -index a41b951..6cbea43 100644 +index a41b951..2f670e9 100644 --- a/private-include/indium/render-command-encoder.private.hpp +++ b/private-include/indium/render-command-encoder.private.hpp @@ -5,12 +5,15 @@ @@ -923,7 +1020,7 @@ index a41b951..6cbea43 100644 namespace Indium { class PrivateCommandBuffer; -@@ -29,18 +32,40 @@ namespace Indium { +@@ -29,19 +32,41 @@ namespace Indium { VkFramebuffer _framebuffer = VK_NULL_HANDLE; VkRenderPass _renderPass = VK_NULL_HANDLE; VkDescriptorPool _pool = VK_NULL_HANDLE; @@ -956,16 +1053,70 @@ index a41b951..6cbea43 100644 public: PrivateRenderCommandEncoder(std::shared_ptr commandBuffer, const RenderPassDescriptor& descriptor); ~PrivateRenderCommandEncoder(); +- +- virtual void setRenderPipelineState(std::shared_ptr renderPipelineState) override; + bool hasHeavyRetirement() const { + return _resolvedBufferOwners.size() + _resolvedTextureOwners.size() + + _pipelineOwners.size() + _retainedResourceOwners.size() >= 64 || + !_retiredPools.empty(); + } - ++ + virtual void memoryBarrier(size_t scope, RenderStages after, RenderStages before) override; - virtual void setRenderPipelineState(std::shared_ptr renderPipelineState) override; ++ virtual void setRenderPipelineState(const std::shared_ptr& renderPipelineState) override; virtual void setFrontFacingWinding(Winding frontFaceWinding) override; virtual void setCullMode(CullMode cullMode) override; + virtual void setDepthBias(float depthBias, float slopeScale, float clamp) override; +@@ -53,7 +78,7 @@ namespace Indium { + virtual void setScissorRects(const ScissorRect* scissorRects, size_t count) override; + virtual void setScissorRects(const std::vector& scissorRects) override; + virtual void setBlendColor(float red, float green, float blue, float alpha) override; +- virtual void setDepthStencilState(std::shared_ptr state) override; ++ virtual void setDepthStencilState(const std::shared_ptr& state) override; + virtual void setTriangleFillMode(TriangleFillMode triangleFillMode) override; + virtual void setStencilReferenceValue(uint32_t value) override; + virtual void setStencilReferenceValue(uint32_t front, uint32_t back) override; +@@ -63,30 +88,30 @@ namespace Indium { + virtual void drawPrimitives(PrimitiveType primitiveType, size_t vertexStart, size_t vertexCount, size_t instanceCount) override; + virtual void drawPrimitives(PrimitiveType primitiveType, size_t vertexStart, size_t vertexCount) override; + +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) override; +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount) override; +- virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset) override; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) override; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount) override; ++ virtual void drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset) override; + + virtual void setVertexBytes(const void* bytes, size_t length, size_t index) override; +- virtual void setVertexBuffer(std::shared_ptr buffer, size_t offset, size_t index) override; ++ virtual void setVertexBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) override; + virtual void setVertexBuffers(const std::vector>& buffers, const std::vector& offsets, Range range) override; + virtual void setVertexBufferOffset(size_t offset, size_t index) override; +- virtual void setVertexSamplerState(std::shared_ptr state, size_t index) override; +- virtual void setVertexSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) override; ++ virtual void setVertexSamplerState(const std::shared_ptr& state, size_t index) override; ++ virtual void setVertexSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) override; + virtual void setVertexSamplerStates(const std::vector>& states, Range range) override; + virtual void setVertexSamplerStates(const std::vector>& states, const std::vector& lodMinClamps, const std::vector& lodMaxClamps, Range range) override; +- virtual void setVertexTexture(std::shared_ptr texture, size_t index) override; ++ virtual void setVertexTexture(const std::shared_ptr& texture, size_t index) override; + virtual void setVertexTextures(const std::vector>& textures, Range range) override; + + virtual void setFragmentBytes(const void* bytes, size_t length, size_t index) override; +- virtual void setFragmentBuffer(std::shared_ptr buffer, size_t offset, size_t index) override; ++ virtual void setFragmentBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) override; + virtual void setFragmentBuffers(const std::vector>& buffers, const std::vector& offsets, Range range) override; + virtual void setFragmentBufferOffset(size_t offset, size_t index) override; +- virtual void setFragmentSamplerState(std::shared_ptr state, size_t index) override; +- virtual void setFragmentSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) override; ++ virtual void setFragmentSamplerState(const std::shared_ptr& state, size_t index) override; ++ virtual void setFragmentSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) override; + virtual void setFragmentSamplerStates(const std::vector>& states, Range range) override; + virtual void setFragmentSamplerStates(const std::vector>& states, const std::vector& lodMinClamps, const std::vector& lodMaxClamps, Range range) override; +- virtual void setFragmentTexture(std::shared_ptr texture, size_t index) override; ++ virtual void setFragmentTexture(const std::shared_ptr& texture, size_t index) override; + virtual void setFragmentTextures(std::vector>& textures, Range range) override; + + virtual void useResource(std::shared_ptr resource, ResourceUsage usage, RenderStages stages) override; @@ -95,8 +120,89 @@ namespace Indium { virtual void useResources(const std::vector>& resources, ResourceUsage usage) override; @@ -1749,18 +1900,19 @@ index 23fd2d3..e73df2a 100644 + return _gpuAddress; }; diff --git a/src/indium/command-buffer.cpp b/src/indium/command-buffer.cpp -index 43f4a84..0283316 100644 +index 43f4a84..46a4f72 100644 --- a/src/indium/command-buffer.cpp +++ b/src/indium/command-buffer.cpp -@@ -1,3 +1,7 @@ +@@ -1,3 +1,8 @@ +#include +#include +#include +#include ++#include #include #include #include -@@ -8,7 +12,116 @@ +@@ -8,7 +13,116 @@ #include #include @@ -1877,10 +2029,40 @@ index 43f4a84..0283316 100644 Indium::CommandBuffer::~CommandBuffer() {}; -@@ -23,14 +136,17 @@ Indium::PrivateCommandBuffer::PrivateCommandBuffer(std::shared_ptr Indium::PrivateCommandBuffer::device() { + return _privateDevice; + }; + ++namespace { ++// OFF by default (round-3 behaviour: the creating thread leases the Vulkan command buffer in the constructor). Guest ++// flag file /tmp/metal-worker-lease (checked once per process, so the constructor and every later recorder agree) ++// makes the lease lazy: the thread that first records leases it, which with deferred encoding is the submit worker. ++// Round 4 had this on unconditionally; that round regressed in matches, so it is an opt-in experiment now. ++bool workerLeaseEnabled() { ++ static const bool value = access("/tmp/metal-worker-lease", F_OK) == 0; ++ return value; ++} ++} ++ + Indium::PrivateCommandBuffer::PrivateCommandBuffer(std::shared_ptr commandQueue): _privateCommandQueue(commandQueue) { _privateDevice = _privateCommandQueue->privateDevice(); ++ // Default: lease now, on the creating thread. With /tmp/metal-worker-lease the Vulkan command buffer (pool reset, ++ // allocation, begin) is leased lazily by the thread that first records into it (see ensureRecordingStarted()). ++ if (!workerLeaseEnabled()) leaseRecording(); ++}; ++ ++// Only called with /tmp/metal-worker-lease (callers test workerLeaseEnabled() first; without the flag the ++// constructor has already leased). ++void Indium::PrivateCommandBuffer::ensureRecordingStarted() { ++ std::scoped_lock lock(_mutex); ++ if (_commandBuffer) return; ++ leaseRecording(); ++}; ++ ++// The lease itself; runs in the constructor or under _mutex in ensureRecordingStarted(). ++void Indium::PrivateCommandBuffer::leaseRecording() { + // A pool is exclusively leased through recording and GPU completion. + // Freeing another buffer from the same pool can otherwise race recording. + _recordingPool = _privateCommandQueue->acquireCommandPool(_commandBuffer); @@ -1897,7 +2079,7 @@ index 43f4a84..0283316 100644 // TODO: handle this in a more C++-friendly way abort(); } -@@ -41,13 +157,46 @@ Indium::PrivateCommandBuffer::PrivateCommandBuffer(std::shared_ptrreleaseCommandPool(pool, buffer); ++ // a command buffer that never recorded (created and dropped without an encoder) leased no pool ++ if (pool) queue->releaseCommandPool(pool, buffer); + queue.reset(); + } +}; @@ -1923,6 +2106,22 @@ index 43f4a84..0283316 100644 +} +namespace Indium { +void finishCommandRetirement() { encoderRetirementQueue().shutdown(); } ++} ++namespace { ++// Live experiment (guest flag, re-read every 256 command buffers like the async-encode flag): ++// /tmp/metal-retire-tiny makes the tiny buffers take the worker path too, i.e. the encoder ++// destructors and the command-pool release run on the retirement thread instead of the glyph ++// thread that released the MTLCommandBuffer. Only the *cost* is in question here - the handoff is ++// the same one heavy buffers already use, the queue's contract is completed GPU work (and an ++// unsubmitted buffer holds no GPU work), and a full queue still falls back to the inline release - ++// so the flag is the measurement, and the default stays the measured-cheaper inline release. ++bool retireTinyBuffers() { ++ static std::atomic calls { 0 }; ++ static std::atomic value { false }; ++ if ((calls.fetch_add(1, std::memory_order_relaxed) & 255) == 0) ++ value.store(access("/tmp/metal-retire-tiny", F_OK) == 0, std::memory_order_relaxed); ++ return value.load(std::memory_order_relaxed); ++} +} Indium::PrivateCommandBuffer::~PrivateCommandBuffer() { - DynamicVK::vkFreeCommandBuffers(_privateDevice->device(), _privateCommandQueue->commandPool(), 1, &_commandBuffer); @@ -1930,7 +2129,7 @@ index 43f4a84..0283316 100644 + // user completion handlers finish. Unsubmitted buffers have no GPU users. + // Tiny buffers cost less to release directly than to signal a Darwin + // condition variable and wake another worker through Darling's server. -+ bool heavy = _commandEncoders.size() >= 4; ++ bool heavy = _commandEncoders.size() >= 4 || retireTinyBuffers(); + if (!heavy) { + for (const auto& encoder : _commandEncoders) { + auto* render = dynamic_cast(encoder.get()); @@ -1943,13 +2142,39 @@ index 43f4a84..0283316 100644 -std::shared_ptr Indium::PrivateCommandBuffer::renderCommandEncoder(const RenderPassDescriptor& descriptor) { +std::shared_ptr Indium::PrivateCommandBuffer::renderCommandEncoderNow(const RenderPassDescriptor& descriptor) { ++ if (workerLeaseEnabled()) ensureRecordingStarted(); auto encoder = std::make_shared(shared_from_this(), descriptor); { std::scoped_lock lock(_mutex); -@@ -56,7 +205,37 @@ std::shared_ptr Indium::PrivateCommandBuffer::rend +@@ -56,7 +250,65 @@ std::shared_ptr Indium::PrivateCommandBuffer::rend return encoder; }; ++std::shared_ptr Indium::PrivateCommandBuffer::blitCommandEncoderNow() { ++ return blitCommandEncoderNow(BlitPassDescriptor {}); ++}; ++ ++// Creates the real blit encoder on whichever thread replays the deferred op list (see deferred-blit-encoder.private.hpp). ++std::shared_ptr Indium::PrivateCommandBuffer::blitCommandEncoderNow(const BlitPassDescriptor& descriptor) { ++ if (workerLeaseEnabled()) ensureRecordingStarted(); ++ auto encoder = std::make_shared(shared_from_this(), descriptor); ++ { ++ std::scoped_lock lock(_mutex); ++ _commandEncoders.push_back(encoder); ++ } ++ return encoder; ++}; ++ ++std::shared_ptr Indium::PrivateCommandBuffer::computeCommandEncoderNow(const ComputePassDescriptor& descriptor) { ++ if (workerLeaseEnabled()) ensureRecordingStarted(); ++ auto encoder = std::make_shared(shared_from_this(), descriptor); ++ { ++ std::scoped_lock lock(_mutex); ++ _commandEncoders.push_back(encoder); ++ } ++ return encoder; ++}; ++ +void Indium::PrivateCommandBuffer::deferredBegin() { + std::lock_guard lock(_deferredMutex); + ++_deferredPending; @@ -1965,7 +2190,7 @@ index 43f4a84..0283316 100644 + +void Indium::PrivateCommandBuffer::waitDeferred() { + std::unique_lock lock(_deferredMutex); -+ _deferredCondvar.wait(lock, [this] { return _deferredPending == 0; }); ++ Waits::blockOn("waitDeferred", _deferredCondvar, lock, [this] { return _deferredPending == 0; }); +} + +std::shared_ptr Indium::PrivateCommandBuffer::renderCommandEncoder(const RenderPassDescriptor& descriptor) { @@ -1980,19 +2205,31 @@ index 43f4a84..0283316 100644 +}; + std::shared_ptr Indium::PrivateCommandBuffer::blitCommandEncoder() { ++ // the opt-in flag is tested first: AsyncEncode::enabled() bumps a shared call counter (and re-reads its flag files ++ // every 256 calls), which the default path must not pay or perturb ++ if (AsyncEncode::deferBlitEnabled() && AsyncEncode::enabled()) return blitCommandEncoder(BlitPassDescriptor {}); + waitDeferred(); // never record into the Vulkan command buffer while deferred render encoders are being replayed auto encoder = std::make_shared(shared_from_this()); { std::scoped_lock lock(_mutex); -@@ -66,6 +245,7 @@ std::shared_ptr Indium::PrivateCommandBuffer::blitCo +@@ -66,6 +318,16 @@ std::shared_ptr Indium::PrivateCommandBuffer::blitCo }; std::shared_ptr Indium::PrivateCommandBuffer::blitCommandEncoder(const BlitPassDescriptor& descriptor) { ++ if (AsyncEncode::deferBlitEnabled() && AsyncEncode::enabled()) { ++ // records on the caller, replays on the ordered worker: no wait, no vkCmd* on the game's thread ++ auto encoder = std::make_shared(shared_from_this(), descriptor); ++ { ++ std::scoped_lock lock(_mutex); ++ _commandEncoders.push_back(encoder); ++ } ++ return encoder; ++ } + waitDeferred(); // never record into the Vulkan command buffer while deferred render encoders are being replayed auto encoder = std::make_shared(shared_from_this(), descriptor); { std::scoped_lock lock(_mutex); -@@ -79,6 +259,7 @@ std::shared_ptr Indium::PrivateCommandBuffer::com +@@ -79,6 +341,7 @@ std::shared_ptr Indium::PrivateCommandBuffer::com }; std::shared_ptr Indium::PrivateCommandBuffer::computeCommandEncoder(const ComputePassDescriptor& descriptor) { @@ -2000,7 +2237,7 @@ index 43f4a84..0283316 100644 auto encoder = std::make_shared(shared_from_this(), descriptor); { std::scoped_lock lock(_mutex); -@@ -91,7 +272,198 @@ std::shared_ptr Indium::PrivateCommandBuffer::com +@@ -91,10 +354,237 @@ std::shared_ptr Indium::PrivateCommandBuffer::com return computeCommandEncoder(ComputePassDescriptor { {}, dispatchType }); }; @@ -2157,6 +2394,13 @@ index 43f4a84..0283316 100644 + auto& state = submitBatchState(); + std::shared_ptr device = state.device; // set under the queue mutex by the first batched commit + if (!device) return; ++ // runs on the game thread (waitUntilCompleted) and on the upload worker; the counter attributes blocks to their caller. ++ if (Waits::enabled() && !SubmitThread::onWorker()) { ++ Waits::blockOnLock("queueMutex", device->graphicsQueueMutex()); ++ std::unique_lock queueLock(device->graphicsQueueMutex(), std::adopt_lock); ++ flushSubmitBatchLocked(); ++ return; ++ } + std::unique_lock queueLock(device->graphicsQueueMutex()); + flushSubmitBatchLocked(); +} @@ -2170,6 +2414,14 @@ index 43f4a84..0283316 100644 + if ((calls.fetch_add(1,std::memory_order_relaxed) & 63)==0) + bindingReuse.store(access("/tmp/metal-no-binding-reuse",F_OK)!=0,std::memory_order_relaxed); +} ++// OFF by default (round-3 behaviour: a presenting commit waits for the whole worker queue with drain()). Guest flag file ++// /tmp/metal-present-dispatch-wait (checked once per process) makes the frame-end sync narrower: the caller waits only ++// until this commit's own task has run (its present dispatch, and by FIFO order every replay queued before it), not for ++// tasks queued after it. Round 4 had this on by default; that round regressed in matches, so it is opt-in now. ++bool presentDispatchWaitEnabled() { ++ static const bool value = access("/tmp/metal-present-dispatch-wait", F_OK) == 0; ++ return value; ++} +} +bool Indium::bindingReuseEnabled() {return bindingReuse.load(std::memory_order_relaxed);} + @@ -2182,24 +2434,48 @@ index 43f4a84..0283316 100644 + auto self = shared_from_this(); + // read on the calling thread: the worker's commit consumes the list + const bool presents = !_drawablesToPresent.empty(); -+ SubmitThread::post([self]() { -+ AsyncEncode::PoolScope pool; -+ self->commit(); -+ }); + // Frame-end sync: the present is queued to the main queue by the worker's commit. The game pumps its run loop + // right after this call, so let the worker catch up first (it ran in parallel during the frame, the wait is its + // remaining lag); otherwise the present misses this pump and is shown one frame late, sometimes two per pump. + // Guest flag file /tmp/metal-async-no-frame-sync disables it for A/B runs. ++ if (presents && presentDispatchWaitEnabled() && access("/tmp/metal-async-no-frame-sync", F_OK) != 0) { ++ // Opt-in narrow wait (/tmp/metal-present-dispatch-wait): this commit's task signals the token when it is ++ // done, i.e. when the present dispatch (and, because the worker is FIFO, every replay queued before it) has ++ // happened. Waiting for the token instead of the queue's idle state does not block on later tasks. ++ SubmitThread::Token token; ++ SubmitThread::post([self, &token]() { ++ struct Signal { ++ SubmitThread::Token* token; ++ ~Signal() { token->signal(); } ++ } signal { &token }; ++ AsyncEncode::PoolScope pool; ++ self->commit(); ++ }); ++ token.wait("presentDispatchWait"); ++ return; ++ } ++ // Default (round 3): post first, then check the flag file and wait for the whole worker queue. ++ SubmitThread::post([self]() { ++ AsyncEncode::PoolScope pool; ++ self->commit(); ++ }); + if (presents && access("/tmp/metal-async-no-frame-sync", F_OK) != 0) { -+ SubmitThread::drain(); ++ SubmitThread::drain("drain@commit"); + } + return; + } + auto* frameSample = beginFrameIntervalSample(!_drawablesToPresent.empty()); ++ // with /tmp/metal-worker-lease the commit is the last recorder of this command buffer, so it leases it when nothing ++ // did before (empty buffer); by default the constructor has leased it ++ if (workerLeaseEnabled()) ensureRecordingStarted(); std::unique_lock lock(_mutex); _committed = true; -@@ -110,16 +482,28 @@ void Indium::PrivateCommandBuffer::commit() { ++ _committedAtomic.store(true, std::memory_order_release); + + for (const auto& encoder: _commandEncoders) { + if (auto renderEncoder = std::dynamic_pointer_cast(encoder)) { +@@ -110,16 +600,28 @@ void Indium::PrivateCommandBuffer::commit() { } } @@ -2229,7 +2505,7 @@ index 43f4a84..0283316 100644 // for the event loop auto timelineSemaphore = _privateDevice->getWrappedTimelineSemaphore(); -@@ -138,14 +522,49 @@ void Indium::PrivateCommandBuffer::commit() { +@@ -138,14 +640,49 @@ void Indium::PrivateCommandBuffer::commit() { readWriteTextures.insert(readWriteTextures.end(), readWrite.begin(), readWrite.end()); } @@ -2282,7 +2558,7 @@ index 43f4a84..0283316 100644 } std::vector signalInfos; -@@ -202,11 +621,13 @@ void Indium::PrivateCommandBuffer::commit() { +@@ -202,11 +739,13 @@ void Indium::PrivateCommandBuffer::commit() { auto privateTexture = std::dynamic_pointer_cast(texture); const auto& presentSema = presentationSemaphores[i]; @@ -2301,7 +2577,7 @@ index 43f4a84..0283316 100644 handleTextureSemaphores(privateTexture); } -@@ -215,7 +636,15 @@ void Indium::PrivateCommandBuffer::commit() { +@@ -215,7 +754,15 @@ void Indium::PrivateCommandBuffer::commit() { // but it also leaves the resources for this command buffer tied up, essentially becoming a memory leak. // i've observed this in the cube example, and it happens more than once (because the example display semaphore is exhausted and never signaled). // UPDATE: upon further testing, it seems that this only occurs when the view is off-screen/hidden. weird. @@ -2318,7 +2594,7 @@ index 43f4a84..0283316 100644 { std::unique_lock lock(self->_mutex); self->_completed = true; -@@ -234,8 +663,47 @@ void Indium::PrivateCommandBuffer::commit() { +@@ -234,8 +781,47 @@ void Indium::PrivateCommandBuffer::commit() { // the handlers may have captured a reference to us (from their surrounding scope), so clear out handlers so those references go away self->_scheduledHandlers.clear(); self->_completedHandlers.clear(); @@ -2366,7 +2642,7 @@ index 43f4a84..0283316 100644 commandBufferInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO; commandBufferInfo.commandBuffer = _commandBuffer; -@@ -252,23 +720,41 @@ void Indium::PrivateCommandBuffer::commit() { +@@ -252,23 +838,41 @@ void Indium::PrivateCommandBuffer::commit() { // the Device constructor tries to choose command queues that support as many operations as possible, but it's possible // that a particular device only supports certain operations on certain queues (e.g. maybe it only supports transfer operations // on an exclusive queue that doesn't support graphics or compute). @@ -2413,19 +2689,20 @@ index 43f4a84..0283316 100644 }; void Indium::PrivateCommandBuffer::presentDrawable(std::shared_ptr drawable) { -@@ -307,9 +793,14 @@ void Indium::PrivateCommandBuffer::addCompletedHandler(std::function(std::chrono::steady_clock::now() - waitStart).count()); }; diff --git a/src/indium/command-queue.cpp b/src/indium/command-queue.cpp @@ -2525,7 +2802,7 @@ index 5f62ad0..5957bec 100644 DynamicVK::vkCmdBindDescriptorSets(buf->commandBuffer(), VK_PIPELINE_BIND_POINT_COMPUTE, _pso->layout(), 0, descriptorSets.size(), descriptorSets.data(), 0, nullptr); }; diff --git a/src/indium/device.cpp b/src/indium/device.cpp -index 322e186..5bc9aad 100644 +index 322e186..70c4a15 100644 --- a/src/indium/device.cpp +++ b/src/indium/device.cpp @@ -1,4 +1,6 @@ @@ -2535,7 +2812,56 @@ index 322e186..5bc9aad 100644 #include #include #include -@@ -83,7 +85,8 @@ Indium::Device::~Device() {}; +@@ -12,10 +14,12 @@ + + #include + ++#include + #include + #include + #include + #include ++#include + + #include + #include +@@ -39,6 +43,15 @@ void Indium::initGlobalDeviceList() { + return; + } + ++ // A VkDevice is created (and its driver initialized) for every supported physical device below, and ++ // createSystemDefaultDevice() takes the first entry. Software devices (lavapipe) can be skipped -- a VkDevice on ++ // one of them starts its rasterizer pool (nproc llvmpipe threads, idle for a whole match) and the loader may order ++ // it first. Only skip them while some other supported device exists. OFF by default (round-3 behaviour: a device is ++ // created for every supported device, in the loader's order); guest flag file /tmp/metal-skip-cpu-devices turns the ++ // skip on. It is read here, at load time, because this list is built once. ++ const bool skipCPUDevices = access("/tmp/metal-skip-cpu-devices", F_OK) == 0; ++ std::vector> candidates; ++ + for (auto&& device: physicalDevices) { + VkPhysicalDeviceProperties props; + DynamicVK::vkGetPhysicalDeviceProperties(device, &props); +@@ -71,7 +84,18 @@ void Indium::initGlobalDeviceList() { + continue; + } + +- globalDeviceList.push_back(std::make_shared(std::move(device))); ++ candidates.emplace_back(device, props.deviceType); ++ } ++ ++ const bool haveHardwareDevice = skipCPUDevices && std::any_of(candidates.begin(), candidates.end(), [](const auto& candidate) { ++ return candidate.second != VK_PHYSICAL_DEVICE_TYPE_CPU; ++ }); ++ ++ for (auto&& candidate: candidates) { ++ if (haveHardwareDevice && candidate.second == VK_PHYSICAL_DEVICE_TYPE_CPU) { ++ continue; ++ } ++ globalDeviceList.push_back(std::make_shared(std::move(candidate.first))); + } + }; + +@@ -83,7 +107,8 @@ Indium::Device::~Device() {}; Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): _physicalDevice(physicalDevice) @@ -2545,7 +2871,7 @@ index 322e186..5bc9aad 100644 DynamicVK::vkGetPhysicalDeviceProperties(_physicalDevice, &_properties); std::vector queueFamilies; -@@ -233,6 +236,36 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): +@@ -233,6 +258,36 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): } } @@ -2582,7 +2908,7 @@ index 322e186..5bc9aad 100644 VkDeviceCreateInfo deviceCreateInfo {}; deviceCreateInfo.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO; deviceCreateInfo.pQueueCreateInfos = queueCreateInfos.data(); -@@ -246,6 +279,7 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): +@@ -246,6 +301,7 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): } _features = indiumFeatures; @@ -2590,7 +2916,7 @@ index 322e186..5bc9aad 100644 for (const auto& index: queueFamilyIndices) { VkQueue queue; -@@ -297,6 +331,9 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): +@@ -297,6 +353,9 @@ Indium::PrivateDevice::PrivateDevice(VkPhysicalDevice physicalDevice): }; Indium::PrivateDevice::~PrivateDevice() { @@ -2600,7 +2926,7 @@ index 322e186..5bc9aad 100644 if (_oneshotCommandPool) { DynamicVK::vkDestroyCommandPool(_device, _oneshotCommandPool, nullptr); } -@@ -308,6 +345,23 @@ std::string Indium::PrivateDevice::name() const { +@@ -308,6 +367,23 @@ std::string Indium::PrivateDevice::name() const { return _properties.deviceName; }; @@ -2624,7 +2950,7 @@ index 322e186..5bc9aad 100644 std::shared_ptr Indium::PrivateDevice::newCommandQueue() { return std::make_shared(shared_from_this()); }; -@@ -317,9 +371,9 @@ std::shared_ptr Indium::PrivateDevice::newRenderPip +@@ -317,9 +393,9 @@ std::shared_ptr Indium::PrivateDevice::newRenderPip }; std::shared_ptr Indium::PrivateDevice::newComputePipelineState(const ComputePipelineDescriptor& descriptor, PipelineOption options, std::shared_ptr reflection) { @@ -2637,7 +2963,7 @@ index 322e186..5bc9aad 100644 return std::make_shared(shared_from_this(), descriptor); }; -@@ -357,6 +411,7 @@ std::shared_ptr Indium::PrivateDevice::newLibrary(const void* d +@@ -357,6 +433,7 @@ std::shared_ptr Indium::PrivateDevice::newLibrary(const void* d break; } @@ -2645,7 +2971,7 @@ index 322e186..5bc9aad 100644 funcInfo.bindings.insert(funcInfo.bindings.end(), info.bindings.begin(), info.bindings.end()); funcInfo.embeddedSamplers.insert(funcInfo.embeddedSamplers.end(), info.embeddedSamplers.begin(), info.embeddedSamplers.end()); } -@@ -480,9 +535,18 @@ void Indium::PrivateDevice::waitForSemaphore(VkSemaphore semaphore, uint64_t tar +@@ -480,9 +557,18 @@ void Indium::PrivateDevice::waitForSemaphore(VkSemaphore semaphore, uint64_t tar wakeupEventLoop(); }; @@ -2665,7 +2991,7 @@ index 322e186..5bc9aad 100644 VkSemaphoreTypeCreateInfo typeInfo {}; typeInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO; typeInfo.semaphoreType = VK_SEMAPHORE_TYPE_TIMELINE; -@@ -505,7 +569,14 @@ Indium::TimelineSemaphore Indium::PrivateDevice::getTimelineSemaphore() { +@@ -505,7 +591,14 @@ Indium::TimelineSemaphore Indium::PrivateDevice::getTimelineSemaphore() { }; void Indium::PrivateDevice::putTimelineSemaphore(const TimelineSemaphore& semaphore) { @@ -2681,7 +3007,7 @@ index 322e186..5bc9aad 100644 }; std::shared_ptr Indium::PrivateDevice::getWrappedTimelineSemaphore() { -@@ -551,3 +622,91 @@ std::shared_ptr Indium::PrivateDevice::getWrappedBinary +@@ -551,3 +644,91 @@ std::shared_ptr Indium::PrivateDevice::getWrappedBinary delete ptr; }); }; @@ -2943,7 +3269,7 @@ index d56a213..66a6019 100644 + +const std::vector& Indium::PrivateFunction::argumentInfo() const { return _functionInfo.arguments; } diff --git a/src/indium/render-command-encoder.cpp b/src/indium/render-command-encoder.cpp -index b27ffe5..105dbc8 100644 +index b27ffe5..dd7434d 100644 --- a/src/indium/render-command-encoder.cpp +++ b/src/indium/render-command-encoder.cpp @@ -1,3 +1,8 @@ @@ -3280,10 +3606,11 @@ index b27ffe5..105dbc8 100644 + for (auto pool : _retiredPools) _privateDevice->releaseDescriptorPool(pool); }; - void Indium::PrivateRenderCommandEncoder::setRenderPipelineState(std::shared_ptr renderPipelineState) { +-void Indium::PrivateRenderCommandEncoder::setRenderPipelineState(std::shared_ptr renderPipelineState) { - auto buf = _privateCommandBuffer.lock(); - _privatePSO = std::dynamic_pointer_cast(renderPipelineState); - _privatePSO->recreatePipeline(_renderPass, false); ++void Indium::PrivateRenderCommandEncoder::setRenderPipelineState(const std::shared_ptr& renderPipelineState) { + if (_privatePSO == renderPipelineState) return; + // Keep each pipeline once, including all layouts and immutable samplers + // referenced by earlier draws, until this encoder retires. @@ -3628,7 +3955,8 @@ index b27ffe5..105dbc8 100644 + if (reuses) Indium::count("descriptorStageReuse", reuses); }; - void Indium::PrivateRenderCommandEncoder::setVertexBuffer(std::shared_ptr buffer, size_t offset, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setVertexBuffer(std::shared_ptr buffer, size_t offset, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setVertexBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) { _functionResources[0].setBuffer(buffer, offset, index); + resolveBuffer(0, index); }; @@ -3641,22 +3969,29 @@ index b27ffe5..105dbc8 100644 } }; -@@ -393,10 +760,12 @@ void Indium::PrivateRenderCommandEncoder::setVertexBufferOffset(size_t offset, s +@@ -391,12 +758,14 @@ void Indium::PrivateRenderCommandEncoder::setVertexBufferOffset(size_t offset, s + _functionResources[0].setBufferOffset(offset, index); + }; - void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(std::shared_ptr state, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(std::shared_ptr state, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(const std::shared_ptr& state, size_t index) { _functionResources[0].setSamplerState(state, std::nullopt, index); + resolveSampler(0, index); }; - void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setVertexSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) { _functionResources[0].setSamplerState(state, std::make_pair(lodMinClamp, lodMaxClamp), index); + resolveSampler(0, index); }; void Indium::PrivateRenderCommandEncoder::setVertexSamplerStates(const std::vector>& states, Range range) { -@@ -413,26 +782,32 @@ void Indium::PrivateRenderCommandEncoder::setVertexSamplerStates(const std::vect +@@ -411,28 +780,34 @@ void Indium::PrivateRenderCommandEncoder::setVertexSamplerStates(const std::vect + } + }; - void Indium::PrivateRenderCommandEncoder::setVertexTexture(std::shared_ptr texture, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setVertexTexture(std::shared_ptr texture, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setVertexTexture(const std::shared_ptr& texture, size_t index) { _functionResources[0].setTexture(texture, index); + resolveTexture(0, index); }; @@ -3676,7 +4011,8 @@ index b27ffe5..105dbc8 100644 + resolveBuffer(1, index); }; - void Indium::PrivateRenderCommandEncoder::setFragmentBuffer(std::shared_ptr buffer, size_t offset, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setFragmentBuffer(std::shared_ptr buffer, size_t offset, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setFragmentBuffer(const std::shared_ptr& buffer, size_t offset, size_t index) { _functionResources[1].setBuffer(buffer, offset, index); + resolveBuffer(1, index); }; @@ -3689,22 +4025,29 @@ index b27ffe5..105dbc8 100644 } }; -@@ -442,10 +817,12 @@ void Indium::PrivateRenderCommandEncoder::setFragmentBufferOffset(size_t offset, +@@ -440,12 +815,14 @@ void Indium::PrivateRenderCommandEncoder::setFragmentBufferOffset(size_t offset, + _functionResources[1].setBufferOffset(offset, index); + }; - void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(std::shared_ptr state, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(std::shared_ptr state, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(const std::shared_ptr& state, size_t index) { _functionResources[1].setSamplerState(state, std::nullopt, index); + resolveSampler(1, index); }; - void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(std::shared_ptr state, float lodMinClamp, float lodMaxClamp, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setFragmentSamplerState(const std::shared_ptr& state, float lodMinClamp, float lodMaxClamp, size_t index) { _functionResources[1].setSamplerState(state, std::make_pair(lodMinClamp, lodMaxClamp), index); + resolveSampler(1, index); }; void Indium::PrivateRenderCommandEncoder::setFragmentSamplerStates(const std::vector>& states, Range range) { -@@ -462,15 +839,20 @@ void Indium::PrivateRenderCommandEncoder::setFragmentSamplerStates(const std::ve +@@ -460,17 +837,22 @@ void Indium::PrivateRenderCommandEncoder::setFragmentSamplerStates(const std::ve + } + }; - void Indium::PrivateRenderCommandEncoder::setFragmentTexture(std::shared_ptr texture, size_t index) { +-void Indium::PrivateRenderCommandEncoder::setFragmentTexture(std::shared_ptr texture, size_t index) { ++void Indium::PrivateRenderCommandEncoder::setFragmentTexture(const std::shared_ptr& texture, size_t index) { _functionResources[1].setTexture(texture, index); + resolveTexture(1, index); }; @@ -3717,14 +4060,15 @@ index b27ffe5..105dbc8 100644 } }; - void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) { +-void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) { ++void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount, int64_t baseVertex, size_t baseInstance) { + if (drawSkip.encoder == this && drawSkip.skip) return; + beginPassIfNeeded(); + GpuTiming::countDraw(); auto buf = _privateCommandBuffer.lock(); // bind the pipeline with the right topology class for this primitive -@@ -490,24 +872,39 @@ void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType pr +@@ -490,39 +872,62 @@ void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType pr default: throw BadEnumValue(); } @@ -3776,9 +4120,18 @@ index b27ffe5..105dbc8 100644 - _savedFunctionResources.push_back(_functionResources[1]); }; - void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount) { -@@ -521,8 +918,16 @@ void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType pr - void Indium::PrivateRenderCommandEncoder::setDepthStencilState(std::shared_ptr state) { +-void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset, size_t instanceCount) { ++void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, size_t instanceCount) { + drawIndexedPrimitives(primitiveType, indexCount, indexType, indexBuffer, indexBufferOffset, instanceCount, 0, 0); + }; + +-void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, std::shared_ptr indexBuffer, size_t indexBufferOffset) { ++void Indium::PrivateRenderCommandEncoder::drawIndexedPrimitives(PrimitiveType primitiveType, size_t indexCount, IndexType indexType, const std::shared_ptr& indexBuffer, size_t indexBufferOffset) { + drawIndexedPrimitives(primitiveType, indexCount, indexType, indexBuffer, indexBufferOffset, 1); + }; + +-void Indium::PrivateRenderCommandEncoder::setDepthStencilState(std::shared_ptr state) { ++void Indium::PrivateRenderCommandEncoder::setDepthStencilState(const std::shared_ptr& state) { auto buf = _privateCommandBuffer.lock(); auto privateState = std::dynamic_pointer_cast(state); - auto& desc = privateState->descriptor(); @@ -3916,10 +4269,10 @@ index 71392df..d82e5f0 100644 info.unnormalizedCoordinates = descriptor.normalizedCoordinates ? VK_FALSE : VK_TRUE; diff --git a/src/indium/texture.cpp b/src/indium/texture.cpp -index 1e3e971..0870d92 100644 +index 1e3e971..4db87d4 100644 --- a/src/indium/texture.cpp +++ b/src/indium/texture.cpp -@@ -1,8 +1,87 @@ +@@ -1,8 +1,90 @@ +#include +#include +#include @@ -3983,6 +4336,7 @@ index 1e3e971..0870d92 100644 + while (!list.empty()) { + auto& front = list.front(); + if (mustWait > 0) { ++ Indium::Waits::Block block("oneshotFence"); + if (Indium::DynamicVK::vkWaitForFences(device->device(), 1, &front.fence, VK_TRUE, 1ull * 1000 * 1000 * 1000) != VK_SUCCESS) abort(); + --mustWait; + } else if (Indium::DynamicVK::vkWaitForFences(device->device(), 1, &front.fence, VK_TRUE, 0) != VK_SUCCESS) { // zero timeout: a status poll @@ -3997,6 +4351,8 @@ index 1e3e971..0870d92 100644 + void finishOneshot(const std::shared_ptr& device, VkFence fence, VkCommandBuffer commandBuffer, VkImage image, std::shared_ptr keepAlive) { + PendingOneshot entry {fence, commandBuffer, image, std::move(keepAlive)}; + if (syncUploads()) { ++ // this is a GPU wait on the uploading thread; counted under /tmp/metal-debug-waits like the hand-off waits ++ Indium::Waits::Block block("oneshotFence"); + if (Indium::DynamicVK::vkWaitForFences(device->device(), 1, &fence, VK_TRUE, 1ull * 1000 * 1000 * 1000) != VK_SUCCESS) abort(); + releaseOneshot(device, entry); + return; @@ -4007,7 +4363,7 @@ index 1e3e971..0870d92 100644 #include -@@ -37,8 +116,8 @@ std::shared_ptr Indium::PrivateTexture::newTextureView(PixelFor +@@ -37,8 +119,8 @@ std::shared_ptr Indium::PrivateTexture::newTextureView(PixelFor return newTextureView( pixelFormat, textureType(), @@ -4018,7 +4374,7 @@ index 1e3e971..0870d92 100644 ); }; -@@ -78,12 +157,16 @@ Indium::TextureView::TextureView(std::shared_ptr original, Pixel +@@ -78,12 +160,16 @@ Indium::TextureView::TextureView(std::shared_ptr original, Pixel _layers(layers), _swizzle(swizzle) { @@ -4037,7 +4393,7 @@ index 1e3e971..0870d92 100644 auto parentLayerEnd = parentLayerStart + _original->arrayLength() - 1; auto layerStart = std::min(parentLayerStart + _layers.start, parentLayerEnd); auto layerEnd = std::min(layerStart + _layers.length - 1, parentLayerEnd); -@@ -109,9 +192,16 @@ Indium::TextureView::TextureView(std::shared_ptr original, Pixel +@@ -109,9 +195,16 @@ Indium::TextureView::TextureView(std::shared_ptr original, Pixel // TODO abort(); } @@ -4054,7 +4410,7 @@ index 1e3e971..0870d92 100644 DynamicVK::vkDestroyImageView(std::dynamic_pointer_cast(_original->device())->device(), _imageView, nullptr); }; -@@ -124,15 +214,15 @@ Indium::TextureType Indium::TextureView::textureType() const { +@@ -124,15 +217,15 @@ Indium::TextureType Indium::TextureView::textureType() const { }; size_t Indium::TextureView::width() const { @@ -4073,7 +4429,7 @@ index 1e3e971..0870d92 100644 }; size_t Indium::TextureView::mipmapLevelCount() const { -@@ -326,13 +416,42 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, +@@ -326,13 +419,42 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, info.tiling = (descriptor.allowGPUOptimizedContents || !canBeLinear) ? VK_IMAGE_TILING_OPTIMAL : VK_IMAGE_TILING_LINEAR; @@ -4120,7 +4476,7 @@ index 1e3e971..0870d92 100644 info.sharingMode = VK_SHARING_MODE_EXCLUSIVE; // TODO: check if this should be "concurrent" instead. i think we're good, though. info.queueFamilyIndexCount = 0; info.pQueueFamilyIndices = nullptr; -@@ -373,19 +492,18 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, +@@ -373,19 +495,18 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, throw std::runtime_error("No suitable memory region found for image with requested storage mode"); } @@ -4149,7 +4505,7 @@ index 1e3e971..0870d92 100644 VkCommandBufferAllocateInfo cmdBufAllocInfo {}; cmdBufAllocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; cmdBufAllocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; -@@ -409,7 +527,7 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, +@@ -409,7 +530,7 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, barrier.srcAccessMask = VK_ACCESS_NONE; barrier.dstAccessMask = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT; barrier.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; @@ -4158,7 +4514,7 @@ index 1e3e971..0870d92 100644 barrier.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; barrier.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; barrier.image = _image; -@@ -438,15 +556,13 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, +@@ -438,15 +559,13 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, submitInfo.commandBufferCount = 1; submitInfo.pCommandBuffers = &cmdBuf; @@ -4180,7 +4536,7 @@ index 1e3e971..0870d92 100644 VkImageViewCreateInfo info2 {}; info2.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; -@@ -463,12 +579,24 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, +@@ -463,12 +582,24 @@ Indium::ConcreteTexture::ConcreteTexture(std::shared_ptr device, // TODO abort(); } @@ -4206,7 +4562,7 @@ index 1e3e971..0870d92 100644 }; Indium::TextureType Indium::ConcreteTexture::textureType() const { -@@ -537,7 +665,20 @@ VkImage Indium::ConcreteTexture::image() { +@@ -537,7 +668,20 @@ VkImage Indium::ConcreteTexture::image() { return _image; }; @@ -4227,7 +4583,7 @@ index 1e3e971..0870d92 100644 return VK_IMAGE_LAYOUT_GENERAL; }; -@@ -546,6 +687,16 @@ void Indium::ConcreteTexture::replaceRegion(Indium::Region region, size_t mipmap +@@ -546,6 +690,16 @@ void Indium::ConcreteTexture::replaceRegion(Indium::Region region, size_t mipmap }; void Indium::ConcreteTexture::replaceRegion(Indium::Region region, size_t mipmapLevel, size_t slice, const void* bytes, size_t bytesPerRow, size_t bytesPerImage) { @@ -4244,7 +4600,7 @@ index 1e3e971..0870d92 100644 if (_storageMode != StorageMode::Managed && _storageMode != StorageMode::Shared) { throw std::runtime_error("Invalid usage of replaceRegion on non-managed and non-shared texture"); } -@@ -556,99 +707,109 @@ void Indium::ConcreteTexture::replaceRegion(Indium::Region region, size_t mipmap +@@ -556,99 +710,109 @@ void Indium::ConcreteTexture::replaceRegion(Indium::Region region, size_t mipmap auto aspect = pixelFormatToVkImageAspectFlags(_descriptor.pixelFormat); @@ -5984,7 +6340,7 @@ diff --git a/private-include/indium/deferred-render-encoder.private.hpp b/privat new file mode 100644 --- /dev/null +++ b/private-include/indium/deferred-render-encoder.private.hpp -@@ -0,0 +1,355 @@ +@@ -0,0 +1,372 @@ +#pragma once + +// Deferred render-command encoding. @@ -6194,6 +6550,18 @@ new file mode 100644 + auto cb = st->commandBuffer; + if (!_begun) { _begun = true; cb->deferredBegin(); } + Buffers* chunk = _rec.release(); ++ if (!cb->replayAllowed()) { ++ // an encoder dropped after the commit (a Metal contract violation): the Vulkan command buffer is ++ // closed, so drop the chunk instead of recording into it, and balance the deferred count ++ releaseBuffers(std::unique_ptr(chunk)); ++ if (last) { ++ cb->deferredEnd(); ++ st.reset(); ++ } ++ if (!last) { _rec = acquireBuffers(); _r = _rec.get(); } ++ else _r = nullptr; ++ return; ++ } + SubmitThread::post([st, cb, chunk, last]() mutable { + std::unique_ptr owned(chunk); + replay(*st, *owned, last); @@ -6213,7 +6581,12 @@ new file mode 100644 + _r->ops.emplace_back(k); + return _r->ops.back(); + } -+ void addRef(Kind k, std::shared_ptr ref, size_t a, size_t b) { Op& o = add(k); o.p = std::move(ref); o.n[0] = a; o.n[1] = b; } ++ // The op keeps the resource alive until the worker replays it; taking the reference ++ // straight from the caller's shared_ptr (the setters now take it by const reference) ++ // costs exactly one increment and no temporary, where the by-value form left an ++ // increment/decrement pair plus a store-to-load chain per forwarded bind. ++ template ++ void addRef(Kind k, const std::shared_ptr& ref, size_t a, size_t b) { Op& o = add(k); o.p = ref; o.n[0] = a; o.n[1] = b; } + + public: + DeferredRenderCommandEncoder(std::shared_ptr cb, const RenderPassDescriptor& d): @@ -6228,7 +6601,7 @@ new file mode 100644 + _state.reset(); + } + -+ void setRenderPipelineState(std::shared_ptr s) override { addRef(Kind::SetPipeline, std::move(s), 0, 0); } ++ void setRenderPipelineState(const std::shared_ptr& s) override { addRef(Kind::SetPipeline, s, 0, 0); } + void setFrontFacingWinding(Winding w) override { add(Kind::FrontFacing).e1 = static_cast(w); } + void setCullMode(CullMode m) override { add(Kind::CullMode).e1 = static_cast(m); } + void setDepthBias(float bias, float slope, float clamp) override { Op& o = add(Kind::DepthBias); o.f[0] = bias; o.f[1] = slope; o.f[2] = clamp; } @@ -6246,7 +6619,7 @@ new file mode 100644 + } + void setScissorRects(const std::vector& s) override { setScissorRects(s.data(), s.size()); } + void setBlendColor(float r, float g, float b, float a) override { Op& o = add(Kind::BlendColor); o.f[0] = r; o.f[1] = g; o.f[2] = b; o.f[3] = a; } -+ void setDepthStencilState(std::shared_ptr s) override { addRef(Kind::DepthStencil, std::move(s), 0, 0); } ++ void setDepthStencilState(const std::shared_ptr& s) override { addRef(Kind::DepthStencil, s, 0, 0); } + void setTriangleFillMode(TriangleFillMode m) override { add(Kind::FillMode).e1 = static_cast(m); } + void setStencilReferenceValue(uint32_t v) override { add(Kind::StencilRef).e1 = v; } + void setStencilReferenceValue(uint32_t front, uint32_t back) override { Op& o = add(Kind::StencilRefFB); o.e1 = front; o.e2 = back; } @@ -6258,17 +6631,17 @@ new file mode 100644 + void drawPrimitives(PrimitiveType t, size_t start, size_t count, size_t instances) override { drawPrimitives(t, start, count, instances, 0); } + void drawPrimitives(PrimitiveType t, size_t start, size_t count) override { drawPrimitives(t, start, count, 1, 0); } + -+ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, std::shared_ptr indexBuffer, size_t indexBufferOffset, ++ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, const std::shared_ptr& indexBuffer, size_t indexBufferOffset, + size_t instanceCount, int64_t baseVertex, size_t baseInstance) override { + Op& o = add(Kind::DrawIdx); -+ o.e1 = static_cast(t); o.e2 = static_cast(it); o.p = std::move(indexBuffer); ++ o.e1 = static_cast(t); o.e2 = static_cast(it); o.p = indexBuffer; + o.n[0] = indexCount; o.n[1] = indexBufferOffset; o.n[2] = instanceCount; o.n[3] = static_cast(baseVertex); o.n[4] = baseInstance; + } -+ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, std::shared_ptr b, size_t off, size_t instances) override { -+ drawIndexedPrimitives(t, indexCount, it, std::move(b), off, instances, 0, 0); ++ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, const std::shared_ptr& b, size_t off, size_t instances) override { ++ drawIndexedPrimitives(t, indexCount, it, b, off, instances, 0, 0); + } -+ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, std::shared_ptr b, size_t off) override { -+ drawIndexedPrimitives(t, indexCount, it, std::move(b), off, 1, 0, 0); ++ void drawIndexedPrimitives(PrimitiveType t, size_t indexCount, IndexType it, const std::shared_ptr& b, size_t off) override { ++ drawIndexedPrimitives(t, indexCount, it, b, off, 1, 0, 0); + } + + void setVertexBytes(const void* bytes, size_t length, size_t index) override { @@ -6276,14 +6649,14 @@ new file mode 100644 + const auto* p = static_cast(bytes); + _r->bytes.insert(_r->bytes.end(), p, p + length); + } -+ void setVertexBuffer(std::shared_ptr b, size_t offset, size_t index) override { addRef(Kind::VBuffer, std::move(b), offset, index); } ++ void setVertexBuffer(const std::shared_ptr& b, size_t offset, size_t index) override { addRef(Kind::VBuffer, b, offset, index); } + void setVertexBuffers(const std::vector>& b, const std::vector& o, Range r) override { + for (size_t i = 0; i < r.length; ++i) addRef(Kind::VBuffer, b[i], o[i], r.start + i); + } + void setVertexBufferOffset(size_t offset, size_t index) override { Op& o = add(Kind::VBufferOffset); o.n[0] = offset; o.n[1] = index; } -+ void setVertexSamplerState(std::shared_ptr s, size_t index) override { addRef(Kind::VSampler, std::move(s), index, 0); } -+ void setVertexSamplerState(std::shared_ptr s, float lo, float hi, size_t index) override { -+ Op& o = add(Kind::VSamplerClamp); o.p = std::move(s); o.f[0] = lo; o.f[1] = hi; o.n[0] = index; ++ void setVertexSamplerState(const std::shared_ptr& s, size_t index) override { addRef(Kind::VSampler, s, index, 0); } ++ void setVertexSamplerState(const std::shared_ptr& s, float lo, float hi, size_t index) override { ++ Op& o = add(Kind::VSamplerClamp); o.p = s; o.f[0] = lo; o.f[1] = hi; o.n[0] = index; + } + void setVertexSamplerStates(const std::vector>& s, Range r) override { + for (size_t i = 0; i < r.length; ++i) setVertexSamplerState(s[i], r.start + i); @@ -6291,7 +6664,7 @@ new file mode 100644 + void setVertexSamplerStates(const std::vector>& s, const std::vector& lo, const std::vector& hi, Range r) override { + for (size_t i = 0; i < r.length; ++i) setVertexSamplerState(s[i], lo[i], hi[i], r.start + i); + } -+ void setVertexTexture(std::shared_ptr t, size_t index) override { addRef(Kind::VTexture, std::move(t), index, 0); } ++ void setVertexTexture(const std::shared_ptr& t, size_t index) override { addRef(Kind::VTexture, t, index, 0); } + void setVertexTextures(const std::vector>& t, Range r) override { + for (size_t i = 0; i < r.length; ++i) addRef(Kind::VTexture, t[i], r.start + i, 0); + } @@ -6301,14 +6674,14 @@ new file mode 100644 + const auto* p = static_cast(bytes); + _r->bytes.insert(_r->bytes.end(), p, p + length); + } -+ void setFragmentBuffer(std::shared_ptr b, size_t offset, size_t index) override { addRef(Kind::FBuffer, std::move(b), offset, index); } ++ void setFragmentBuffer(const std::shared_ptr& b, size_t offset, size_t index) override { addRef(Kind::FBuffer, b, offset, index); } + void setFragmentBuffers(const std::vector>& b, const std::vector& o, Range r) override { + for (size_t i = 0; i < r.length; ++i) addRef(Kind::FBuffer, b[i], o[i], r.start + i); + } + void setFragmentBufferOffset(size_t offset, size_t index) override { Op& o = add(Kind::FBufferOffset); o.n[0] = offset; o.n[1] = index; } -+ void setFragmentSamplerState(std::shared_ptr s, size_t index) override { addRef(Kind::FSampler, std::move(s), index, 0); } -+ void setFragmentSamplerState(std::shared_ptr s, float lo, float hi, size_t index) override { -+ Op& o = add(Kind::FSamplerClamp); o.p = std::move(s); o.f[0] = lo; o.f[1] = hi; o.n[0] = index; ++ void setFragmentSamplerState(const std::shared_ptr& s, size_t index) override { addRef(Kind::FSampler, s, index, 0); } ++ void setFragmentSamplerState(const std::shared_ptr& s, float lo, float hi, size_t index) override { ++ Op& o = add(Kind::FSamplerClamp); o.p = s; o.f[0] = lo; o.f[1] = hi; o.n[0] = index; + } + void setFragmentSamplerStates(const std::vector>& s, Range r) override { + for (size_t i = 0; i < r.length; ++i) setFragmentSamplerState(s[i], r.start + i); @@ -6316,7 +6689,7 @@ new file mode 100644 + void setFragmentSamplerStates(const std::vector>& s, const std::vector& lo, const std::vector& hi, Range r) override { + for (size_t i = 0; i < r.length; ++i) setFragmentSamplerState(s[i], lo[i], hi[i], r.start + i); + } -+ void setFragmentTexture(std::shared_ptr t, size_t index) override { addRef(Kind::FTexture, std::move(t), index, 0); } ++ void setFragmentTexture(const std::shared_ptr& t, size_t index) override { addRef(Kind::FTexture, t, index, 0); } + void setFragmentTextures(std::vector>& t, Range r) override { + for (size_t i = 0; i < r.length; ++i) addRef(Kind::FTexture, t[i], r.start + i, 0); + } @@ -7247,7 +7620,7 @@ diff --git a/private-include/indium/submit-thread.private.hpp b/private-include/ new file mode 100644 --- /dev/null +++ b/private-include/indium/submit-thread.private.hpp -@@ -0,0 +1,136 @@ +@@ -0,0 +1,159 @@ +#pragma once + +// Ordered queue-submission worker. @@ -7260,6 +7633,8 @@ new file mode 100644 +// pending commit. Opt-in: guest flag file /tmp/metal-submit-thread (checked once) enables it; the A/B showed no gain, so submission stays synchronous. +// Header-only with function-local statics: no class gains a member. + ++#include ++ +#include +#include +#include @@ -7279,6 +7654,27 @@ new file mode 100644 + const bool enabled = access("/tmp/metal-submit-thread", F_OK) == 0; + }; + ++ // One-shot completion token for a queued task: the worker signals it at the end of the task, so a caller that ++ // only has to wait for *its own* commit (the present dispatch it just queued) does not wait for the tasks that ++ // were queued after it. The token must outlive the task, which the waiting caller guarantees. ++ struct Token { ++ std::mutex mutex; ++ std::condition_variable condvar; ++ bool done = false; ++ void signal() { ++ { ++ std::lock_guard lock(mutex); ++ done = true; ++ } ++ condvar.notify_all(); ++ } ++ // blocks until the task carrying this token has finished (counting the block under `site`) ++ void wait(const char* site) { ++ std::unique_lock lock(mutex); ++ Waits::blockOn(site, condvar, lock, [this] { return done; }); ++ } ++ }; ++ + inline State& state() { + static State* instance = new State; + return *instance; @@ -7376,11 +7772,11 @@ new file mode 100644 + } + + // block until every task queued so far has finished (no-op on the worker itself) -+ inline void drain() { ++ inline void drain(const char* site = "drain") { + auto& s = state(); + if (onWorker()) return; + std::unique_lock lock(s.mutex); -+ s.idle.wait(lock, [&] { return s.queue.empty() && s.active == 0; }); ++ Waits::blockOn(site, s.idle, lock, [&] { return s.queue.empty() && s.active == 0; }); + } + } +} @@ -7410,3 +7806,397 @@ index b000d25..98baa9b 100644 + virtual void generateMipmapsForTexture(const std::shared_ptr& texture) = 0; }; }; +diff --git a/private-include/indium/deferred-blit-encoder.private.hpp b/private-include/indium/deferred-blit-encoder.private.hpp +new file mode 100644 +--- /dev/null ++++ b/private-include/indium/deferred-blit-encoder.private.hpp +@@ -0,0 +1,239 @@ ++#pragma once ++ ++// Deferred blit-command encoding. ++// ++// A blit encoder used to record its vkCmd* calls (and their barriers) directly into the command buffer, on the game's ++// thread, after waiting for every pending render replay of that command buffer (PrivateCommandBuffer::waitDeferred()). ++// Both the wait and the driver calls are on the frame's critical path, and nothing about them has to happen there: ++// Metal only requires that the calls are applied in order before the command buffer is committed. ++// ++// This facade implements the same public BlitCommandEncoder interface by recording the calls into a compact op list ++// (holding a reference to every buffer and texture they mention) and, at endEncoding(), posting the list to the ordered ++// worker of submit-thread.private.hpp -- the same FIFO the deferred render chunks and the commit itself run on. The ++// real PrivateBlitCommandEncoder is created, fed and ended there, in command-buffer order, before the commit that ++// closes the Vulkan command buffer. The deferred count keeps waitDeferred() correct for anything that still records ++// directly (compute encoders, and every encoder while deferred render encoding is off). ++// ++// Only used while deferred render encoding is on: with it off, render encoders record into the Vulkan command buffer ++// on the calling thread, so a blit replay has to stay on that thread too (the direct path keeps its waitDeferred()). ++// OFF by default (round-3 behaviour: blit encoders record directly on the calling thread after waitDeferred()). Guest ++// flag file /tmp/metal-deferred-blit (checked once per process) turns the facade on. Round 4 had it on by default; that ++// round regressed in matches, so it is an opt-in experiment now. ++ ++#include ++#include ++#include ++#include ++#include ++ ++#include ++#include ++#include ++#include ++#include ++#include ++ ++namespace Indium { ++ namespace AsyncEncode { ++ inline bool deferBlitEnabled() { ++ static const bool value = access("/tmp/metal-deferred-blit", F_OK) == 0; ++ return value; ++ } ++ ++ // Blits are short lists of large operations; one chunk per encoder (posted at endEncoding) is enough, and it ++ // keeps the whole pass on one worker task, i.e. in one command-buffer order. ++ enum class BlitKind: uint8_t { ++ BufferToBuffer, BufferToTexture, TextureToBuffer, TextureToTexture, TextureToTextureLevels, TextureToTextureRegion, Mipmaps, FillBuffer, ++ }; ++ ++ struct BlitOp { ++ BlitKind kind = BlitKind::BufferToBuffer; ++ uint8_t value = 0; ++ // per-kind arguments, in the order of the call (offsets, slices, levels, row/image strides) ++ size_t n[8] = {0, 0, 0, 0, 0, 0, 0, 0}; ++ Origin origin {}; ++ Origin destinationOrigin {}; ++ Size size {}; ++ Range range {0, 0}; ++ std::shared_ptr source; ++ std::shared_ptr destination; ++ }; ++ ++ // One blit pass being recorded. `real` and `failed` are only touched by the worker. ++ struct BlitState { ++ std::shared_ptr commandBuffer; ++ BlitPassDescriptor descriptor; ++ std::shared_ptr real; ++ bool failed = false; ++ BlitState(std::shared_ptr cb, const BlitPassDescriptor& d): commandBuffer(std::move(cb)), descriptor(d) {} ++ }; ++ ++ inline void blitLog(const char* message) { ++ if (FILE* f = fopen("/tmp/metal-trace.log", "a")) { ++ fprintf(f, "async-blit: %s\n", message); ++ fclose(f); ++ } ++ } ++ ++ // Apply the recorded calls and end the real encoder. Runs on the worker. ++ inline void replayBlit(BlitState& st, std::vector& ops) { ++ PoolScope pool; ++ if (!st.real && !st.failed) { ++ try { ++ st.real = st.commandBuffer->blitCommandEncoderNow(st.descriptor); ++ } catch (const std::exception& e) { ++ st.failed = true; ++ if (FILE* f = fopen("/tmp/metal-trace.log", "a")) { fprintf(f, "async-blit: encoder creation failed: %s\n", e.what()); fclose(f); } ++ } ++ } ++ if (st.real) { ++ PrivateBlitCommandEncoder* real = st.real.get(); ++ for (BlitOp& op: ops) { ++ try { ++ switch (op.kind) { ++ case BlitKind::BufferToBuffer: ++ real->copy(std::static_pointer_cast(op.source), op.n[0], std::static_pointer_cast(op.destination), op.n[1], op.n[2]); ++ break; ++ case BlitKind::BufferToTexture: ++ real->copy(std::static_pointer_cast(op.source), op.n[0], op.n[1], op.n[2], op.size, ++ std::static_pointer_cast(op.destination), op.n[3], op.n[4], op.origin); ++ break; ++ case BlitKind::TextureToBuffer: ++ real->copy(std::static_pointer_cast(op.source), op.n[0], op.n[1], op.origin, op.size, ++ std::static_pointer_cast(op.destination), op.n[2], op.n[3], op.n[4]); ++ break; ++ case BlitKind::TextureToTexture: ++ real->copy(std::static_pointer_cast(op.source), std::static_pointer_cast(op.destination)); ++ break; ++ case BlitKind::TextureToTextureLevels: ++ real->copy(std::static_pointer_cast(op.source), op.n[0], op.n[1], ++ std::static_pointer_cast(op.destination), op.n[2], op.n[3], op.n[4], op.n[5]); ++ break; ++ case BlitKind::TextureToTextureRegion: ++ real->copy(std::static_pointer_cast(op.source), op.n[0], op.n[1], op.origin, op.size, ++ std::static_pointer_cast(op.destination), op.n[2], op.n[3], op.destinationOrigin); ++ break; ++ case BlitKind::Mipmaps: ++ real->generateMipmapsForTexture(std::static_pointer_cast(op.source)); ++ break; ++ case BlitKind::FillBuffer: ++ real->fillBuffer(std::static_pointer_cast(op.source), op.range, op.value); ++ break; ++ } ++ } catch (const std::exception& e) { ++ if (FILE* f = fopen("/tmp/metal-trace.log", "a")) { fprintf(f, "async-blit: op %d failed: %s\n", (int) op.kind, e.what()); fclose(f); } ++ } ++ } ++ try { real->endEncoding(); } catch (...) {} ++ st.real.reset(); ++ } ++ } ++ ++ class DeferredBlitCommandEncoder final: public BlitCommandEncoder { ++ std::shared_ptr _state; ++ std::vector _ops; ++ bool _begun = false; ++ bool _ended = false; ++ ++ // Hand everything recorded so far to the worker. The ops own the resources they reference, so the game may ++ // release them (and us) as soon as this returns. ++ void flush() { ++ auto st = _state; ++ auto cb = st->commandBuffer; ++ if (!_begun) { _begun = true; cb->deferredBegin(); } ++ auto ops = std::make_shared>(std::move(_ops)); ++ _ops.clear(); ++ if (!cb->replayAllowed()) { ++ // an encoder dropped after the commit: the Vulkan command buffer is closed, so balance the deferred ++ // count and drop the ops instead of recording into it ++ blitLog("encoder dropped after commit"); ++ cb->deferredEnd(); ++ return; ++ } ++ SubmitThread::post([st, cb, ops]() mutable { ++ replayBlit(*st, *ops); ++ st.reset(); ++ cb->deferredEnd(); ++ }); ++ } ++ ++ BlitOp& add(BlitKind kind) { ++ _ops.emplace_back(); ++ BlitOp& op = _ops.back(); ++ op.kind = kind; ++ return op; ++ } ++ ++ public: ++ DeferredBlitCommandEncoder(std::shared_ptr cb, const BlitPassDescriptor& d): ++ _state(std::make_shared(std::move(cb), d)) {} ++ explicit DeferredBlitCommandEncoder(std::shared_ptr cb): ++ DeferredBlitCommandEncoder(std::move(cb), BlitPassDescriptor {}) {} ++ ++ ~DeferredBlitCommandEncoder() override { endEncoding(); } ++ ++ void endEncoding() override { ++ if (_ended) return; ++ _ended = true; ++ flush(); ++ _state.reset(); ++ } ++ ++ // The real encoder rejects blit options; doing it here keeps the exception on the calling thread. ++ static void rejectOptions(BlitOption options) { ++ if (options != BlitOption::None) throw std::runtime_error("TODO: support blit options"); ++ } ++ ++ void copy(const std::shared_ptr& source, size_t sourceOffset, const std::shared_ptr& destination, size_t destinationOffset, size_t size) override { ++ BlitOp& op = add(BlitKind::BufferToBuffer); ++ op.source = source; op.destination = destination; ++ op.n[0] = sourceOffset; op.n[1] = destinationOffset; op.n[2] = size; ++ } ++ void copy(const std::shared_ptr& source, size_t sourceOffset, size_t sourceBytesPerRow, size_t sourceBytesPerImage, Size sourceSize, const std::shared_ptr& destination, size_t destinationSlice, size_t destinationLevel, Origin destinationOrigin, BlitOption options = BlitOption::None) override { ++ rejectOptions(options); ++ BlitOp& op = add(BlitKind::BufferToTexture); ++ op.source = source; op.destination = destination; ++ op.n[0] = sourceOffset; op.n[1] = sourceBytesPerRow; op.n[2] = sourceBytesPerImage; ++ op.n[3] = destinationSlice; op.n[4] = destinationLevel; ++ op.origin = destinationOrigin; op.size = sourceSize; ++ } ++ void copy(const std::shared_ptr& source, size_t sourceSlice, size_t sourceLevel, Origin sourceOrigin, Size sourceSize, const std::shared_ptr& destination, size_t destinationOffset, size_t destinationBytesPerRow, size_t destinationBytesPerImage, BlitOption options = BlitOption::None) override { ++ rejectOptions(options); ++ BlitOp& op = add(BlitKind::TextureToBuffer); ++ op.source = source; op.destination = destination; ++ op.n[0] = sourceSlice; op.n[1] = sourceLevel; ++ op.n[2] = destinationOffset; op.n[3] = destinationBytesPerRow; op.n[4] = destinationBytesPerImage; ++ op.origin = sourceOrigin; op.size = sourceSize; ++ } ++ void copy(const std::shared_ptr& source, const std::shared_ptr& destination) override { ++ BlitOp& op = add(BlitKind::TextureToTexture); ++ op.source = source; op.destination = destination; ++ } ++ void copy(const std::shared_ptr& source, size_t sourceSlice, size_t sourceLevel, const std::shared_ptr& destination, size_t destinationSlice, size_t destinationLevel, size_t sliceCount, size_t levelCount) override { ++ BlitOp& op = add(BlitKind::TextureToTextureLevels); ++ op.source = source; op.destination = destination; ++ op.n[0] = sourceSlice; op.n[1] = sourceLevel; op.n[2] = destinationSlice; op.n[3] = destinationLevel; ++ op.n[4] = sliceCount; op.n[5] = levelCount; ++ } ++ void copy(const std::shared_ptr& source, size_t sourceSlice, size_t sourceLevel, Origin sourceOrigin, Size sourceSize, const std::shared_ptr& destination, size_t destinationSlice, size_t destinationLevel, Origin destinationOrigin) override { ++ BlitOp& op = add(BlitKind::TextureToTextureRegion); ++ op.source = source; op.destination = destination; ++ op.n[0] = sourceSlice; op.n[1] = sourceLevel; op.n[2] = destinationSlice; op.n[3] = destinationLevel; ++ op.origin = sourceOrigin; op.destinationOrigin = destinationOrigin; op.size = sourceSize; ++ } ++ void fillBuffer(const std::shared_ptr& buffer, Range range, uint8_t value) override { ++ // the real encoder rejects misaligned ranges; keep the exception on the calling thread ++ if ((range.start & 3) != 0) throw std::runtime_error("Misaligned fillBuffer offset"); ++ if ((range.length & 3) != 0) throw std::runtime_error("Misaligned fillBuffer length"); ++ BlitOp& op = add(BlitKind::FillBuffer); ++ op.source = buffer; ++ op.range = range; ++ op.value = value; ++ } ++ void generateMipmapsForTexture(const std::shared_ptr& texture) override { ++ BlitOp& op = add(BlitKind::Mipmaps); ++ op.source = texture; ++ } ++ }; ++ } ++} +diff --git a/private-include/indium/waits.private.hpp b/private-include/indium/waits.private.hpp +new file mode 100644 +--- /dev/null ++++ b/private-include/indium/waits.private.hpp +@@ -0,0 +1,145 @@ ++#pragma once ++ ++// Wait-site counters. ++// ++// The main thread's frame budget is spent partly on blocking in our own hand-offs (the submit worker's drain, the ++// deferred-encoder sync, texture uploads) and in the compositor's drawable hand-out. A sample profile cannot attribute ++// that time (the blocked PC is inside a futex helper), so every wait site counts how often the calling thread *actually ++// blocked* and how long, and appends a summary to the guest's /tmp/metal-trace.log every 5 s. ++// ++// Everything here is always compiled in; the counters are only active when /tmp/metal-debug-waits exists when the ++// process starts (checked once). Disabled, each site costs one predictable branch: a call site must establish that it ++// is about to block (a false predicate under the lock, a free-drawable check) before it constructs a Block, and that ++// check is the only work done when the flag is absent. ++// ++// Header-only: the counter state is per dylib (each library writes its own summary), so no class gains a member. The ++// functions are declared hidden: an inline function's static locals are otherwise exported as weak definitions (even ++// under -fvisibility-inlines-hidden), and dyld coalesces those across images, so libindium and QuartzCore would share ++// one copy -- harmless only while both images were built from the same version of this header. Hidden keeps the state ++// in each image, as documented. ++// ++// Flag file pitfall (why a match can show no waits lines although the file "exists"): the guest /tmp is ++// ~/.darling/private/tmp *inside Darling's overlay mount*, i.e. the host path is the overlay's upper directory. A file ++// created there from the host while the Darling namespace is running is not guaranteed to be visible to guest ++// processes: once a guest process has looked the name up and got ENOENT, the overlay caches the negative entry and ++// keeps answering ENOENT. Create flag files from inside the guest (`darling shell touch /tmp/metal-debug-waits`) or ++// while Darling is shut down. When the counters are on, the first check writes a "waits enabled" line, so its absence ++// from /tmp/metal-trace.log means the flag was not seen. ++ ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++ ++#define INDIUM_WAITS_LOCAL __attribute__((visibility("hidden"))) ++ ++namespace Indium { ++ namespace Waits { ++ // The summaries append to the guest's /tmp/metal-trace.log like the other diagnostics; DARLING_WAITS_LOG ++ // overrides the path so the host-side test does not append to the runtime's log. Only read when the counters ++ // are on (i.e. from the dump path). ++ INDIUM_WAITS_LOCAL inline const char* logPath() { ++ static const char* path = []() -> const char* { ++ const char* override = getenv("DARLING_WAITS_LOG"); ++ return override && *override ? override : "/tmp/metal-trace.log"; ++ }(); ++ return path; ++ } ++ ++ // Guest flag file, read once per process (per image) at the first wait site that is reached. When it is on, that ++ // first check appends one "waits enabled" line naming the image, so a match log proves the path is live even if ++ // no site ever blocks. ++ INDIUM_WAITS_LOCAL inline bool readEnabled() { ++ if (access("/tmp/metal-debug-waits", F_OK) != 0) return false; ++ Dl_info info {}; ++ const char* image = dladdr(reinterpret_cast(&readEnabled), &info) && info.dli_fname ? info.dli_fname : "?"; ++ if (FILE* log = fopen(logPath(), "a")) { ++ fprintf(log, "waits enabled@%ld: pid=%d image=%s\n", (long) time(nullptr), (int) getpid(), image); ++ fclose(log); ++ } ++ return true; ++ } ++ ++ INDIUM_WAITS_LOCAL inline bool enabled() { ++ static const bool value = readEnabled(); ++ return value; ++ } ++ ++ // Monotonic nanoseconds. steady_clock is what the rest of Indium uses (libc++ maps it to CLOCK_UPTIME_RAW, i.e. ++ // mach_absolute_time, on Darwin); clock_gettime(CLOCK_MONOTONIC) is wall-clock-derived in Darwin's libc ++ // (gettimeofday minus the boot time, with a kern.boottime sysctl fallback), so it is neither monotonic nor cheap ++ // under Darling. ++ INDIUM_WAITS_LOCAL inline uint64_t now() { ++ return static_cast(std::chrono::duration_cast( ++ std::chrono::steady_clock::now().time_since_epoch()).count()); ++ } ++ ++ INDIUM_WAITS_LOCAL inline void record(const char* site, uint64_t nanoseconds) { ++ struct Stat { uint64_t blocks = 0, sum = 0, max = 0; }; ++ static std::mutex lock; ++ static std::map stats; ++ static time_t last = 0; ++ std::lock_guard guard(lock); ++ Stat& stat = stats[site]; ++ ++stat.blocks; ++ stat.sum += nanoseconds; ++ if (nanoseconds > stat.max) stat.max = nanoseconds; ++ time_t now = time(nullptr); ++ if (now - last >= 5) { ++ last = now; ++ if (FILE* log = fopen(logPath(), "a")) { ++ fprintf(log, "waits@%ld:", (long) now); ++ for (auto& entry: stats) { ++ fprintf(log, " %s[n=%llu sum=%.1fus max=%.1fus]", entry.first.c_str(), ++ (unsigned long long) entry.second.blocks, entry.second.sum / 1000.0, entry.second.max / 1000.0); ++ } ++ fprintf(log, "\n"); ++ fclose(log); ++ } ++ stats.clear(); ++ } ++ } ++ ++ // One blocking episode of `site`. The constructor checks the flag and the destructor records (a no-op when the ++ // counter is off). Use it around a wait that is known to block; blockOn()/blockOnLock() do that check for the ++ // sites where the caller cannot know in advance. ++ struct INDIUM_WAITS_LOCAL Block { ++ const char* site; ++ uint64_t start = 0; ++ explicit Block(const char* site_): site(site_) { if (enabled()) start = now(); } ++ ~Block() { if (start != 0) record(site, now() - start); } ++ }; ++ ++ // Wait for `predicate` on `condvar`, counting the block only when the predicate is already false at the call ++ // (i.e. the wait really has to sleep). The predicate is re-checked by the wait itself, so both paths are safe. ++ template ++ INDIUM_WAITS_LOCAL inline void blockOn(const char* site, std::condition_variable& condvar, std::unique_lock& lock, Predicate predicate) { ++ if (!enabled() || predicate()) { ++ condvar.wait(lock, predicate); ++ return; ++ } ++ Block block(site); ++ condvar.wait(lock, predicate); ++ } ++ ++ // Lock `mutex`, counting the block only when it is contended at the time of the call. try_lock only runs when ++ // the flag is on, so the disabled cost is the same single branch. ++ INDIUM_WAITS_LOCAL inline void blockOnLock(const char* site, std::mutex& mutex) { ++ if (!enabled()) { ++ mutex.lock(); ++ return; ++ } ++ if (mutex.try_lock()) return; ++ Block block(site); ++ mutex.lock(); ++ } ++ } ++} diff --git a/patches/metal.patch b/patches/metal.patch index e2fe41b..d529851 100644 --- a/patches/metal.patch +++ b/patches/metal.patch @@ -2639,10 +2639,10 @@ index 37cccb3..79b2350 100644 - @end diff --git a/src/Metal/MTLVertexDescriptor.mm b/src/Metal/MTLVertexDescriptor.mm -index cde968d..a3819d0 100644 +index cde968d..aa7fe05 100644 --- a/src/Metal/MTLVertexDescriptor.mm +++ b/src/Metal/MTLVertexDescriptor.mm -@@ -1,24 +1,205 @@ +@@ -1,24 +1,435 @@ // SPDX-FileCopyrightText: 2026 Darling Developers // SPDX-License-Identifier: MPL-2.0 @@ -2656,15 +2656,25 @@ index cde968d..a3819d0 100644 - -- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector +#import ++#import +#import +// The bridge owns these concrete descriptor implementations. Keep their +// scalar storage accessible within this translation unit so reset/conversion +// can batch it. Subclasses continue through their public property methods. ++// Every item is created by an array for one slot, and the descriptor resets a slot by writing ++// the defaults into the item that holds it (in place, so an item a caller kept reads them back). ++// The array therefore hands each item the word and bit of the descriptor mask that reset reads ++// (`_dirtyWord`/`_dirtyBit`, see MTLVertexDescriptor below): a setter marks its own slot, and ++// reset only has to visit the slots that may differ from the defaults instead of every item in ++// the table. An item that was not handed a slot (a copy, or an array nobody owns) marks nothing, ++// which is what the "always" mask in the descriptor is for. +@interface MTLVertexBufferLayoutDescriptor () { +@public + NSUInteger _stride; + MTLVertexStepFunction _stepFunction; + NSUInteger _stepRate; ++ uint32_t* _dirtyWord; // the descriptor's layout mask, or NULL when the item has no slot ++ uint32_t _dirtyBit; // 1u << slot, meaningful while _dirtyWord is not NULL +} +@end +@interface MTLVertexAttributeDescriptor () { @@ -2672,12 +2682,98 @@ index cde968d..a3819d0 100644 + MTLVertexFormat _format; + NSUInteger _offset; + NSUInteger _bufferIndex; ++ uint32_t* _dirtyWord; // the descriptor's attribute mask, or NULL when the item has no slot ++ uint32_t _dirtyBit; // 1u << slot, meaningful while _dirtyWord is not NULL +} +@end ++// The masks that make -reset cheap, declared here so the two arrays (same file) can set them. ++// (darling-lol:) A descriptor's reset used to write the defaults into every item the arrays had ++// ever handed out - 16-31 attributes plus 16-31 layouts for the game's descriptors, each item a ++// separate allocation last touched a frame ago, i.e. a cache miss per item even though only the ++// few slots the game refilled in the previous pass could differ from the defaults. The game ++// profile put 1.5% of the main thread there (build/perf-research/metal-vdesc/REPORT.md). ++// `_attrDirty`/`_layoutDirty` name the slots whose values may differ from the defaults and must be ++// written by the next reset; `_attrPinned`/`_layoutPinned` name the slots whose item is not ++// tracked (a subclass instance, an item installed with setObject:, or a copy), which are written ++// every time. Over-marking is always safe - the reset writes the defaults either way - so the ++// only invariant that has to hold is that a slot whose value can differ from the defaults is ++// marked. Mutations mark through the item (see MTLVertexDescriptorItemTouched), which is why ++// every fallback pins the slot instead of tracking it. ++@interface MTLVertexDescriptor () { ++@public ++ MTLVertexBufferLayoutDescriptorArray* _layouts; ++ MTLVertexAttributeDescriptorArray* _attributes; ++ uint32_t _attrDirty; // attributes: slots to write at the next reset ++ uint32_t _attrPinned; // attributes: slots whose item cannot be tracked ++ uint32_t _layoutDirty; // layouts: slots to write at the next reset ++ uint32_t _layoutPinned; // layouts: slots whose item cannot be tracked ++} ++@end ++// "Is this item exactly the bridge class?" - the class that declares the plain ivars both the ++// reset and the marking write. A hook rather than the inline expression it replaces, so the host ++// model (tools/riot-metal-bind-model) can compile the helpers that call it. ++static inline bool MTLVertexBufferLayoutItemIsExactClass(const MTLVertexBufferLayoutDescriptor* item) + { +- return [NSMethodSignature signatureWithObjCTypes: "v@:"]; ++ return object_getClass((id) item) == [MTLVertexBufferLayoutDescriptor class]; + } +- +-- (void)forwardInvocation:(NSInvocation *)anInvocation ++static inline bool MTLVertexAttributeItemIsExactClass(const MTLVertexAttributeDescriptor* item) + { +- NSLog(@"Stub called: %@ in %@", NSStringFromSelector([anInvocation selector]), [self class]); ++ return object_getClass((id) item) == [MTLVertexAttributeDescriptor class]; ++} ++// Marking an untouched item is harmless (reset would write the same default value), so the ++// marking is allowed to over-approximate; it must never miss a mutation. See the model in ++// tools/riot-metal-bind-model/ for the mutation paths this covers. ++static inline void MTLVertexDescriptorItemTouched(uint32_t* word, uint32_t bit) ++{ ++ if (word) *word |= bit; ++} ++// The decision a -setObject:atIndexedSubscript: makes for one slot: 1 when the slot's item is one ++// this descriptor tracks at exactly this slot (the caller marks the dirty bit), 2 when the slot ++// has to be pinned because the item is not trackable (a copy with no slot, an item of another ++// descriptor, or the same item at another slot). Only the concrete classes declare the two fields, ++// so a subclass instance is never passed here (see the caller). ++static inline int MTLVertexSlotTrackingFor(uint32_t* itemWord, uint32_t itemBit, uint32_t* ownerWord, uint32_t bit) ++{ ++ if (!ownerWord) return 2; ++ if (!itemWord) return 2; ++ if (itemWord != ownerWord) return 2; ++ return itemBit == bit ? 1 : 2; ++} ++static bool MTLVertexDescriptorResetCounters() ++{ ++ static const bool enabled = MTLDebugEnabled() || access("/tmp/metal-debug-binds", F_OK) == 0; ++ return enabled; ++} ++static void MTLVertexDescriptorResetCount(unsigned long written, unsigned long skipped) ++{ ++ static std::mutex lock; ++ static unsigned long calls = 0, writtenTotal = 0, skippedTotal = 0; ++ static time_t last = 0; ++ std::lock_guard guard(lock); ++ ++calls; writtenTotal += written; skippedTotal += skipped; ++ time_t now = time(nullptr); ++ if (now - last >= 3) { ++ last = now; ++ if (FILE* log = fopen("/tmp/metal-trace.log", "a")) { ++ fprintf(log, "vdesc-reset@%ld: calls=%lu slotsWritten=%lu slotsAtDefaults=%lu\n", ++ (long) now, calls, writtenTotal, skippedTotal); ++ fclose(log); ++ } ++ } ++} +@implementation MTLVertexBufferLayoutDescriptor +@synthesize stride = _stride; +@synthesize stepFunction = _stepFunction; +@synthesize stepRate = _stepRate; ++// The setters are hand-written so they can mark their slot (over-marking never changes what a ++// reset writes); the getters stay synthesized. ++- (void)setStride:(NSUInteger)stride { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stride = stride; } ++- (void)setStepFunction:(MTLVertexStepFunction)stepFunction { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepFunction = stepFunction; } ++- (void)setStepRate:(NSUInteger)stepRate { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepRate = stepRate; } +- (instancetype)init { self = [super init]; if (self) { _stepFunction = MTLVertexStepFunctionPerVertex; _stepRate = 1; } return self; } +- (id)copyWithZone:(NSZone*)zone { MTLVertexBufferLayoutDescriptor* copy = [[[self class] allocWithZone:zone] init]; copy.stride = _stride; copy.stepFunction = _stepFunction; copy.stepRate = _stepRate; return copy; } +@end @@ -2690,6 +2786,7 @@ index cde968d..a3819d0 100644 + MTLVertexBufferLayoutDescriptor* _items[31]; + uint32_t _present; // bit i: _items[i] exists + uint32_t _concrete; // bit i: _items[i] is exactly MTLVertexBufferLayoutDescriptor (ivars can be written directly) ++ MTLVertexDescriptor* _owner; // the descriptor whose mask the items mark (not retained; nil after it dies) +} +- (MTLVertexBufferLayoutDescriptor*)existingObjectAtIndex:(NSUInteger)index; +- (void)resetAll; @@ -2699,6 +2796,7 @@ index cde968d..a3819d0 100644 + MTLVertexAttributeDescriptor* _items[31]; + uint32_t _present; // bit i: _items[i] exists + uint32_t _concrete; // bit i: _items[i] is exactly MTLVertexAttributeDescriptor (ivars can be written directly) ++ MTLVertexDescriptor* _owner; // the descriptor whose mask the items mark (not retained; nil after it dies) +} +- (MTLVertexAttributeDescriptor*)existingObjectAtIndex:(NSUInteger)index; +- (void)resetAll; @@ -2708,27 +2806,24 @@ index cde968d..a3819d0 100644 +// line so the common all-concrete loop does not carry the selector references and +// the message-send stub in its preheader. +__attribute__((noinline)) static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item) - { -- return [NSMethodSignature signatureWithObjCTypes: "v@:"]; ++{ + item.stride = 0; item.stepFunction = MTLVertexStepFunctionPerVertex; item.stepRate = 1; - } -- --- (void)forwardInvocation:(NSInvocation *)anInvocation ++} +__attribute__((noinline)) static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item) - { -- NSLog(@"Stub called: %@ in %@", NSStringFromSelector([anInvocation selector]), [self class]); ++{ + item.format = MTLVertexFormatInvalid; item.offset = 0; item.bufferIndex = 0; +} -+// Reset every entry the array has handed out, in place: an object a caller still -+// holds must read back the defaults. Exact-class entries are written through -+// their ivars; the present-but-non-concrete mask is taken once so the loop does -+// not reload it per entry. -+static void MTLVertexBufferLayoutArrayResetItems(MTLVertexBufferLayoutDescriptorArray* array) ++// Reset the slots whose mask bit is set, in ascending order like the old full-table loop: an ++// object a caller still holds must read back the defaults, and absent (nil) slots stay absent. ++// Exact-class entries are written through their ivars; an entry that is not exactly the bridge ++// class (a subclass an array was handed, or a graft) keeps its own property setters. ++static void MTLVertexBufferLayoutArrayResetSlots(MTLVertexBufferLayoutDescriptorArray* array, uint32_t slots) +{ -+ const uint32_t exotic = array->_present & ~array->_concrete; -+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) { ++ const uint32_t exotic = slots & ~array->_concrete; ++ for (uint32_t bits = slots; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexBufferLayoutDescriptor* item = array->_items[i]; ++ if (!item) continue; + if (exotic & (1u << i)) { + MTLVertexBufferLayoutSetDefaults(item); + } else { @@ -2736,12 +2831,13 @@ index cde968d..a3819d0 100644 + } + } +} -+static void MTLVertexAttributeArrayResetItems(MTLVertexAttributeDescriptorArray* array) ++static void MTLVertexAttributeArrayResetSlots(MTLVertexAttributeDescriptorArray* array, uint32_t slots) +{ -+ const uint32_t exotic = array->_present & ~array->_concrete; -+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) { ++ const uint32_t exotic = slots & ~array->_concrete; ++ for (uint32_t bits = slots; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexAttributeDescriptor* item = array->_items[i]; ++ if (!item) continue; + if (exotic & (1u << i)) { + MTLVertexAttributeSetDefaults(item); + } else { @@ -2749,32 +2845,107 @@ index cde968d..a3819d0 100644 + } + } +} ++// An item that leaves its slot (a -setObject: replaces it, or installs nil) may still be held by ++// the caller. If no slot of this array holds it any more, its mask pointer must be cleared: the ++// descriptor's -dealloc only revokes the items its arrays still hold, so a stale pointer would ++// write into the descriptor's storage after it is gone. An item that is still in a slot must keep ++// its pointer - that slot is tracked through it. ++static void MTLVertexBufferLayoutArrayRetireItem(MTLVertexBufferLayoutDescriptorArray* array, MTLVertexBufferLayoutDescriptor* item) ++{ ++ if (!item || !array->_owner) return; ++ if (!MTLVertexBufferLayoutItemIsExactClass(item)) return; ++ if (item->_dirtyWord != &array->_owner->_layoutDirty) return; ++ for (int i = 0; i < 31; ++i) if (array->_items[i] == item) return; ++ item->_dirtyWord = NULL; item->_dirtyBit = 0; ++} ++static void MTLVertexAttributeArrayRetireItem(MTLVertexAttributeDescriptorArray* array, MTLVertexAttributeDescriptor* item) ++{ ++ if (!item || !array->_owner) return; ++ if (!MTLVertexAttributeItemIsExactClass(item)) return; ++ if (item->_dirtyWord != &array->_owner->_attrDirty) return; ++ for (int i = 0; i < 31; ++i) if (array->_items[i] == item) return; ++ item->_dirtyWord = NULL; item->_dirtyBit = 0; ++} ++// The arrays are released with the descriptor, but a caller may hold an array (or an item) past ++// it: the array must then not reach the dead descriptor again, and an item that still points at ++// the descriptor's mask must not mark it. Both are cleared here, while the items are alive (the ++// descriptor keeps the arrays, and they keep the items). ++static void MTLVertexBufferLayoutArrayRevokeItems(MTLVertexBufferLayoutDescriptorArray* array, uint32_t* word) ++{ ++ if (!array) return; ++ array->_owner = nil; ++ for (uint32_t bits = array->_present; bits; bits &= bits - 1) { ++ const int i = __builtin_ctz(bits); ++ MTLVertexBufferLayoutDescriptor* item = array->_items[i]; ++ if (item && MTLVertexBufferLayoutItemIsExactClass(item) && item->_dirtyWord == word) { ++ item->_dirtyWord = NULL; item->_dirtyBit = 0; ++ } ++ } ++} ++static void MTLVertexAttributeArrayRevokeItems(MTLVertexAttributeDescriptorArray* array, uint32_t* word) ++{ ++ if (!array) return; ++ array->_owner = nil; ++ for (uint32_t bits = array->_present; bits; bits &= bits - 1) { ++ const int i = __builtin_ctz(bits); ++ MTLVertexAttributeDescriptor* item = array->_items[i]; ++ if (item && MTLVertexAttributeItemIsExactClass(item) && item->_dirtyWord == word) { ++ item->_dirtyWord = NULL; item->_dirtyBit = 0; ++ } ++ } ++} +// The arrays used to be NSMutableDictionaries keyed by boxed NSNumbers, and converting a descriptor created all 31 entries; +// under a game that builds vertex descriptors constantly that was ~30% of the render thread. Fixed storage, entries created lazily. +@implementation MTLVertexBufferLayoutDescriptorArray +- (void)dealloc { for (int i = 0; i < 31; ++i) [_items[i] release]; [super dealloc]; } +- (MTLVertexBufferLayoutDescriptor*)objectAtIndexedSubscript:(NSUInteger)index { + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; -+ if (!_items[index]) { _items[index] = [MTLVertexBufferLayoutDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; } ++ if (!_items[index]) { ++ _items[index] = [MTLVertexBufferLayoutDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; ++ // a fresh item holds the defaults, so nothing is marked until it is written; the slot is ++ // what a later setter marks ++ if (_owner) { _items[index]->_dirtyWord = &_owner->_layoutDirty; _items[index]->_dirtyBit = 1u << index; } ++ } + return _items[index]; +} +- (MTLVertexBufferLayoutDescriptor*)existingObjectAtIndex:(NSUInteger)index { return index < 31 ? _items[index] : nil; } +// reset in place: the game calls -[MTLVertexDescriptor reset] for every draw and then refills it, so keep the objects -+- (void)resetAll { MTLVertexBufferLayoutArrayResetItems(self); } ++- (void)resetAll { ++ MTLVertexBufferLayoutArrayResetSlots(self, _present); ++ if (_owner) _owner->_layoutDirty &= ~_present; ++} +- (void)setObject:(MTLVertexBufferLayoutDescriptor*)descriptor atIndexedSubscript:(NSUInteger)index { + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; -+ id old = _items[index]; ++ MTLVertexBufferLayoutDescriptor* old = _items[index]; + _items[index] = [descriptor retain]; + [old release]; ++ MTLVertexBufferLayoutArrayRetireItem(self, old); + const uint32_t bit = 1u << index; + if (descriptor) _present |= bit; else _present &= ~bit; + if (descriptor && object_getClass(descriptor) == [MTLVertexBufferLayoutDescriptor class]) _concrete |= bit; else _concrete &= ~bit; ++ // The next reset must write this slot unless its item is one this descriptor tracks at exactly ++ // this slot: a subclass with its own setters, an item that belongs to another descriptor or ++ // another slot, or an item with no slot at all is marked "always" instead, which keeps the ++ // reset exact (over-marking never changes what a reset writes). ++ if (_owner) { ++ if (!descriptor) { ++ _owner->_layoutDirty &= ~bit; ++ _owner->_layoutPinned &= ~bit; ++ } else { ++ const int tracking = MTLVertexBufferLayoutItemIsExactClass(descriptor) ++ ? MTLVertexSlotTrackingFor(descriptor->_dirtyWord, descriptor->_dirtyBit, &_owner->_layoutDirty, bit) ++ : 2; ++ if (tracking == 1) _owner->_layoutDirty |= bit; else _owner->_layoutPinned |= bit; ++ } ++ } +} +- (id)copyWithZone:(NSZone*)zone { + MTLVertexBufferLayoutDescriptorArray* copy = [[[self class] allocWithZone:zone] init]; + for (int i = 0; i < 31; ++i) if (_items[i]) copy->_items[i] = [_items[i] copy]; + copy->_present = _present; + copy->_concrete = _concrete; // -copy of a concrete item returns the same class ++ // the copied items are new objects and are not tracked; the descriptor that adopts this array ++ // binds them and marks their slots (see MTLVertexDescriptor -copyWithZone:) + return copy; +} +@end @@ -2782,25 +2953,50 @@ index cde968d..a3819d0 100644 +@synthesize format = _format; +@synthesize offset = _offset; +@synthesize bufferIndex = _bufferIndex; ++// Hand-written for the same reason as MTLVertexBufferLayoutDescriptor's: the setter marks the ++// descriptor slot this item fills. ++- (void)setFormat:(MTLVertexFormat)format { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _format = format; } ++- (void)setOffset:(NSUInteger)offset { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _offset = offset; } ++- (void)setBufferIndex:(NSUInteger)bufferIndex { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _bufferIndex = bufferIndex; } +- (id)copyWithZone:(NSZone*)zone { MTLVertexAttributeDescriptor* copy = [[[self class] allocWithZone:zone] init]; copy.format = _format; copy.offset = _offset; copy.bufferIndex = _bufferIndex; return copy; } +@end +@implementation MTLVertexAttributeDescriptorArray +- (void)dealloc { for (int i = 0; i < 31; ++i) [_items[i] release]; [super dealloc]; } +- (MTLVertexAttributeDescriptor*)objectAtIndexedSubscript:(NSUInteger)index { + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; -+ if (!_items[index]) { _items[index] = [MTLVertexAttributeDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; } ++ if (!_items[index]) { ++ _items[index] = [MTLVertexAttributeDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; ++ // a fresh item holds the defaults; the slot is what a later setter marks ++ if (_owner) { _items[index]->_dirtyWord = &_owner->_attrDirty; _items[index]->_dirtyBit = 1u << index; } ++ } + return _items[index]; +} +- (MTLVertexAttributeDescriptor*)existingObjectAtIndex:(NSUInteger)index { return index < 31 ? _items[index] : nil; } -+- (void)resetAll { MTLVertexAttributeArrayResetItems(self); } ++- (void)resetAll { ++ MTLVertexAttributeArrayResetSlots(self, _present); ++ if (_owner) _owner->_attrDirty &= ~_present; ++} +- (void)setObject:(MTLVertexAttributeDescriptor*)descriptor atIndexedSubscript:(NSUInteger)index { + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; -+ id old = _items[index]; ++ MTLVertexAttributeDescriptor* old = _items[index]; + _items[index] = [descriptor retain]; + [old release]; ++ MTLVertexAttributeArrayRetireItem(self, old); + const uint32_t bit = 1u << index; + if (descriptor) _present |= bit; else _present &= ~bit; + if (descriptor && object_getClass(descriptor) == [MTLVertexAttributeDescriptor class]) _concrete |= bit; else _concrete &= ~bit; ++ // see MTLVertexBufferLayoutDescriptorArray -setObject:atIndexedSubscript: for the rule ++ if (_owner) { ++ if (!descriptor) { ++ _owner->_attrDirty &= ~bit; ++ _owner->_attrPinned &= ~bit; ++ } else { ++ const int tracking = MTLVertexAttributeItemIsExactClass(descriptor) ++ ? MTLVertexSlotTrackingFor(descriptor->_dirtyWord, descriptor->_dirtyBit, &_owner->_attrDirty, bit) ++ : 2; ++ if (tracking == 1) _owner->_attrDirty |= bit; else _owner->_attrPinned |= bit; ++ } ++ } +} +- (id)copyWithZone:(NSZone*)zone { + MTLVertexAttributeDescriptorArray* copy = [[[self class] allocWithZone:zone] init]; @@ -2810,31 +3006,65 @@ index cde968d..a3819d0 100644 + return copy; +} +@end ++// The reset work itself, kept in a static so the host model can run the shipping code (see ++// tools/riot-metal-bind-model/extract-vertex-reset.py): the mask words name the slots whose values ++// may differ from the defaults, and the helpers write exactly those, in ascending slot order. ++static void MTLVertexDescriptorResetMarks(MTLVertexDescriptor* descriptor) ++{ ++ const uint32_t layouts = descriptor->_layoutDirty | descriptor->_layoutPinned; ++ const uint32_t attributes = descriptor->_attrDirty | descriptor->_attrPinned; ++ descriptor->_layoutDirty = descriptor->_layoutPinned; ++ descriptor->_attrDirty = descriptor->_attrPinned; ++ MTLVertexBufferLayoutArrayResetSlots(descriptor->_layouts, layouts); ++ MTLVertexAttributeArrayResetSlots(descriptor->_attributes, attributes); ++ if (MTLVertexDescriptorResetCounters()) { ++ MTLVertexDescriptorResetCount(__builtin_popcount(layouts) + __builtin_popcount(attributes), ++ __builtin_popcount(descriptor->_layouts->_present & ~layouts) + __builtin_popcount(descriptor->_attributes->_present & ~attributes)); ++ } ++} ++// Guest diagnosis, same shape as the bind counters in MTLRenderCommandEncoder.mm: with ++// /tmp/metal-debug-binds (or the trace flag) present every reset reports how many slots it had to ++// write and how many it left at the defaults, so the next match measures what the marks save. ++// The flag is tested inline at the call site, so an ordinary match pays one load and a branch. +@implementation MTLVertexDescriptor +@synthesize layouts = _layouts; +@synthesize attributes = _attributes; ++ (instancetype)vertexDescriptor { return [[[self alloc] init] autorelease]; } +- (instancetype)init { self = [super init]; if (self) [self reset]; return self; } -+- (void)dealloc { [_layouts release]; [_attributes release]; [super dealloc]; } ++- (void)dealloc { ++ // an item (or an array) a caller kept must not reach this descriptor's storage after this ++ MTLVertexBufferLayoutArrayRevokeItems(_layouts, &_layoutDirty); ++ MTLVertexAttributeArrayRevokeItems(_attributes, &_attrDirty); ++ [_layouts release]; [_attributes release]; [super dealloc]; ++} +- (void)reset { + if (_layouts && _attributes) { -+ // Direct call instead of [_layouts resetAll]/[_attributes resetAll]: the game -+ // resets a vertex descriptor for every draw. Both arrays are only ever created -+ // by -init/-reset/-copyWithZone: in this file and the two properties are -+ // readonly, so their storage is the concrete class. The per-entry subclass -+ // case below keeps using the property setters through their own message sends. -+ MTLVertexBufferLayoutArrayResetItems(_layouts); -+ MTLVertexAttributeArrayResetItems(_attributes); ++ // Only the slots whose values may differ from the defaults are written, in the same ++ // ascending order as before; the rest were left at the defaults by the previous reset (or ++ // hold a fresh item), so writing them would change nothing. The dirty marks are cleared as ++ // the slots are written; the pinned marks stay (their items are written every time). ++ MTLVertexDescriptorResetMarks(self); + return; + } + [_layouts release]; [_attributes release]; + _layouts = [MTLVertexBufferLayoutDescriptorArray new]; + _attributes = [MTLVertexAttributeDescriptorArray new]; ++ ((MTLVertexBufferLayoutDescriptorArray*)_layouts)->_owner = self; ++ ((MTLVertexAttributeDescriptorArray*)_attributes)->_owner = self; ++ _attrDirty = 0; _attrPinned = 0; _layoutDirty = 0; _layoutPinned = 0; +} +- (id)copyWithZone:(NSZone*)zone { + MTLVertexDescriptor* copy = [[[self class] allocWithZone:zone] init]; + [copy->_layouts release]; [copy->_attributes release]; + copy->_layouts = [_layouts copy]; copy->_attributes = [_attributes copy]; ++ // the copied items are new objects nothing tracks; bind them to the copy and let its resets ++ // write every slot they occupy (a copy is rare, and this keeps it exact without tracking it) ++ MTLVertexBufferLayoutDescriptorArray* layouts = copy->_layouts; ++ MTLVertexAttributeDescriptorArray* attributes = copy->_attributes; ++ layouts->_owner = copy; ++ attributes->_owner = copy; ++ copy->_layoutPinned = layouts->_present; ++ copy->_attrPinned = attributes->_present; + return copy; +} +#if DARLING_METAL_ENABLED diff --git a/tools/install-riot-metal-pair.sh b/tools/install-riot-metal-pair.sh index a7f0f93..115525b 100755 --- a/tools/install-riot-metal-pair.sh +++ b/tools/install-riot-metal-pair.sh @@ -1,7 +1,7 @@ #!/bin/sh # Install the Metal wrapper and Indium overlays built by build-riot-metal-runtime.py / build-riot-indium-runtime.py # as a pair (the Indium blit interface they share changed), with a backup of the files they replace. -# --rollback puts the backed-up pair back. Neither stops a process: exit the game and both clients, then +# RIOT_METAL_ARTIFACT / RIOT_INDIUM_ARTIFACT select other builds. --rollback puts the backed-up pair back. Neither stops a process: exit the game and both clients, then # `darling shutdown` and `sh scripts/run.sh` to activate. set -eu project_dir=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) @@ -13,8 +13,8 @@ if [ "${1:-}" = "--rollback" ]; then source_metal="$backups/Metal" source_indium="$backups/libindium.dylib" else - source_metal="$project_dir/build/riot-Metal" - source_indium="$project_dir/build/riot-libindium.dylib" + source_metal=${RIOT_METAL_ARTIFACT:-$project_dir/build/riot-Metal} + source_indium=${RIOT_INDIUM_ARTIFACT:-$project_dir/build/riot-libindium.dylib} mkdir -p "$backups" # a repeated install must not replace the backup with the pair it installed last time cmp -s "$metal" "$source_metal" || cp -p "$metal" "$backups/Metal" diff --git a/tools/riot-metal-bind-model/bind-refcount-model.cpp b/tools/riot-metal-bind-model/bind-refcount-model.cpp new file mode 100644 index 0000000..2099abc --- /dev/null +++ b/tools/riot-metal-bind-model/bind-refcount-model.cpp @@ -0,0 +1,120 @@ +// Host model of the render-encoder resource arguments (Metal wrapper + Indium). +// +// The wrapper used to hand each resource to Indium by value, `setFragmentTexture(std::shared_ptr< +// Texture>, size_t)`: the caller built a temporary shared_ptr for the argument (one atomic +// increment), stored it so the callee could move it into the recorded op, and the callee's +// parameter handling zeroed that temporary again; the deferred encoder then had to load back what +// the caller had just stored (the store-to-load chain the profile shows in +// DeferredRenderCommandEncoder::setFragmentTexture). The interface now takes +// `const std::shared_ptr&`, and the op takes exactly one reference into the record that +// keeps the resource alive until the worker replays it. +// +// What a model can prove here is the lifetime rule that makes both shapes correct - and that had +// to survive the change: the op owns a strong reference, so the resource outlives the game's own +// references and is still alive when the worker replays the op (this is what makes the round-2 +// bind record's raw Indium address meaningful: a recorded address cannot be reused while the op +// holds the resource). The model runs both call shapes over a counting reference count and checks +// that property, plus that the reference is released exactly once when the op is. +// +// The cost side (what the change removed) is measured in the report from the two builds' +// disassembly, not from this model: the by-value shape compiles an increment plus a stack copy in +// the caller and a store-to-load chain in the callee, the reference shape neither. +// +// Compile with: c++ -std=c++17 -O2 -Wall -Wextra -o bind-refcount-model bind-refcount-model.cpp +#include +#include +#include + +static unsigned long g_increments = 0, g_decrements = 0, g_destructions = 0, g_replays = 0; + +struct Resource { + int refs = 0; + bool alive = false; +}; + +static void IncRef(Resource* r) { ++g_increments; ++r->refs; r->alive = true; } +static void DecRef(Resource* r) +{ + ++g_decrements; + if (--r->refs == 0) { ++g_destructions; r->alive = false; } +} + +// a shared_ptr in shape: a strong reference, moved without touching the count +struct Shared { + Resource* p = nullptr; + Shared() {} + explicit Shared(Resource* r) : p(r) { if (p) IncRef(p); } + Shared(const Shared& o) : p(o.p) { if (p) IncRef(p); } + Shared(Shared&& o) noexcept : p(o.p) { o.p = nullptr; } + Shared& operator=(const Shared& o) { if (o.p) IncRef(o.p); if (p) DecRef(p); p = o.p; return *this; } + Shared& operator=(Shared&& o) noexcept { if (p) DecRef(p); p = o.p; o.p = nullptr; return *this; } + ~Shared() { if (p) DecRef(p); } +}; + +struct Op { Shared ref; }; // one recorded call: root of the resource's life + +struct DeferredEncoder { + std::vector ops; + // the shipping addRef(): `o.p = ref` takes one reference into the op + void record(const Shared& reference) { ops.emplace_back(); ops.back().ref = reference; } + // the old by-value signature: the parameter is a shared_ptr the callee moves into the op + void recordByValue(Shared parameter) { ops.emplace_back(); ops.back().ref = std::move(parameter); } + void replay() { for (auto& op: ops) if (op.ref.p && op.ref.p->alive) ++g_replays; ops.clear(); } +}; + +struct Wrapper { Shared resource; }; // the game's MTLTexture/MTLBuffer object + +// the shipping shape (from the wrapper's setFragmentTexture:atIndex: and the deferred encoder's +// setFragmentTexture(const std::shared_ptr&)): the wrapper passes its own reference +static void BindByReference(DeferredEncoder& encoder, const Wrapper& wrapper) +{ + encoder.record(wrapper.resource); +} + +// the pre-change shape: the by-value argument is a caller-side temporary of the resource (the +// builds without the change compile its store plus an increment here) and the callee moves it +// into the op, so the temporary is empty afterwards and only the *store-to-load* chain - not a +// second reference - separates the two shapes +static void BindByValue(DeferredEncoder& encoder, const Wrapper& wrapper) +{ + Shared argument(wrapper.resource); // the caller's temporary: one increment + encoder.recordByValue(std::move(argument)); // the callee moves it into the op +} + +static bool Run(const char* name, bool byValue) +{ + g_increments = g_decrements = g_destructions = g_replays = 0; + Resource resource; + Shared gameReference(&resource); // the game's reference to the Indium object + bool aliveAtReplay = false; + unsigned long incrementsDuringBind = 0, decrementsDuringBind = 0; + { + Wrapper wrapper; + wrapper.resource = gameReference; // the wrapper holds its own reference + DeferredEncoder encoder; + const unsigned long inc0 = g_increments, dec0 = g_decrements; + if (byValue) BindByValue(encoder, wrapper); else BindByReference(encoder, wrapper); + incrementsDuringBind = g_increments - inc0; + decrementsDuringBind = g_decrements - dec0; + // the game drops every reference of its own before the worker replays the op + wrapper.resource = Shared(); + gameReference = Shared(); + aliveAtReplay = !encoder.ops.empty() && encoder.ops.front().ref.p && encoder.ops.front().ref.p->alive; + encoder.replay(); + } + printf("bind-refcount %-10s inc=%lu dec=%lu duringBind (inc=%lu dec=%lu) resourceAliveAtReplay=%s replaySeesResource=%s releasedAfterReplay=%s\n", + name, g_increments, g_decrements, incrementsDuringBind, decrementsDuringBind, + aliveAtReplay ? "yes" : "no", g_replays == 1 ? "yes" : "no", g_destructions == 1 ? "yes" : "no"); + // the rule both shapes must keep: the op's reference outlives the game's own references, the + // replay still sees the resource, and the reference is released exactly once + return aliveAtReplay && g_replays == 1 && g_destructions == 1; +} + +int main() +{ + const bool oldOk = Run("by-value", true); + const bool newOk = Run("by-ref", false); + printf("BIND_REFCOUNT_MODEL by-value=%s by-ref=%s checks=%s\n", oldOk ? "ok" : "FAIL", newOk ? "ok" : "FAIL", + (oldOk && newOk) ? "pass" : "FAIL"); + return (oldOk && newOk) ? 0 : 1; +} diff --git a/tools/riot-metal-bind-model/extract-vertex-reset.py b/tools/riot-metal-bind-model/extract-vertex-reset.py new file mode 100644 index 0000000..162c18b --- /dev/null +++ b/tools/riot-metal-bind-model/extract-vertex-reset.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +"""Extract the shipping vertex-descriptor reset/marking helpers verbatim, for the host model. + +The model (vertex-reset-model.cpp) must exercise the code that ships, not a copy of it: this +pulls three regions out of src/Metal/MTLVertexDescriptor.mm - the item marking and the +slot-tracking decision, the reset/revoke helpers, and the reset-marks function - and writes them +as a C++ include fragment with the source line numbers in its header. It fails loudly if the +anchor comments move. + +Run it before compiling the model (tools/riot-metal-bind-model/run.sh does). +""" +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent.parent +SOURCE = ROOT / 'build/darling-src/src/external/metal/src/Metal/MTLVertexDescriptor.mm' +OUT = ROOT / 'build/riot-metal-bind-model/vertex-reset.extract.inc' +OUT.parent.mkdir(parents=True, exist_ok=True) + +# (name, start anchor, end anchor); the end anchor is exclusive +CHUNKS = [ + ('item-marks-and-tracking', '// Marking an untouched item is harmless', 'static bool MTLVertexDescriptorResetCounters()'), + ('reset-slots-and-revoke', 'static void MTLVertexBufferLayoutArrayResetSlots', '// The arrays used to be NSMutableDictionaries'), + ('reset-marks', '// The reset work itself, kept in a static', '// Guest diagnosis, same shape as the bind counters'), +] + +text = SOURCE.read_text().splitlines(True) +out = ['// Generated by extract-vertex-reset.py from %s. Do not edit.\n' % SOURCE.name] +for name, start_anchor, end_anchor in CHUNKS: + start = next((i for i, line in enumerate(text) if line.startswith(start_anchor)), None) + end = next((i for i, line in enumerate(text) if i > (start or 0) and line.startswith(end_anchor)), None) + if start is None or end is None or end <= start: + sys.exit('vertex-reset markers not found for %s in %s' % (name, SOURCE)) + out.append('// ---- %s (MTLVertexDescriptor.mm:%d-%d, verbatim) ----\n' % (name, start + 1, end)) + out.extend(text[start:end]) +region = ''.join(out).rstrip() + '\n' +OUT.write_text(region) +print('wrote %s (%d lines)' % (OUT, region.count('\n'))) diff --git a/tools/riot-metal-bind-model/run.sh b/tools/riot-metal-bind-model/run.sh index 6b53ddd..2dce2a7 100755 --- a/tools/riot-metal-bind-model/run.sh +++ b/tools/riot-metal-bind-model/run.sh @@ -1,11 +1,17 @@ #!/bin/sh -# Host models of the Metal wrapper's bind deduplication (the shipped helpers, extracted verbatim) and of the Indium -# blit cast change. Linux only; never runs a Mach-O binary. +# Host models of the Metal wrapper's bind deduplication (the shipped helpers, extracted verbatim), of the +# vertex-descriptor reset marks (ditto), of the Indium blit cast change and of the render-encoder resource +# arguments (const references + the deferred op's single reference). Linux only; never runs a Mach-O binary. set -e here=$(cd "$(dirname "$0")" && pwd) out=$(cd "$here/../.." && pwd)/build/riot-metal-bind-model python3 "$here/extract-bind-tracking.py" +python3 "$here/extract-vertex-reset.py" c++ -std=c++17 -O2 -Wall -I"$out" -o "$out/bind-dedup-model" "$here/bind-dedup-model.cpp" c++ -std=c++17 -O2 -Wall -o "$out/blit-cast-model" "$here/blit-cast-model.cpp" +c++ -std=c++17 -O2 -Wall -I"$out" -o "$out/vertex-reset-model" "$here/vertex-reset-model.cpp" +c++ -std=c++17 -O2 -Wall -o "$out/bind-refcount-model" "$here/bind-refcount-model.cpp" "$out/bind-dedup-model" "$out/blit-cast-model" +"$out/vertex-reset-model" +"$out/bind-refcount-model" diff --git a/tools/riot-metal-bind-model/vertex-reset-model.cpp b/tools/riot-metal-bind-model/vertex-reset-model.cpp new file mode 100644 index 0000000..d18989a --- /dev/null +++ b/tools/riot-metal-bind-model/vertex-reset-model.cpp @@ -0,0 +1,671 @@ +// Host model of the MTLVertexDescriptor reset marks (Metal wrapper, MTLVertexDescriptor.mm). +// +// The wrapper now writes the defaults only into the slots whose values may differ from them: +// every item carries the mask word and bit of the descriptor slot it fills, its setters mark that +// slot, and the -setObject:/copy paths pin the slots whose items cannot be tracked. Over-marking +// is the only safe direction, so the invariant this model checks after every operation is: +// * the values of the descriptor's items after reset are the defaults (like the pre-change +// "reset everything" behaviour), and +// * a slot whose item holds non-default values is marked (dirty or pinned). +// Two worlds are driven with the same operations: `shipped` runs the shipping helpers (extracted +// verbatim into vertex-reset.extract.inc) and `reference` uses the pre-change reset (every present +// item). Any missed mark makes the two worlds' values differ after a reset, which is counted as a +// divergence and printed. +// +// Compile with: c++ -std=c++17 -O2 -Wall -Wextra -o vertex-reset-model vertex-reset-model.cpp +#include +#include +#include +#include +#include +#include + +// --------------------------------------------------------------------------------------------- +// The model world: the same shapes as the wrapper's classes (ivar names match the shipping code), +// plus one flag the wrapper does not have (`_subclass`: an item a caller installed that is not the +// bridge class, i.e. an object whose setters this file does not control). +// --------------------------------------------------------------------------------------------- +using NSUInteger = unsigned long; +#define nil nullptr // the shipping helpers assign nil to mean "no owner/descriptor any more" +enum { MTLVertexFormatInvalid = 0, MTLVertexStepFunctionPerVertex = 1, MTLVertexStepFunctionPerInstance = 2 }; +enum { MTLVertexFormatFloat4 = 31 }; // the game's setup fills Float4 into every attribute + +// defined by the extracted shipping helpers (vertex-reset.extract.inc, included below) +static inline void MTLVertexDescriptorItemTouched(uint32_t* word, uint32_t bit); + +struct MTLVertexDescriptor; + +struct MTLVertexBufferLayoutDescriptor { + // ivars, in the shipping order (MTLVertexDescriptor.mm, MTLVertexBufferLayoutDescriptor ()) + NSUInteger _stride = 0; + NSUInteger _stepFunction = MTLVertexStepFunctionPerVertex; + NSUInteger _stepRate = 1; + uint32_t* _dirtyWord = nullptr; + uint32_t _dirtyBit = 0; + bool _subclass = false; // model only: setters do not mark (the wrapper cannot see them) + + // the shipping setters: they mark the slot (MTLVertexDescriptor.mm). The model adds the + // _subclass case: an item a caller installed is not the bridge class, so its own setters are + // the ones that run and they may not mark at all - that is what the pinned mask is for. + void setStride(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stride = v; } + void setStepFunction(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepFunction = v; } + void setStepRate(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepRate = v; } +}; +struct MTLVertexAttributeDescriptor { + NSUInteger _format = MTLVertexFormatInvalid; + NSUInteger _offset = 0; + NSUInteger _bufferIndex = 0; + uint32_t* _dirtyWord = nullptr; + uint32_t _dirtyBit = 0; + bool _subclass = false; + + void setFormat(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _format = v; } + void setOffset(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _offset = v; } + void setBufferIndex(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _bufferIndex = v; } +}; + +struct MTLVertexBufferLayoutDescriptorArray { + MTLVertexBufferLayoutDescriptor* _items[31] = {}; + uint32_t _present = 0; + uint32_t _concrete = 0; + MTLVertexDescriptor* _owner = nullptr; +}; +struct MTLVertexAttributeDescriptorArray { + MTLVertexAttributeDescriptor* _items[31] = {}; + uint32_t _present = 0; + uint32_t _concrete = 0; + MTLVertexDescriptor* _owner = nullptr; +}; +struct MTLVertexDescriptor { + MTLVertexBufferLayoutDescriptorArray* _layouts = nullptr; + MTLVertexAttributeDescriptorArray* _attributes = nullptr; + uint32_t _attrDirty = 0, _attrPinned = 0, _layoutDirty = 0, _layoutPinned = 0; +}; + +// the hooks the extracted helpers use (the shipping file defines them with object_getClass) +static inline bool MTLVertexBufferLayoutItemIsExactClass(const MTLVertexBufferLayoutDescriptor* item) { return !item->_subclass; } +static inline bool MTLVertexAttributeItemIsExactClass(const MTLVertexAttributeDescriptor* item) { return !item->_subclass; } +// the model has no trace log to write to +static bool MTLVertexDescriptorResetCounters() { return false; } +static void MTLVertexDescriptorResetCount(unsigned long, unsigned long) {} + +// the item's own setters (the shipping SetDefaults helpers call the public property setters) +static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item); +static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item); + +// the shipping helpers, verbatim (extract-vertex-reset.py) +#include "vertex-reset.extract.inc" + +// The shipping MTLVertexBufferLayoutSetDefaults/MTLVertexAttributeSetDefaults (MTLVertexDescriptor.mm) +// write the defaults through the item's own property setters, which is what an item that is not the +// bridge class needs; the same three values here. +static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item) +{ + item->setStride(0); item->setStepFunction(MTLVertexStepFunctionPerVertex); item->setStepRate(1); +} +static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item) +{ + item->setFormat(MTLVertexFormatInvalid); item->setOffset(0); item->setBufferIndex(0); +} + +// --------------------------------------------------------------------------------------------- +// World: a descriptor plus the items the driver holds on to (to graft them, as the game may hold +// an item across a reset and install it elsewhere), and a write counter so a scenario can report +// how many items a reset touched. +// --------------------------------------------------------------------------------------------- +struct World { + MTLVertexDescriptor* d = nullptr; + std::vector heldLayouts; + std::vector heldAttrs; + + void alloc() // -reset's fresh path + { + d = new MTLVertexDescriptor; + d->_layouts = new MTLVertexBufferLayoutDescriptorArray; + d->_attributes = new MTLVertexAttributeDescriptorArray; + d->_layouts->_owner = d; + d->_attributes->_owner = d; + d->_attrDirty = d->_attrPinned = d->_layoutDirty = d->_layoutPinned = 0; + } + + // the model's -objectAtIndexedSubscript: (creation; a fresh item is at the defaults, so it is + // not marked, but it knows its slot) + MTLVertexBufferLayoutDescriptor* layoutAt(NSUInteger index) + { + MTLVertexBufferLayoutDescriptorArray* a = d->_layouts; + if (!a->_items[index]) { + a->_items[index] = new MTLVertexBufferLayoutDescriptor; + a->_present |= 1u << index; + a->_concrete |= 1u << index; + if (a->_owner) { a->_items[index]->_dirtyWord = &a->_owner->_layoutDirty; a->_items[index]->_dirtyBit = 1u << index; } + } + return a->_items[index]; + } + MTLVertexAttributeDescriptor* attrAt(NSUInteger index) + { + MTLVertexAttributeDescriptorArray* a = d->_attributes; + if (!a->_items[index]) { + a->_items[index] = new MTLVertexAttributeDescriptor; + a->_present |= 1u << index; + a->_concrete |= 1u << index; + if (a->_owner) { a->_items[index]->_dirtyWord = &a->_owner->_attrDirty; a->_items[index]->_dirtyBit = 1u << index; } + } + return a->_items[index]; + } + + // the tracking half of -setObject:atIndexedSubscript: / removal with nil + void setLayout(NSUInteger index, MTLVertexBufferLayoutDescriptor* item) + { + MTLVertexBufferLayoutDescriptorArray* a = d->_layouts; + const uint32_t bit = 1u << index; + MTLVertexBufferLayoutDescriptor* old = a->_items[index]; + a->_items[index] = item; + MTLVertexBufferLayoutArrayRetireItem(a, old); // -setObject: retires the replaced item + if (item) a->_present |= bit; else a->_present &= ~bit; + if (item && !item->_subclass) a->_concrete |= bit; else a->_concrete &= ~bit; + if (a->_owner) { + if (!item) { + a->_owner->_layoutDirty &= ~bit; + a->_owner->_layoutPinned &= ~bit; + } else { + const int tracking = MTLVertexBufferLayoutItemIsExactClass(item) + ? MTLVertexSlotTrackingFor(item->_dirtyWord, item->_dirtyBit, &a->_owner->_layoutDirty, bit) + : 2; + if (tracking == 1) a->_owner->_layoutDirty |= bit; else a->_owner->_layoutPinned |= bit; + } + } + } + void setAttr(NSUInteger index, MTLVertexAttributeDescriptor* item) + { + MTLVertexAttributeDescriptorArray* a = d->_attributes; + const uint32_t bit = 1u << index; + MTLVertexAttributeDescriptor* old = a->_items[index]; + a->_items[index] = item; + MTLVertexAttributeArrayRetireItem(a, old); // -setObject: retires the replaced item + if (item) a->_present |= bit; else a->_present &= ~bit; + if (item && !item->_subclass) a->_concrete |= bit; else a->_concrete &= ~bit; + if (a->_owner) { + if (!item) { + a->_owner->_attrDirty &= ~bit; + a->_owner->_attrPinned &= ~bit; + } else { + const int tracking = MTLVertexAttributeItemIsExactClass(item) + ? MTLVertexSlotTrackingFor(item->_dirtyWord, item->_dirtyBit, &a->_owner->_attrDirty, bit) + : 2; + if (tracking == 1) a->_owner->_attrDirty |= bit; else a->_owner->_attrPinned |= bit; + } + } + } + + // the shipping reset (marked path); the fresh path is alloc() + void reset() { if (d->_layouts && d->_attributes) MTLVertexDescriptorResetMarks(d); } + // -dealloc: revoke the items that point at this descriptor's masks + void revoke() { MTLVertexBufferLayoutArrayRevokeItems(d->_layouts, &d->_layoutDirty); MTLVertexAttributeArrayRevokeItems(d->_attributes, &d->_attrDirty); } + + // -copyWithZone: the arrays are copied (new items), then the copy binds its arrays and pins + // every present slot: nothing tracks the copies + World copy() const + { + World c; + c.d = new MTLVertexDescriptor; + MTLVertexBufferLayoutDescriptorArray* layouts = new MTLVertexBufferLayoutDescriptorArray; + MTLVertexAttributeDescriptorArray* attributes = new MTLVertexAttributeDescriptorArray; + for (int i = 0; i < 31; ++i) { + if (d->_layouts->_items[i]) layouts->_items[i] = new MTLVertexBufferLayoutDescriptor(*d->_layouts->_items[i]); + if (d->_attributes->_items[i]) attributes->_items[i] = new MTLVertexAttributeDescriptor(*d->_attributes->_items[i]); + if (layouts->_items[i]) { layouts->_items[i]->_dirtyWord = nullptr; layouts->_items[i]->_dirtyBit = 0; } + if (attributes->_items[i]) { attributes->_items[i]->_dirtyWord = nullptr; attributes->_items[i]->_dirtyBit = 0; } + } + layouts->_present = d->_layouts->_present; + layouts->_concrete = d->_layouts->_concrete; + attributes->_present = d->_attributes->_present; + attributes->_concrete = d->_attributes->_concrete; + c.d->_layouts = layouts; + c.d->_attributes = attributes; + layouts->_owner = c.d; + attributes->_owner = c.d; + c.d->_layoutPinned = layouts->_present; + c.d->_attrPinned = attributes->_present; + return c; + } +}; + +// reference implementation: the pre-change reset, i.e. write the defaults into every present item +static void ReferenceReset(MTLVertexDescriptor* d) +{ + for (uint32_t bits = d->_layouts->_present; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexBufferLayoutDescriptor* item = d->_layouts->_items[i]; + if (!item) continue; + item->_stride = 0; item->_stepFunction = MTLVertexStepFunctionPerVertex; item->_stepRate = 1; + } + for (uint32_t bits = d->_attributes->_present; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexAttributeDescriptor* item = d->_attributes->_items[i]; + if (!item) continue; + item->_format = MTLVertexFormatInvalid; item->_offset = 0; item->_bufferIndex = 0; + } +} + +static bool IsDefault(const MTLVertexBufferLayoutDescriptor* item) +{ + return item->_stride == 0 && item->_stepFunction == MTLVertexStepFunctionPerVertex && item->_stepRate == 1; +} +static bool IsDefault(const MTLVertexAttributeDescriptor* item) +{ + return item->_format == MTLVertexFormatInvalid && item->_offset == 0 && item->_bufferIndex == 0; +} + +// every present slot whose item is not at the defaults must be marked (dirty or pinned); a slot +// holding a subclass instance must be pinned, because its setters cannot be seen +static bool Invariant(const World& w, std::string& why) +{ + for (uint32_t bits = w.d->_layouts->_present; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexBufferLayoutDescriptor* item = w.d->_layouts->_items[i]; + if (!item) continue; + const uint32_t bit = 1u << i; + const bool marked = (w.d->_layoutDirty | w.d->_layoutPinned) & bit; + if (item->_subclass && !(w.d->_layoutPinned & bit)) { why = "subclass layout slot not pinned"; return false; } + if (!IsDefault(item) && !marked) { why = "layout slot " + std::to_string(i) + " not marked"; return false; } + } + for (uint32_t bits = w.d->_attributes->_present; bits; bits &= bits - 1) { + const int i = __builtin_ctz(bits); + MTLVertexAttributeDescriptor* item = w.d->_attributes->_items[i]; + if (!item) continue; + const uint32_t bit = 1u << i; + const bool marked = (w.d->_attrDirty | w.d->_attrPinned) & bit; + if (item->_subclass && !(w.d->_attrPinned & bit)) { why = "subclass attribute slot not pinned"; return false; } + if (!IsDefault(item) && !marked) { why = "attribute slot " + std::to_string(i) + " not marked"; return false; } + } + return true; +} + +// the observable state: present flags and every item's values +static bool SameValues(const World& a, const World& b, std::string& why) +{ + if (a.d->_layouts->_present != b.d->_layouts->_present) { why = "layout present differs"; return false; } + if (a.d->_attributes->_present != b.d->_attributes->_present) { why = "attribute present differs"; return false; } + for (int i = 0; i < 31; ++i) { + MTLVertexBufferLayoutDescriptor* la = a.d->_layouts->_items[i]; + MTLVertexBufferLayoutDescriptor* lb = b.d->_layouts->_items[i]; + if ((la == nullptr) != (lb == nullptr)) { why = "layout presence differs at " + std::to_string(i); return false; } + if (la && lb && (la->_stride != lb->_stride || la->_stepFunction != lb->_stepFunction || la->_stepRate != lb->_stepRate)) { + why = "layout value differs at " + std::to_string(i); return false; + } + MTLVertexAttributeDescriptor* aa = a.d->_attributes->_items[i]; + MTLVertexAttributeDescriptor* ab = b.d->_attributes->_items[i]; + if ((aa == nullptr) != (ab == nullptr)) { why = "attribute presence differs at " + std::to_string(i); return false; } + if (aa && ab && (aa->_format != ab->_format || aa->_offset != ab->_offset || aa->_bufferIndex != ab->_bufferIndex)) { + why = "attribute value differs at " + std::to_string(i); return false; + } + } + return true; +} + +// --------------------------------------------------------------------------------------------- +// Driver: two worlds (shipped / reference) get the same operations; after every one the values +// must match and the shipped world's marks must cover every non-default slot. +// --------------------------------------------------------------------------------------------- +struct Op { + enum Kind { Reset, CreateLayout, CreateAttr, ValueLayout, ValueAttr, GraftLayout, GraftAttr, + GraftSubclassLayout, GraftSubclassAttr, RemoveLayout, RemoveAttr, Copy, RevokeUse } kind; + int slot = 0; + int held = 0; + NSUInteger value = 0; +}; + +struct Counters { + unsigned long ops = 0, resets = 0, written = 0, atDefaults = 0, divergences = 0; +}; + +static unsigned long MarkedSlots(const World& w) +{ + const uint32_t l = (w.d->_layoutDirty | w.d->_layoutPinned) & w.d->_layouts->_present; + const uint32_t a = (w.d->_attrDirty | w.d->_attrPinned) & w.d->_attributes->_present; + return __builtin_popcount(l) + __builtin_popcount(a); +} + +static std::vector g_history; // the last ops, for a divergence report + +static std::string Describe(const Op& op) +{ + static const char* kinds[] = {"reset", "create-layout", "create-attr", "value-layout", "value-attr", + "graft-layout", "graft-attr", "graft-subclass-layout", "graft-subclass-attr", + "remove-layout", "remove-attr", "copy", "revoke+use"}; + std::string s = kinds[(int) op.kind]; + if (op.kind == Op::ValueLayout || op.kind == Op::ValueAttr) + s += " held=" + std::to_string(op.held) + " field=" + std::to_string(op.slot) + " value=" + std::to_string(op.value); + else if (op.kind == Op::GraftLayout || op.kind == Op::GraftAttr) + s += std::string(" slot=") + std::to_string(op.slot) + " held=" + std::to_string(op.held); + else if (op.kind == Op::GraftSubclassLayout || op.kind == Op::GraftSubclassAttr) + s += std::string(" slot=") + std::to_string(op.slot) + " (caller-made item)"; + else if (op.kind == Op::CreateLayout || op.kind == Op::CreateAttr || op.kind == Op::RemoveLayout || op.kind == Op::RemoveAttr) + s += " slot=" + std::to_string(op.slot); + return s; +} + +// applies one op to both worlds and checks the invariants; returns false on a divergence +static bool Apply(std::vector& worlds, const Op& op, Counters& c, std::string& why) +{ + g_history.push_back(Describe(op)); + if (g_history.size() > 40) g_history.erase(g_history.begin()); + World& shipped = worlds[0]; + World& reference = worlds[1]; + c.ops++; + switch (op.kind) { + case Op::Reset: { + const unsigned long marked = MarkedSlots(shipped); + shipped.reset(); + ReferenceReset(reference.d); + c.resets++; + c.written += marked; + c.atDefaults += __builtin_popcount(shipped.d->_layouts->_present) + __builtin_popcount(shipped.d->_attributes->_present) - marked; + break; + } + case Op::CreateLayout: + shipped.heldLayouts.push_back(shipped.layoutAt(op.slot)); + reference.heldLayouts.push_back(reference.layoutAt(op.slot)); + break; + case Op::CreateAttr: + shipped.heldAttrs.push_back(shipped.attrAt(op.slot)); + reference.heldAttrs.push_back(reference.attrAt(op.slot)); + break; + case Op::ValueLayout: { + MTLVertexBufferLayoutDescriptor* a = shipped.heldLayouts.at(op.held); + MTLVertexBufferLayoutDescriptor* b = reference.heldLayouts.at(op.held); + switch (op.slot & 3) { + case 0: a->setStride(op.value); b->setStride(op.value); break; + case 1: a->setStepFunction(op.value); b->setStepFunction(op.value); break; + default: a->setStepRate(op.value); b->setStepRate(op.value); break; + } + break; + } + case Op::ValueAttr: { + MTLVertexAttributeDescriptor* a = shipped.heldAttrs.at(op.held); + MTLVertexAttributeDescriptor* b = reference.heldAttrs.at(op.held); + switch (op.slot & 3) { + case 0: a->setFormat(op.value); b->setFormat(op.value); break; + case 1: a->setOffset(op.value); b->setOffset(op.value); break; + default: a->setBufferIndex(op.value); b->setBufferIndex(op.value); break; + } + break; + } + case Op::GraftLayout: + shipped.setLayout(op.slot, shipped.heldLayouts.at(op.held)); + reference.setLayout(op.slot, reference.heldLayouts.at(op.held)); + break; + case Op::GraftAttr: + shipped.setAttr(op.slot, shipped.heldAttrs.at(op.held)); + reference.setAttr(op.slot, reference.heldAttrs.at(op.held)); + break; + case Op::GraftSubclassLayout: { + // a caller-made layout descriptor (not the bridge class): its setters are not ours, so + // the slot has to be pinned + shipped.heldLayouts.push_back(new MTLVertexBufferLayoutDescriptor); + shipped.heldLayouts.back()->_subclass = true; + reference.heldLayouts.push_back(new MTLVertexBufferLayoutDescriptor); + reference.heldLayouts.back()->_subclass = true; + shipped.setLayout(op.slot, shipped.heldLayouts.back()); + reference.setLayout(op.slot, reference.heldLayouts.back()); + break; + } + case Op::GraftSubclassAttr: { + shipped.heldAttrs.push_back(new MTLVertexAttributeDescriptor); + shipped.heldAttrs.back()->_subclass = true; + reference.heldAttrs.push_back(new MTLVertexAttributeDescriptor); + reference.heldAttrs.back()->_subclass = true; + shipped.setAttr(op.slot, shipped.heldAttrs.back()); + reference.setAttr(op.slot, reference.heldAttrs.back()); + break; + } + case Op::RemoveLayout: + shipped.setLayout(op.slot, nullptr); + reference.setLayout(op.slot, nullptr); + break; + case Op::RemoveAttr: + shipped.setAttr(op.slot, nullptr); + reference.setAttr(op.slot, nullptr); + break; + case Op::Copy: { + // -[MTLVertexDescriptor copyWithZone:]: the copy holds new items (nothing tracks them, + // so the descriptor pins every present slot). The model re-seats its held pointers to the + // copies' items so later operations can still mutate through a held item. The assign keeps + // the vector elements - and the references above - in place. + for (size_t w = 0; w < 2; ++w) { + World& old = worlds[w]; + World copy = old.copy(); + std::vector heldLayouts; + std::vector heldAttrs; + for (auto* item: old.heldLayouts) + for (int i = 0; i < 31; ++i) if (old.d->_layouts->_items[i] == item && copy.d->_layouts->_items[i]) { heldLayouts.push_back(copy.d->_layouts->_items[i]); break; } + for (auto* item: old.heldAttrs) + for (int i = 0; i < 31; ++i) if (old.d->_attributes->_items[i] == item && copy.d->_attributes->_items[i]) { heldAttrs.push_back(copy.d->_attributes->_items[i]); break; } + copy.heldLayouts = heldLayouts; + copy.heldAttrs = heldAttrs; + old = copy; + } + break; + } + case Op::RevokeUse: { + // a caller keeps one of the items past the descriptor's death: the revocation (and the + // retirement an earlier -setObject: did) must have cleared the item's pointer, so a + // later mutation cannot write into the dead descriptor's storage - modelled as masks + // reset to zero, which any write would set a bit of + shipped.revoke(); + reference.revoke(); + shipped.d->_attrDirty = shipped.d->_attrPinned = shipped.d->_layoutDirty = shipped.d->_layoutPinned = 0; + if (!shipped.heldLayouts.empty()) { + shipped.heldLayouts[0]->setStride(7); + reference.heldLayouts[0]->setStride(7); + } + if (!shipped.heldAttrs.empty()) { + shipped.heldAttrs[0]->setFormat(MTLVertexFormatFloat4); + reference.heldAttrs[0]->setFormat(MTLVertexFormatFloat4); + } + if (shipped.d->_attrDirty != 0 || shipped.d->_layoutDirty != 0 || shipped.d->_attrPinned != 0 || shipped.d->_layoutPinned != 0) { + why = "revoked item still marks the descriptor"; + return false; + } + break; + } + } + if (op.kind == Op::RevokeUse) { + // the descriptor is dead (and its masks are poisoned below): the values of a dead + // descriptor are not comparable to the reference, only the poison check above matters + return true; + } + std::string reason; + if (!Invariant(shipped, reason)) { + why = reason; + for (const auto& line: g_history) why += "\n " + line; + c.divergences++; + return false; + } + if (!SameValues(shipped, reference, reason)) { + why = reason; + for (const auto& line: g_history) why += "\n " + line; + c.divergences++; + return false; + } + return true; +} + +static void Report(const char* name, const Counters& c, bool ok, const std::string& why) +{ + printf("scenario %-34s ops=%6lu resets=%6lu slotsWritten=%7lu slotsAtDefaults=%7lu %s\n", + name, c.ops, c.resets, c.written, c.atDefaults, ok ? "ok" : ("DIVERGED: " + why).c_str()); +} + +static bool RunDirected(const char* name, const std::vector& ops) +{ + std::vector worlds(2); + for (World& w: worlds) w.alloc(); + Counters c; + std::string why; + bool ok = true; + g_history.clear(); + for (const Op& op: ops) { + if (!Apply(worlds, op, c, why)) { ok = false; break; } + } + Report(name, c, ok, why); + return ok; +} + +int main() +{ + unsigned failures = 0; + + // 1. the game's descriptor: a setup that fills 16 attributes and 16 layouts once, then rebuilds + // a few slots per pass. The reset must write only the slots the rebuild touched. + { + std::vector ops; + for (int i = 0; i < 16; ++i) { + ops.push_back({Op::CreateAttr, i, 0, MTLVertexFormatFloat4}); + ops.push_back({Op::ValueAttr, 2, i, (NSUInteger) i}); // bufferIndex = i + ops.push_back({Op::CreateLayout, i, 0, 0}); + ops.push_back({Op::ValueLayout, 0, i, 16}); // stride = 16 + } + for (int pass = 0; pass < 8; ++pass) { + ops.push_back({Op::Reset, 0, 0, 0}); + for (int i = 0; i < 4; ++i) { // the pass's elements + const int slot = (pass + i) % 16; + ops.push_back({Op::ValueAttr, 0, slot, 3}); + ops.push_back({Op::ValueAttr, 1, slot, (NSUInteger) (4 * i)}); + ops.push_back({Op::ValueAttr, 2, slot, (NSUInteger) (30 - i)}); + ops.push_back({Op::CreateLayout, 30 - i, 0, 0}); + ops.push_back({Op::ValueLayout, 0, 15 - i, 32}); + } + } + if (!RunDirected("game-rebuild", ops)) failures++; + } + + // 2. reset with nothing ever mutated: only the fresh items exist, so a reset writes nothing + { + std::vector ops; + for (int i = 0; i < 8; ++i) ops.push_back({Op::CreateAttr, i, 0, 0}); + for (int i = 0; i < 4; ++i) ops.push_back({Op::Reset, 0, 0, 0}); + if (!RunDirected("never-mutated", ops)) failures++; + } + + // 3. a subclass instance a caller installed keeps its own setters: the slot must stay pinned, + // and a mutation through it (which cannot mark) must still be reset + { + std::vector ops; + ops.push_back({Op::CreateAttr, 3, 0, 0}); + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::CreateAttr, 5, 0, 0}); + ops.push_back({Op::RemoveAttr, 5, 0, 0}); + ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4}); // through the held item + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::GraftAttr, 5, 0, 0}); // install the held item + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::GraftSubclassAttr, 6, 0, 0}); // a caller-made item + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::ValueAttr, 0, 1, MTLVertexFormatFloat4}); // through the subclass item + ops.push_back({Op::Reset, 0, 0, 0}); + if (!RunDirected("graft-held-and-subclass", ops)) failures++; + } + + // 4. the same item in two slots: the second slot cannot be tracked by the item, so it must be + // pinned (the reset still writes it) + { + std::vector ops; + ops.push_back({Op::CreateLayout, 1, 0, 0}); + ops.push_back({Op::ValueLayout, 0, 0, 64}); + ops.push_back({Op::GraftLayout, 2, 0, 0}); // layout[2] = the item of 1 + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::ValueLayout, 1, 0, 5}); // mutate through slot 1's item + ops.push_back({Op::Reset, 0, 0, 0}); + if (!RunDirected("item-in-two-slots", ops)) failures++; + } + + // 5. remove and re-create a slot between resets + { + std::vector ops; + ops.push_back({Op::CreateAttr, 7, 0, 0}); + ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4}); + ops.push_back({Op::RemoveAttr, 7, 0, 0}); + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::CreateAttr, 7, 0, 0}); + ops.push_back({Op::ValueAttr, 1, 1, 12}); + ops.push_back({Op::Reset, 0, 0, 0}); + if (!RunDirected("remove-recreate", ops)) failures++; + } + + // 6. copy the descriptor in the middle (the copy pins everything it holds), keep mutating both + { + std::vector ops; + for (int i = 0; i < 5; ++i) { ops.push_back({Op::CreateAttr, i, 0, 0}); ops.push_back({Op::ValueAttr, 0, i, MTLVertexFormatFloat4}); } + ops.push_back({Op::Copy, 0, 0, 0}); + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::ValueAttr, 1, 2, 8}); + ops.push_back({Op::Reset, 0, 0, 0}); + if (!RunDirected("copy-descriptor", ops)) failures++; + } + + // 7. a caller keeps an item past the descriptor's death: the revocation must clear its slot + // pointer, so the later mutation cannot write into the dead descriptor's masks + { + std::vector ops; + ops.push_back({Op::CreateLayout, 0, 0, 0}); + ops.push_back({Op::CreateAttr, 0, 0, 0}); + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::RevokeUse, 0, 0, 0}); + if (!RunDirected("revoke-then-mutate", ops)) failures++; + } + + // 7b. an item a caller removed and kept, then the descriptor dies: the removal must have + // cleared its mask pointer (the revocation only walks the slots the array still holds) + { + std::vector ops; + ops.push_back({Op::CreateAttr, 0, 0, 0}); + ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4}); + ops.push_back({Op::RemoveAttr, 0, 0, 0}); + ops.push_back({Op::Reset, 0, 0, 0}); + ops.push_back({Op::RevokeUse, 0, 0, 0}); + if (!RunDirected("remove-then-revoke", ops)) failures++; + } + + // 8. randomised: every operation class, random slots and values + { + std::mt19937 rng(20261014); + std::vector worlds(2); + for (World& w: worlds) w.alloc(); + Counters c; + std::string why; + bool ok = true; + g_history.clear(); + for (int i = 0; i < 200000 && ok; ++i) { + Op op; + const unsigned pick = rng() % 100; + if (pick < 18) op.kind = Op::Reset; + else if (pick < 30) { op.kind = (rng() & 1) ? Op::CreateLayout : Op::CreateAttr; op.slot = rng() % 12; } + else if (pick < 50 && !worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty()) { + op.kind = (rng() & 1) ? Op::ValueLayout : Op::ValueAttr; + op.held = rng() % (op.kind == Op::ValueLayout ? worlds[0].heldLayouts.size() : worlds[0].heldAttrs.size()); + op.slot = rng() % 3; op.value = rng() % 40; + } else if (pick < 72 && !worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty()) { + op.slot = rng() % 12; + const unsigned flavour = rng() % 6; + op.kind = flavour == 0 ? Op::GraftSubclassLayout + : flavour == 1 ? Op::GraftSubclassAttr + : (flavour & 1) ? Op::GraftLayout : Op::GraftAttr; + op.held = rng() % (op.kind == Op::GraftLayout ? worlds[0].heldLayouts.size() : worlds[0].heldAttrs.size()); + } else if (pick < 88) { op.kind = (rng() & 1) ? Op::RemoveLayout : Op::RemoveAttr; op.slot = rng() % 12; } + else op.kind = Op::Copy; + if (!Apply(worlds, op, c, why)) { ok = false; break; } + if (!worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty() && (rng() % 4096) == 0) { + // occasionally end a world's life (the revocation path) and start a fresh one + Op revoke {Op::RevokeUse, 0, 0, 0}; + if (!Apply(worlds, revoke, c, why)) { ok = false; break; } + for (World& w: worlds) { w = World(); w.alloc(); } + } + } + Report("randomized", c, ok, why); + if (!ok) failures++; + } + + printf("VERTEX_RESET_MODEL failures=%u\n", failures); + return failures == 0 ? 0 : 1; +} diff --git a/tools/run-deferred-handoff-tests.sh b/tools/run-deferred-handoff-tests.sh new file mode 100755 index 0000000..431832e --- /dev/null +++ b/tools/run-deferred-handoff-tests.sh @@ -0,0 +1,25 @@ +#!/bin/sh +# Build and run the host-side tests of the Indium deferred hand-off (Linux host, no guest, no GPU). +# +# tools/test-deferred-handoff.cpp the ordered worker's FIFO, the present-dispatch token, the wait counters +# tools/test-deferred-encoder-model.cpp the deferred blit/render op hand-off: order, resource lifetime, drop safety +# +# Both compile the real headers from deps/indium/private-include with the host compiler. The counter test writes its +# summary to /tmp/opencode/waits-test.log (DARLING_WAITS_LOG) and needs /tmp/metal-debug-waits for the length of the +# run; it removes the flag again if it created it (and first checks, in a forked child, that the counters stay off and +# write nothing without it). +# +# In the runtime the mechanisms exercised here are opt-in experiments (round-3 behaviour is the default): the narrow +# present-dispatch wait needs /tmp/metal-present-dispatch-wait, the deferred blit list /tmp/metal-deferred-blit, the +# counters /tmp/metal-debug-waits. The tests exercise the mechanisms themselves, independent of those flag files. +set -eu +project_dir=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) +out_dir=${RIOT_TEST_OUT:-/tmp/opencode} +include="$project_dir/build/darling-src/src/external/metal/deps/indium/private-include" +mkdir -p "$out_dir" +for name in test-deferred-handoff test-deferred-encoder-model; do + c++ -std=c++17 -O1 -pthread -I "$include" "$project_dir/tools/$name.cpp" -o "$out_dir/$name" + echo "== $name" + "$out_dir/$name" +done +echo "all host deferred-hand-off tests passed" diff --git a/tools/test-deferred-encoder-model.cpp b/tools/test-deferred-encoder-model.cpp new file mode 100644 index 0000000..d8f97dd --- /dev/null +++ b/tools/test-deferred-encoder-model.cpp @@ -0,0 +1,286 @@ +// Host test for the deferred blit/render hand-off model (Indium, Linux build of the real worker header). +// +// The deferred blit encoder (deferred-blit-encoder.private.hpp) records its calls into an op list that owns the +// resources it mentions and posts the list to the ordered worker; the real encoder is created, fed and ended there. +// What has to hold for that to be correct is not the Vulkan part (which needs the guest GPU) but the hand-off: +// +// 1. everything a command buffer defers -- render chunks (posted while the pass is recorded), a blit list (posted at +// endEncoding), the commit and the present -- is applied in command-buffer order, because one FIFO worker runs it; +// 2. the ops own their resources: a texture/buffer the game releases right after the call stays alive until the +// worker replayed the op, and is released afterwards; +// 3. a chunk posted after the commit is dropped and the deferred count is balanced, so waitDeferred() cannot hang. +// +// This is a model: it uses the real SubmitThread/Token and the real waits header, and mirrors the structure of the two +// facades (op list owning shared_ptr, deferredBegin/End around the posted replay, replayAllowed() check at post time). +// +// Build (Linux host): +// g++ -std=c++17 -O1 -pthread -I build/darling-src/src/external/metal/deps/indium/private-include \ +// tools/test-deferred-encoder-model.cpp -o /tmp/opencode/test-deferred-encoder-model +// Run: /tmp/opencode/test-deferred-encoder-model + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + std::mutex logMutex; + std::vector applied; // what the worker "recorded into the Vulkan command buffer", in order + std::atomic droppedChunks {0}; + + void append(const std::string& entry) { + std::lock_guard lock(logMutex); + applied.push_back(entry); + } + + std::vector snapshot() { + std::lock_guard lock(logMutex); + return applied; + } + + struct Gate { + std::mutex mutex; + std::condition_variable condvar; + bool open = false; + void wait() { + std::unique_lock lock(mutex); + condvar.wait(lock, [this] { return open; }); + } + void release() { + { + std::lock_guard lock(mutex); + open = true; + } + condvar.notify_all(); + } + }; + + // the parts of PrivateCommandBuffer the facades depend on + struct ModelCommandBuffer { + std::mutex mutex; + std::atomic committed {false}; + unsigned deferredPending = 0; + std::mutex deferredMutex; + std::condition_variable deferredCondvar; + Gate* replayGate = nullptr; // test hook: holds the worker inside a replay + + bool replayAllowed() const { return !committed.load(std::memory_order_acquire); } + void deferredBegin() { + std::lock_guard lock(deferredMutex); + ++deferredPending; + } + void deferredEnd() { + { + std::lock_guard lock(deferredMutex); + --deferredPending; + } + deferredCondvar.notify_all(); + } + void waitDeferred() { + std::unique_lock lock(deferredMutex); + Indium::Waits::blockOn("model.waitDeferred", deferredCondvar, lock, [this] { return deferredPending == 0; }); + } + void commit() { + committed.store(true, std::memory_order_release); + Indium::SubmitThread::post([this]() { append("commit"); }); + } + void present() { Indium::SubmitThread::post([this]() { append("present"); }); } + }; + + // stands in for a Buffer/Texture: the op list owns the last reference + struct Resource { + std::string name; + explicit Resource(std::string name_): name(std::move(name_)) {} + }; + + // mirrors DeferredBlitCommandEncoder: record, own the resources, post once at endEncoding + class ModelBlitEncoder { + std::shared_ptr _commandBuffer; + std::vector> _resources; + std::vector _ops; + bool _begun = false; + bool _ended = false; + + void flush() { + auto cb = _commandBuffer; + if (!_begun) { _begun = true; cb->deferredBegin(); } + auto ops = std::make_shared>(std::move(_ops)); + auto resources = std::make_shared>>(std::move(_resources)); + if (!cb->replayAllowed()) { + ++droppedChunks; + cb->deferredEnd(); + return; + } + Indium::SubmitThread::post([cb, ops, resources]() { + if (cb->replayGate) cb->replayGate->wait(); + for (const auto& op: *ops) append(op); + append("blit-end"); // the real encoder's endEncoding(): the end-of-encoder barrier + // the ops (and the resources they own) die here, on the worker + resources->clear(); + cb->deferredEnd(); + }); + } + + public: + explicit ModelBlitEncoder(std::shared_ptr cb): _commandBuffer(std::move(cb)) {} + ~ModelBlitEncoder() { endEncoding(); } + + void copy(const std::shared_ptr& source, const std::string& label) { + _resources.push_back(source); + _ops.push_back("blit:" + label); + } + void endEncoding() { + if (_ended) return; + _ended = true; + flush(); + } + }; + + // mirrors DeferredRenderCommandEncoder's chunking: one op per chunk here, posted while the pass is recorded + class ModelRenderEncoder { + std::shared_ptr _commandBuffer; + bool _begun = false; + bool _ended = false; + + void flush(const char* entry) { + auto cb = _commandBuffer; + if (!_begun) { _begun = true; cb->deferredBegin(); } + if (!cb->replayAllowed()) { + ++droppedChunks; + cb->deferredEnd(); + return; + } + Indium::SubmitThread::post([cb, entry]() { + if (cb->replayGate) cb->replayGate->wait(); + append(entry); + }); + } + + public: + explicit ModelRenderEncoder(std::shared_ptr cb): _commandBuffer(std::move(cb)) {} + ~ModelRenderEncoder() { endEncoding(); } + void draw() { flush("render"); } + void endEncoding() { + if (_ended) return; + _ended = true; + flush("render-end"); // the last chunk also ends the real encoder (end-of-encoder barrier) + } + }; + + void expectOrder(const std::vector& expected, const char* what) { + auto actual = snapshot(); + bool same = actual.size() == expected.size(); + for (size_t i = 0; same && i < actual.size(); ++i) same = actual[i] == expected[i]; + if (!same) { + printf("FAIL %s: expected", what); + for (const auto& entry: expected) printf(" %s", entry.c_str()); + printf("; got"); + for (const auto& entry: actual) printf(" %s", entry.c_str()); + printf("\n"); + fflush(stdout); + assert(false); + } + } + + void testOrder() { + applied.clear(); + auto cb = std::make_shared(); + { + ModelRenderEncoder render(cb); + render.draw(); // render chunk 1 + render.draw(); // render chunk 2 + ModelBlitEncoder blit(cb); + blit.copy(std::make_shared("upload"), "upload"); + blit.endEncoding(); // blit list posted at endEncoding + render.draw(); // render chunk 3 (after the blit) + render.endEncoding(); + } + cb->commit(); + cb->present(); + Indium::SubmitThread::drain("model.drain"); + std::vector expected {"render", "render", "blit:upload", "blit-end", "render", "render-end", "commit", "present"}; + expectOrder(expected, "hand-off order"); + printf("PASS hand-off order: render chunks, blit list, later render chunk, commit, present\n"); + } + + void testResourceLifetime() { + applied.clear(); + auto cb = std::make_shared(); + Gate gate; + cb->replayGate = &gate; + auto resource = std::make_shared("texture"); + { + ModelBlitEncoder blit(cb); + blit.copy(resource, "toDrawable"); + blit.endEncoding(); + } + resource.reset(); // the game drops its reference right after the call + // hold the worker inside the replay, then check that only the posted op keeps the resource alive + gate.release(); + Indium::SubmitThread::drain("model.drain"); + cb->replayGate = nullptr; + bool replayed = false; + for (const auto& entry: snapshot()) if (entry == "blit:toDrawable") replayed = true; + assert(replayed); + assert(resource.use_count() == 0); // released on the worker after the replay + printf("PASS resource lifetime: op owned the resource until its replay, released afterwards\n"); + } + + void testDropAfterCommit() { + applied.clear(); + auto cb = std::make_shared(); + cb->commit(); // the command buffer is closed + { + ModelBlitEncoder blit(cb); + blit.copy(std::make_shared("late"), "late"); + blit.endEncoding(); // must be dropped, not recorded into a closed command buffer + } + Indium::SubmitThread::drain("model.drain"); + assert(droppedChunks.load() == 1); + for (const auto& entry: snapshot()) assert(entry != "blit:late"); + cb->waitDeferred(); // balanced deferred count: returns immediately + printf("PASS drop-after-commit: chunk dropped, deferred count balanced\n"); + } + + void testWaitDeferred() { + applied.clear(); + auto cb = std::make_shared(); + Gate gate; + cb->replayGate = &gate; + { + ModelBlitEncoder blit(cb); + blit.copy(std::make_shared("held"), "held"); + blit.endEncoding(); + } + std::atomic returned {false}; + std::thread waiter([&]() { + cb->waitDeferred(); + returned.store(true); + }); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + assert(!returned.load()); // still replaying: the waiter must block + gate.release(); + waiter.join(); + assert(returned.load()); + cb->replayGate = nullptr; + cb->waitDeferred(); // and be a no-op once the replay ended + printf("PASS waitDeferred blocks until the deferred op was replayed\n"); + } +} + +int main() { + testOrder(); + testResourceLifetime(); + testDropAfterCommit(); + testWaitDeferred(); + printf("PASS all deferred-encoder model tests\n"); + return 0; +} diff --git a/tools/test-deferred-handoff.cpp b/tools/test-deferred-handoff.cpp new file mode 100644 index 0000000..9399d50 --- /dev/null +++ b/tools/test-deferred-handoff.cpp @@ -0,0 +1,214 @@ +// Host test for the ordered submit worker's hand-off primitives (Indium, Linux build of the real headers). +// +// The game's thread and the submit worker exchange work through submit-thread.private.hpp; the two things that decide +// frame pacing are (a) FIFO order of everything posted, and (b) how long a caller waits at the frame end. This test +// exercises the real header on the host (no Vulkan, no guest): +// +// 1. post() runs tasks on one worker thread, in post order; +// 2. the present-dispatch Token returns as soon as *its own* task ran, even with a later task still queued, while +// drain() waits for everything; +// 3. the wait-site counters (waits.private.hpp) count only waits that actually blocked, and produce the documented +// summary line (the test points DARLING_WAITS_LOG at a temporary file and creates /tmp/metal-debug-waits); +// 4. enabling them writes exactly one "waits enabled" line at the first check, and without the flag (a child +// process forked before the flag file is created) they stay off and write nothing, even for a blocking wait. +// +// Build (Linux host): +// g++ -std=c++17 -O1 -pthread -I build/darling-src/src/external/metal/deps/indium/private-include \ +// tools/test-deferred-handoff.cpp -o /tmp/opencode/test-deferred-handoff +// Run: /tmp/opencode/test-deferred-handoff + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + std::mutex logMutex; + std::vector log; + std::atomic sawWorkerThread {false}; + + void append(const std::string& entry) { + std::lock_guard lock(logMutex); + log.push_back(entry); + } + + bool contains(const std::string& entry) { + std::lock_guard lock(logMutex); + for (const auto& item: log) if (item == entry) return true; + return false; + } + + std::vector snapshot() { + std::lock_guard lock(logMutex); + return log; + } + + void sleepMs(int ms) { + std::this_thread::sleep_for(std::chrono::milliseconds(ms)); + } + + // a task that only runs once the test opens its gate + struct Gate { + std::mutex mutex; + std::condition_variable condvar; + bool open = false; + void wait() { + std::unique_lock lock(mutex); + condvar.wait(lock, [this] { return open; }); + } + void release() { + { + std::lock_guard lock(mutex); + open = true; + } + condvar.notify_all(); + } + }; + + void testFifo() { + constexpr int count = 200; + for (int i = 0; i < count; ++i) { + Indium::SubmitThread::post([i]() { + sawWorkerThread = Indium::SubmitThread::onWorker(); + append("t" + std::to_string(i)); + }); + } + sleepMs(100); // let the worker finish, so the drain below is the idle (uncounted) kind + Indium::SubmitThread::drain("test.fifo.idle"); + auto entries = snapshot(); + assert(entries.size() == (size_t) count); + for (int i = 0; i < count; ++i) assert(entries[i] == "t" + std::to_string(i)); + assert(sawWorkerThread.load()); + assert(!Indium::SubmitThread::onWorker()); // the test thread is the caller, not the worker + printf("PASS fifo-order (%d tasks, post order preserved)\n", count); + } + + void testNarrowToken() { + Gate gate; + Indium::SubmitThread::Token token; + // this commit's task: the "present dispatch" happens here, then the token signals + Indium::SubmitThread::post([&token]() { + sleepMs(2); + append("present"); + token.signal(); + }); + // a later task (another frame's upload) that the caller does not depend on + Indium::SubmitThread::post([&gate]() { + gate.wait(); + append("upload"); + }); + token.wait("test.token"); + assert(contains("present")); + assert(!contains("upload")); // the narrow wait must not wait for tasks queued after its own + printf("PASS present-dispatch token (returned while a later task was still queued)\n"); + gate.release(); + Indium::SubmitThread::drain("test.gated"); + assert(contains("upload")); // drain does wait for it + printf("PASS drain-waits-for-later-tasks\n"); + } + + void testIdleDrainIsNotCounted() { + Indium::SubmitThread::drain("test.idle"); + printf("PASS idle drain returns immediately\n"); + } + + std::vector waitsLines() { + std::ifstream file(getenv("DARLING_WAITS_LOG")); + std::vector lines; + std::string line; + while (std::getline(file, line)) if (line.compare(0, 6, "waits@") == 0) lines.push_back(line); + return lines; + } + + size_t enabledLines() { + std::ifstream file(getenv("DARLING_WAITS_LOG")); + size_t count = 0; + std::string line; + while (std::getline(file, line)) if (line.compare(0, 14, "waits enabled@") == 0) ++count; + return count; + } + + bool anyLineHas(const std::vector& lines, const std::string& needle) { + for (const auto& line: lines) if (line.find(needle) != std::string::npos) return true; + return false; + } + + void testCounters() { + // a blocking drain: one task sleeps, so the caller really blocks + Indium::SubmitThread::post([]() { sleepMs(20); append("slow"); }); + Indium::SubmitThread::drain("test.blocking"); + assert(contains("slow")); + + // a fresh dump window, then one idle drain (which must not be counted) and a probe record + sleepMs(5200); + Indium::SubmitThread::drain("test.idle2"); + Indium::Waits::record("test.probe", 1000); + + auto lines = waitsLines(); + assert(!lines.empty()); + assert(enabledLines() == 1); // the first check announced the counters, exactly once + assert(anyLineHas(lines, "test.blocking[n=1 ")); + assert(anyLineHas(lines, "test.probe[n=1 ")); + assert(anyLineHas(lines, "test.token[n=1 ")); + assert(anyLineHas(lines, "test.gated[n=1 ")); + // every wait that found its predicate already true: never recorded + assert(!anyLineHas(lines, "test.idle")); + printf("PASS counters: only blocking waits recorded, summary format ok\n"); + { + std::ifstream file(getenv("DARLING_WAITS_LOG")); + std::string line; + while (std::getline(file, line)) if (line.compare(0, 14, "waits enabled@") == 0) printf(" %s\n", line.c_str()); + } + for (const auto& line: lines) printf(" %s\n", line.c_str()); + } +} + +int main() { + // The counters latch their flag at the first use; point the summary at a temporary file and enable the counters. + std::string logPath = "/tmp/opencode/waits-test.log"; + unlink(logPath.c_str()); + setenv("DARLING_WAITS_LOG", logPath.c_str(), 1); + bool flagExisted = access("/tmp/metal-debug-waits", F_OK) == 0; + if (!flagExisted) { + // default (no flag file): a child that has never checked the flag must see the counters off, and a blocking + // wait must leave the log untouched (no "waits enabled" line, no summary) + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + if (Indium::Waits::enabled()) _exit(2); + Indium::SubmitThread::post([]() { sleepMs(20); }); + Indium::SubmitThread::drain("test.disabled"); + { Indium::Waits::Block block("test.disabled.block"); sleepMs(1); } + _exit(access(logPath.c_str(), F_OK) == 0 ? 3 : 0); + } + int status = 0; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + printf("PASS counters off without the flag file (nothing written)\n"); + + FILE* flag = fopen("/tmp/metal-debug-waits", "w"); + assert(flag != nullptr); + fclose(flag); + } + assert(Indium::Waits::enabled()); + + testIdleDrainIsNotCounted(); + testFifo(); + testNarrowToken(); + testCounters(); + + if (!flagExisted) unlink("/tmp/metal-debug-waits"); + printf("PASS all deferred hand-off tests\n"); + return 0; +} diff --git a/upstream/manifest.json b/upstream/manifest.json index e4ba3d2..c15be4d 100644 --- a/upstream/manifest.json +++ b/upstream/manifest.json @@ -279,6 +279,7 @@ "modified_files": 36, "new_files": [ "private-include/indium/counters.private.hpp", + "private-include/indium/deferred-blit-encoder.private.hpp", "private-include/indium/deferred-render-encoder.private.hpp", "private-include/indium/gpu-timing.private.hpp", "private-include/indium/memory-pool.private.hpp", @@ -288,7 +289,8 @@ "private-include/indium/retirement-queue.private.hpp", "private-include/indium/submission-batch.private.hpp", "private-include/indium/submit-batch.private.hpp", - "private-include/indium/submit-thread.private.hpp" + "private-include/indium/submit-thread.private.hpp", + "private-include/indium/waits.private.hpp" ] } ],