Make the Indium hand-off changes opt-in and record the Hyprland findings

Two matches with the Indium hand-off round (narrow present wait, deferred
blits, worker-side command-buffer lease, skipping CPU Vulkan devices) ran
slower than before, with more main-thread futex waiting. Each change is now
behind a guest flag file and off by default, so the default libindium matches
the round-3 build (vertex-descriptor dirty masks, cheaper resource setters).

The wait counters stay opt-in, write a "waits enabled" line when they are on
and keep their state per image. The docs explain that flag files created from
the host while Darling runs may stay invisible through the overlay mount.

Also adds a passive cursor watcher. The stuck in-game cursor is Hyprland's
direct scanout, not Darling. The first match with direct scanout off averaged
135 FPS. STATUS, TROUBLESHOOTING and README are updated with the new matches.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
lucannez64andClaude Opus 5.5 committed 2026-10-15 00:29:19 +02:00
1 parent 6559f19fcc
commit 4027cb9aa7
17 files changed
+2605 -160

No files matched your search

+2 -2
View File
@@ -2,7 +2,7 @@
Experimental source patches and tools for running the **macOS League of Legends client on x86-64 Linux through Darling**.
This is a development snapshot, not a finished launcher. Riot and League clients can launch and manually played ARAM games have run. Graphics, audio, input, startup crashes and memory reclamation have received compatibility fixes. On the Ryzen 5 5500 / RX 6600 PC, ARAM now averages about 120–127 FPS, with the slowest 10-second stretches around 103–110 (fights were 70–80 FPS before the 2026-10-13/14 work, versus 170–190 on Windows). The effect of the memory changes on client RAM remains unmeasured. A fresh checkout has not been validated end to end on another machine.
This is a development snapshot, not a finished launcher. Riot and League clients can launch and manually played ARAM games have run. Graphics, audio, input, startup crashes and memory reclamation have received compatibility fixes. On the Ryzen 5 5500 / RX 6600 PC, ARAM now averages about 118–135 FPS, with the slowest 10-second stretches around 106–119 (the higher figures with Hyprland's direct scanout off; fights were 70–80 FPS before the 2026-10-13/14 work, versus 170–190 on Windows). The effect of the memory changes on client RAM remains unmeasured. A fresh checkout has not been validated end to end on another machine.
## Included
@@ -64,7 +64,7 @@ Start with standalone regression checks, then play a match manually. See [testin
FPS counters, game presentation requests and physical display refresh are different measurements. Record average and p99 frame times, the active scene, graphics settings, charger/power state and whether the game stayed focused. See [current status](docs/STATUS.md).
On the Ryzen 5 5500 / RX 6600 test PC, ARAM on Hexgate went from 84 to 127 FPS average over 2026-10-13/14, with the slowest 10-second stretches around 110 ([status](docs/STATUS.md#game-fps-2026-10-1314)). The game-process speed switches and their opt-out files are listed in [setup](docs/SETUP.md#game-process-switches), and passive match capture is described in [testing](docs/TESTING.md#passive-match-capture).
On the Ryzen 5 5500 / RX 6600 test PC, ARAM on Hexgate went from 84 to 135 FPS average over 2026-10-13/14, with the slowest 10-second stretches around 119 in the best match (one match so far with Hyprland's direct scanout off; [status](docs/STATUS.md#game-fps-2026-10-1314)). The game-process speed switches and their opt-out files are listed in [setup](docs/SETUP.md#game-process-switches), and passive match capture is described in [testing](docs/TESTING.md#passive-match-capture).
For existing installations, [incremental update instructions](docs/SETUP.md#updating-an-existing-runtime) explain the server and memory installers, user-owned backups and the full Darling restart needed to activate replacements. Perform activation only after manually exiting the game and both clients.
+12
View File
@@ -89,6 +89,18 @@ sh build/runtime/tools/install-riot-metal-pair.sh
The installer copies `build/riot-Metal` and `build/riot-libindium.dylib` into the guest `RuntimeFrameworks/Metal.framework` and `RuntimeLibraries`. It backs up the pair it replaces in `build/metal-pair-backups`; a repeated install keeps the older backup. `sh build/runtime/tools/install-riot-metal-pair.sh --rollback` restores that backup. `tools/install-riot-gpu-runtime.sh` still builds and installs the whole GPU set, including these two. Activation needs the namespace restart below.
Indium also contains experimental hand-off changes. They are **off by default**: without the files below, libindium behaves like the build measured before them, and two matches with them on had lower FPS. Each one is a guest file, read once per game process when the game starts.
| Guest file | Effect when present |
|---|---|
| `/tmp/metal-present-dispatch-wait` | a presenting commit waits only for its own present dispatch on the submit thread, instead of the whole submit queue |
| `/tmp/metal-deferred-blit` | blit encoders are recorded on the game thread and replayed on the submit thread (only while deferred render encoding is on) |
| `/tmp/metal-worker-lease` | the Vulkan command buffer is set up by the first thread that records into it (the submit thread) instead of when `MTLCommandBuffer` is created |
| `/tmp/metal-skip-cpu-devices` | software Vulkan devices (lavapipe) are skipped while a hardware device exists |
| `/tmp/metal-debug-waits` | diagnostic: counts how often and how long threads block at the hand-off waits and in `nextDrawable`, every 5 s in `/tmp/metal-trace.log`; the first check writes a `waits enabled` line |
Create these files from inside the guest, for example `darling shell touch /tmp/metal-debug-waits`, or while Darling is shut down. The guest `/tmp` is `~/.darling/private/tmp` seen through Darling's overlay mount. A file created on the host in that directory while Darling runs may stay invisible to guest processes that already looked for it. The same applies to removing a file. The older `/tmp/metal-no-present-dispatch-wait`, `/tmp/metal-no-deferred-blit` and `/tmp/metal-no-skip-cpu-devices` files are ignored.
### Game-process switches
The kernel library decides these once when a process starts, and only for the League game process (`.../LeagueofLegends.app/Contents/MacOS/LeagueofLegends`). The Riot and League clients keep the server path. Each switch is a host file, so it takes effect at the next game start without reinstalling:
+8
View File
@@ -16,9 +16,17 @@ All matches were ARAM, played manually with unchanged game settings; FPS is the
| 10-14 14:18 | Hexgate | + core isolation (since made opt-in) | 102.9 | 92.0 |
| 10-14 14:42 | Bilgewater | local mutexes, condition variables and semaphores | 119.5 | 103.4 |
| 10-14 16:26 | Hexgate | + Metal bind deduplication and cheaper blits | 126.7 | 109.6 |
| 10-14 20:07 | Hexgate | + Indium hand-off changes (since made opt-in) | 114.8 | 102.9 |
| 10-14 20:27 | Bilgewater | same | 107.2 | 97.2 |
| 10-14 23:19 | Hexgate | hand-off removed; + vertex-descriptor dirty masks | 118.5 | 106.0 |
| 10-14 23:48 | Hexgate | same, Hyprland `render:direct_scanout = 0` | 135.4 | 118.7 |
The 13:04 and 16:26 Hexgate matches were captured with the passive sampler (`tools/riot-match-capture`). Per frame, the game's main thread waited 1.36 ms on darlingserver in the first and 0.10 ms in the second. It blocked 39 times per frame in the first and 14 in the second. darlingserver CPU fell from 82% to 61% of a core. GPU utilization stayed at 70–74%: the main thread is the limit and is now about 90% busy running code. Frame times within each 10-second window vary by about 1–1.4 ms (standard deviation) and never more than 2.7 ms, versus hitch windows of 77–168 ms in the matches of 10-13 morning.
The Indium hand-off round moved command-buffer setup, blits and the frame-end synchronisation onto Indium's submit thread. In both of its matches the main thread waited on futexes 0.77–0.79 ms per frame instead of about 0.5 ms, and FPS fell below every earlier match on the same map, so those changes are now opt-in (see [setup](SETUP.md#metal-and-indium-overlays)).
The 23:48 match ran the same Darling build as 23:19, with Hyprland's direct scanout turned off to fix a cursor that stopped changing shape in fullscreen (see [troubleshooting](TROUBLESHOOTING.md#cursor-stops-changing-shape-in-fullscreen)). Over the first ten minutes of each match, average FPS rose from 120.6 to 136.3 and the slowest quarter from 107.3 to 118.8; main-thread CPU per frame fell from 7.1–7.7 ms to 6.5 ms. That match lasted 11 minutes, so one more match is needed before crediting the whole gain to the compositor setting.
What changed in the runtime (all in the patches):
- **Process-local pthread mutexes and condition variables** (`psynch_mutexwait.c`, `bsdthread_register.c`): contended first-fit mutexes and their condition variables sleep on futexes inside the game instead of making two darlingserver round trips. On by default in the League game process only.
+8
View File
@@ -226,6 +226,14 @@ sh tools/riot-metal-bind-model/run.sh
The bind-tracking helpers are extracted verbatim from `MTLRenderCommandEncoder.mm`. The model drives two encoder states, one receiving every call and one only the forwarded calls, and requires them to be equal after every call. That covers directed scenarios, among them a freed wrapper whose address is reused by a different resource and depth-stencil retention, plus 119,401 random calls. Expected: `randomized calls=119401 forwarded=110294 dropped=9107 (7.6%)`, `BIND_DEDUP_MODEL failures=0` and `BLIT_CAST_MODEL failures=0`.
## Indium hand-off tests
```sh
sh tools/run-deferred-handoff-tests.sh
```
Both tests compile the real Indium private headers with the host compiler. `test-deferred-handoff` checks four things. The submit thread runs tasks in post order. The present-dispatch token returns while a later task is still queued, and `drain()` waits for that task. The wait counters write nothing without `/tmp/metal-debug-waits` (checked in a forked child). With the flag, they write one `waits enabled` line and then count only waits that actually blocked. `test-deferred-encoder-model` checks the deferred blit and render hand-off: order, resource lifetime and the drop after commit. The test creates the host `/tmp/metal-debug-waits` for its run and removes it afterwards. In the runtime these mechanisms are opt-in (see [setup](SETUP.md#metal-and-indium-overlays)). Expected: `all host deferred-hand-off tests passed`.
## Passive match capture
These tools observe a manually played match from the host. They wait for the game process, stop when it exits, and never attach to it, write to it, or read its memory.
+21
View File
@@ -87,3 +87,24 @@ If FPS drops back towards the 10-13 morning numbers (75–85 on Hexgate):
3. Capture a match with `tools/riot-match-capture/sample-match.py` and compare it with an earlier one using `compare-matches.py` ([testing](TESTING.md#passive-match-capture)). A server wait above about 0.5 ms per frame means one of the local paths is off. A runqueue wait above a few percent on the main thread means the CPU is oversubscribed, for example by `/tmp/rclol-core-isolation` or background load.
Compare matches on the same map: Bilgewater runs a few percent below Hexgate.
On Hyprland, also check that direct scanout is off (`hyprctl getoption render:direct_scanout` prints `int: 0`). The first match with it off ran about 16 FPS faster than the previous one on the same build; see the next section.
## Cursor stops changing shape in fullscreen
On Hyprland the game's cursor can keep its first shape (for example, it never switches to the attack cursor) until you leave and re-enter fullscreen. Darling sends every change: `tools/riot-cursor-watch/cursor-watch.c` logs the X server's displayed cursor and saw about 1,500 changes in a 12-minute match. What freezes the shape on screen is Hyprland's direct scanout of the fullscreen Xwayland window. Turn it off in `hyprland.conf`:
```
render {
direct_scanout = 0
}
```
`hyprctl keyword render:direct_scanout 0` applies it until Hyprland restarts. To check the X side yourself, build and run the watcher while playing:
```sh
cc -O2 -o build/riot-cursor-watch tools/riot-cursor-watch/cursor-watch.c -lX11 -lXfixes
build/riot-cursor-watch
```
It prints one line per cursor change. It only listens for X server notifications and does not touch the game.
+26 -13
View File
@@ -6822,12 +6822,17 @@ index d6f86b31..81a4ac10 100644
static void glDebugCallback(GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, const void* context) {
diff --git a/QuartzCore/CAMetalLayer.mm b/QuartzCore/CAMetalLayer.mm
index b81e2e14..7b9891fb 100644
index b81e2e14..63533444 100644
--- a/QuartzCore/CAMetalLayer.mm
+++ b/QuartzCore/CAMetalLayer.mm
@@ -26,6 +26,74 @@
@@ -25,7 +25,79 @@
#import <QuartzCore/CALayerContext.h>
#import <Metal/stubs.h>
+// Wait-site counters (flag file /tmp/metal-debug-waits, checked once at process start): the same summary the Indium
+// hand-off waits write, so the drawable hand-out and the command-buffer hand-offs appear in one histogram.
+#include <indium/waits.private.hpp>
+
#include <algorithm>
+#include <cstdio>
+#include <unistd.h>
@@ -6900,7 +6905,7 @@ index b81e2e14..7b9891fb 100644
#import "CALayerInternal.h"
@@ -205,6 +273,65 @@ static void reportGLErrors(void) {
@@ -205,6 +277,65 @@ static void reportGLErrors(void) {
@end
@@ -6966,7 +6971,7 @@ index b81e2e14..7b9891fb 100644
@implementation CAMetalLayerInternal
#if DARLING_METAL_ENABLED
@@ -363,6 +490,7 @@ static void reportGLErrors(void) {
@@ -363,12 +494,15 @@ static void reportGLErrors(void) {
- (id<CAMetalDrawable>)nextDrawable
{
std::shared_ptr<CAMetalDrawableActual> drawable = nullptr;
@@ -6974,7 +6979,15 @@ index b81e2e14..7b9891fb 100644
[_drawableCondition lock];
@@ -380,6 +508,8 @@ static void reportGLErrors(void) {
NSDate* endTime = [NSDate dateWithTimeIntervalSinceNow: 1];
while (_usableDrawablesBitmap == 0) {
+ // No drawable is free: this thread really blocks here (the wait is what the pool starvation costs the game).
+ Indium::Waits::Block block("nextDrawable");
if (_allowsNextDrawableTimeout) {
if (![_drawableCondition waitUntilDate: endTime]) {
break;
@@ -380,6 +514,8 @@ static void reportGLErrors(void) {
if (_usableDrawablesBitmap == 0) {
// timeout reached
@@ -6983,7 +6996,7 @@ index b81e2e14..7b9891fb 100644
[_drawableCondition unlock];
return nil;
}
@@ -400,6 +530,8 @@ static void reportGLErrors(void) {
@@ -400,6 +536,8 @@ static void reportGLErrors(void) {
}
_usableDrawablesBitmap &= ~(1u << drawableID);
@@ -6992,7 +7005,7 @@ index b81e2e14..7b9891fb 100644
drawable = _drawables[drawableID];
@@ -410,20 +542,36 @@ static void reportGLErrors(void) {
@@ -410,20 +548,36 @@ static void reportGLErrors(void) {
- (void)queuePresent: (NSUInteger)drawableID
{
@@ -7030,7 +7043,7 @@ index b81e2e14..7b9891fb 100644
[_drawableCondition lock];
_usableDrawablesBitmap |= 1u << drawableID;
[_drawableCondition signal];
@@ -455,7 +603,7 @@ static void reportGLErrors(void) {
@@ -455,7 +609,7 @@ static void reportGLErrors(void) {
}
_usableDrawablesBitmap = 0;
@@ -7039,7 +7052,7 @@ index b81e2e14..7b9891fb 100644
// can't create new drawables
[_drawableCondition unlock];
return;
@@ -506,6 +654,10 @@ static void reportGLErrors(void) {
@@ -506,6 +660,10 @@ static void reportGLErrors(void) {
glTextureParameteri(_tex, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTextureParameteri(_tex, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTextureParameteri(_tex, GL_TEXTURE_WRAP_T, GL_REPEAT);
@@ -7050,7 +7063,7 @@ index b81e2e14..7b9891fb 100644
reportGLErrors();
}
}
@@ -516,6 +668,10 @@ static void reportGLErrors(void) {
@@ -516,6 +674,10 @@ static void reportGLErrors(void) {
- (void)prepareRender
{
std::shared_ptr<CAMetalDrawableActual> drawable = nullptr;
@@ -7061,7 +7074,7 @@ index b81e2e14..7b9891fb 100644
[_drawableCondition lock];
if (_queuedDrawableCount > 0) {
@@ -528,11 +684,62 @@ static void reportGLErrors(void) {
@@ -528,11 +690,62 @@ static void reportGLErrors(void) {
if (drawable) {
// wait for the drawable to be fully rendered before copying it to the render texture
// (this is a GPU-side wait)
@@ -7124,7 +7137,7 @@ index b81e2e14..7b9891fb 100644
}
@synchronized(self) {
@@ -575,6 +782,11 @@ static void reportGLErrors(void) {
@@ -575,6 +788,11 @@ static void reportGLErrors(void) {
{
[super _setContext: context];
@@ -7136,7 +7149,7 @@ index b81e2e14..7b9891fb 100644
if (_tex == 0) {
// create our textures now using the context's CGLContext
CGLContextObj prev = CGLGetCurrentContext();
@@ -589,10 +801,15 @@ static void reportGLErrors(void) {
@@ -589,10 +807,15 @@ static void reportGLErrors(void) {
glTextureParameteri(_tex, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTextureParameteri(_tex, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTextureParameteri(_tex, GL_TEXTURE_WRAP_T, GL_REPEAT);
File diff suppressed because it is too large. Load diff
+263 -33
View File
@@ -2639,10 +2639,10 @@ index 37cccb3..79b2350 100644
-
@end
diff --git a/src/Metal/MTLVertexDescriptor.mm b/src/Metal/MTLVertexDescriptor.mm
index cde968d..a3819d0 100644
index cde968d..aa7fe05 100644
--- a/src/Metal/MTLVertexDescriptor.mm
+++ b/src/Metal/MTLVertexDescriptor.mm
@@ -1,24 +1,205 @@
@@ -1,24 +1,435 @@
// SPDX-FileCopyrightText: 2026 Darling Developers
// SPDX-License-Identifier: MPL-2.0
@@ -2656,15 +2656,25 @@ index cde968d..a3819d0 100644
-
-- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
+#import <Metal/MTLVertexDescriptorInternal.h>
+#import <Metal/MTLTrace.h>
+#import <objc/runtime.h>
+// The bridge owns these concrete descriptor implementations. Keep their
+// scalar storage accessible within this translation unit so reset/conversion
+// can batch it. Subclasses continue through their public property methods.
+// Every item is created by an array for one slot, and the descriptor resets a slot by writing
+// the defaults into the item that holds it (in place, so an item a caller kept reads them back).
+// The array therefore hands each item the word and bit of the descriptor mask that reset reads
+// (`_dirtyWord`/`_dirtyBit`, see MTLVertexDescriptor below): a setter marks its own slot, and
+// reset only has to visit the slots that may differ from the defaults instead of every item in
+// the table. An item that was not handed a slot (a copy, or an array nobody owns) marks nothing,
+// which is what the "always" mask in the descriptor is for.
+@interface MTLVertexBufferLayoutDescriptor () {
+@public
+ NSUInteger _stride;
+ MTLVertexStepFunction _stepFunction;
+ NSUInteger _stepRate;
+ uint32_t* _dirtyWord; // the descriptor's layout mask, or NULL when the item has no slot
+ uint32_t _dirtyBit; // 1u << slot, meaningful while _dirtyWord is not NULL
+}
+@end
+@interface MTLVertexAttributeDescriptor () {
@@ -2672,12 +2682,98 @@ index cde968d..a3819d0 100644
+ MTLVertexFormat _format;
+ NSUInteger _offset;
+ NSUInteger _bufferIndex;
+ uint32_t* _dirtyWord; // the descriptor's attribute mask, or NULL when the item has no slot
+ uint32_t _dirtyBit; // 1u << slot, meaningful while _dirtyWord is not NULL
+}
+@end
+// The masks that make -reset cheap, declared here so the two arrays (same file) can set them.
+// (darling-lol:) A descriptor's reset used to write the defaults into every item the arrays had
+// ever handed out - 16-31 attributes plus 16-31 layouts for the game's descriptors, each item a
+// separate allocation last touched a frame ago, i.e. a cache miss per item even though only the
+// few slots the game refilled in the previous pass could differ from the defaults. The game
+// profile put 1.5% of the main thread there (build/perf-research/metal-vdesc/REPORT.md).
+// `_attrDirty`/`_layoutDirty` name the slots whose values may differ from the defaults and must be
+// written by the next reset; `_attrPinned`/`_layoutPinned` name the slots whose item is not
+// tracked (a subclass instance, an item installed with setObject:, or a copy), which are written
+// every time. Over-marking is always safe - the reset writes the defaults either way - so the
+// only invariant that has to hold is that a slot whose value can differ from the defaults is
+// marked. Mutations mark through the item (see MTLVertexDescriptorItemTouched), which is why
+// every fallback pins the slot instead of tracking it.
+@interface MTLVertexDescriptor () {
+@public
+ MTLVertexBufferLayoutDescriptorArray* _layouts;
+ MTLVertexAttributeDescriptorArray* _attributes;
+ uint32_t _attrDirty; // attributes: slots to write at the next reset
+ uint32_t _attrPinned; // attributes: slots whose item cannot be tracked
+ uint32_t _layoutDirty; // layouts: slots to write at the next reset
+ uint32_t _layoutPinned; // layouts: slots whose item cannot be tracked
+}
+@end
+// "Is this item exactly the bridge class?" - the class that declares the plain ivars both the
+// reset and the marking write. A hook rather than the inline expression it replaces, so the host
+// model (tools/riot-metal-bind-model) can compile the helpers that call it.
+static inline bool MTLVertexBufferLayoutItemIsExactClass(const MTLVertexBufferLayoutDescriptor* item)
{
- return [NSMethodSignature signatureWithObjCTypes: "v@:"];
+ return object_getClass((id) item) == [MTLVertexBufferLayoutDescriptor class];
}
-
-- (void)forwardInvocation:(NSInvocation *)anInvocation
+static inline bool MTLVertexAttributeItemIsExactClass(const MTLVertexAttributeDescriptor* item)
{
- NSLog(@"Stub called: %@ in %@", NSStringFromSelector([anInvocation selector]), [self class]);
+ return object_getClass((id) item) == [MTLVertexAttributeDescriptor class];
+}
+// Marking an untouched item is harmless (reset would write the same default value), so the
+// marking is allowed to over-approximate; it must never miss a mutation. See the model in
+// tools/riot-metal-bind-model/ for the mutation paths this covers.
+static inline void MTLVertexDescriptorItemTouched(uint32_t* word, uint32_t bit)
+{
+ if (word) *word |= bit;
+}
+// The decision a -setObject:atIndexedSubscript: makes for one slot: 1 when the slot's item is one
+// this descriptor tracks at exactly this slot (the caller marks the dirty bit), 2 when the slot
+// has to be pinned because the item is not trackable (a copy with no slot, an item of another
+// descriptor, or the same item at another slot). Only the concrete classes declare the two fields,
+// so a subclass instance is never passed here (see the caller).
+static inline int MTLVertexSlotTrackingFor(uint32_t* itemWord, uint32_t itemBit, uint32_t* ownerWord, uint32_t bit)
+{
+ if (!ownerWord) return 2;
+ if (!itemWord) return 2;
+ if (itemWord != ownerWord) return 2;
+ return itemBit == bit ? 1 : 2;
+}
+static bool MTLVertexDescriptorResetCounters()
+{
+ static const bool enabled = MTLDebugEnabled() || access("/tmp/metal-debug-binds", F_OK) == 0;
+ return enabled;
+}
+static void MTLVertexDescriptorResetCount(unsigned long written, unsigned long skipped)
+{
+ static std::mutex lock;
+ static unsigned long calls = 0, writtenTotal = 0, skippedTotal = 0;
+ static time_t last = 0;
+ std::lock_guard<std::mutex> guard(lock);
+ ++calls; writtenTotal += written; skippedTotal += skipped;
+ time_t now = time(nullptr);
+ if (now - last >= 3) {
+ last = now;
+ if (FILE* log = fopen("/tmp/metal-trace.log", "a")) {
+ fprintf(log, "vdesc-reset@%ld: calls=%lu slotsWritten=%lu slotsAtDefaults=%lu\n",
+ (long) now, calls, writtenTotal, skippedTotal);
+ fclose(log);
+ }
+ }
+}
+@implementation MTLVertexBufferLayoutDescriptor
+@synthesize stride = _stride;
+@synthesize stepFunction = _stepFunction;
+@synthesize stepRate = _stepRate;
+// The setters are hand-written so they can mark their slot (over-marking never changes what a
+// reset writes); the getters stay synthesized.
+- (void)setStride:(NSUInteger)stride { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stride = stride; }
+- (void)setStepFunction:(MTLVertexStepFunction)stepFunction { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepFunction = stepFunction; }
+- (void)setStepRate:(NSUInteger)stepRate { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepRate = stepRate; }
+- (instancetype)init { self = [super init]; if (self) { _stepFunction = MTLVertexStepFunctionPerVertex; _stepRate = 1; } return self; }
+- (id)copyWithZone:(NSZone*)zone { MTLVertexBufferLayoutDescriptor* copy = [[[self class] allocWithZone:zone] init]; copy.stride = _stride; copy.stepFunction = _stepFunction; copy.stepRate = _stepRate; return copy; }
+@end
@@ -2690,6 +2786,7 @@ index cde968d..a3819d0 100644
+ MTLVertexBufferLayoutDescriptor* _items[31];
+ uint32_t _present; // bit i: _items[i] exists
+ uint32_t _concrete; // bit i: _items[i] is exactly MTLVertexBufferLayoutDescriptor (ivars can be written directly)
+ MTLVertexDescriptor* _owner; // the descriptor whose mask the items mark (not retained; nil after it dies)
+}
+- (MTLVertexBufferLayoutDescriptor*)existingObjectAtIndex:(NSUInteger)index;
+- (void)resetAll;
@@ -2699,6 +2796,7 @@ index cde968d..a3819d0 100644
+ MTLVertexAttributeDescriptor* _items[31];
+ uint32_t _present; // bit i: _items[i] exists
+ uint32_t _concrete; // bit i: _items[i] is exactly MTLVertexAttributeDescriptor (ivars can be written directly)
+ MTLVertexDescriptor* _owner; // the descriptor whose mask the items mark (not retained; nil after it dies)
+}
+- (MTLVertexAttributeDescriptor*)existingObjectAtIndex:(NSUInteger)index;
+- (void)resetAll;
@@ -2708,27 +2806,24 @@ index cde968d..a3819d0 100644
+// line so the common all-concrete loop does not carry the selector references and
+// the message-send stub in its preheader.
+__attribute__((noinline)) static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item)
{
- return [NSMethodSignature signatureWithObjCTypes: "v@:"];
+{
+ item.stride = 0; item.stepFunction = MTLVertexStepFunctionPerVertex; item.stepRate = 1;
}
-
-- (void)forwardInvocation:(NSInvocation *)anInvocation
+}
+__attribute__((noinline)) static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item)
{
- NSLog(@"Stub called: %@ in %@", NSStringFromSelector([anInvocation selector]), [self class]);
+{
+ item.format = MTLVertexFormatInvalid; item.offset = 0; item.bufferIndex = 0;
+}
+// Reset every entry the array has handed out, in place: an object a caller still
+// holds must read back the defaults. Exact-class entries are written through
+// their ivars; the present-but-non-concrete mask is taken once so the loop does
+// not reload it per entry.
+static void MTLVertexBufferLayoutArrayResetItems(MTLVertexBufferLayoutDescriptorArray* array)
+// Reset the slots whose mask bit is set, in ascending order like the old full-table loop: an
+// object a caller still holds must read back the defaults, and absent (nil) slots stay absent.
+// Exact-class entries are written through their ivars; an entry that is not exactly the bridge
+// class (a subclass an array was handed, or a graft) keeps its own property setters.
+static void MTLVertexBufferLayoutArrayResetSlots(MTLVertexBufferLayoutDescriptorArray* array, uint32_t slots)
+{
+ const uint32_t exotic = array->_present & ~array->_concrete;
+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) {
+ const uint32_t exotic = slots & ~array->_concrete;
+ for (uint32_t bits = slots; bits; bits &= bits - 1) {
+ const int i = __builtin_ctz(bits);
+ MTLVertexBufferLayoutDescriptor* item = array->_items[i];
+ if (!item) continue;
+ if (exotic & (1u << i)) {
+ MTLVertexBufferLayoutSetDefaults(item);
+ } else {
@@ -2736,12 +2831,13 @@ index cde968d..a3819d0 100644
+ }
+ }
+}
+static void MTLVertexAttributeArrayResetItems(MTLVertexAttributeDescriptorArray* array)
+static void MTLVertexAttributeArrayResetSlots(MTLVertexAttributeDescriptorArray* array, uint32_t slots)
+{
+ const uint32_t exotic = array->_present & ~array->_concrete;
+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) {
+ const uint32_t exotic = slots & ~array->_concrete;
+ for (uint32_t bits = slots; bits; bits &= bits - 1) {
+ const int i = __builtin_ctz(bits);
+ MTLVertexAttributeDescriptor* item = array->_items[i];
+ if (!item) continue;
+ if (exotic & (1u << i)) {
+ MTLVertexAttributeSetDefaults(item);
+ } else {
@@ -2749,32 +2845,107 @@ index cde968d..a3819d0 100644
+ }
+ }
+}
+// An item that leaves its slot (a -setObject: replaces it, or installs nil) may still be held by
+// the caller. If no slot of this array holds it any more, its mask pointer must be cleared: the
+// descriptor's -dealloc only revokes the items its arrays still hold, so a stale pointer would
+// write into the descriptor's storage after it is gone. An item that is still in a slot must keep
+// its pointer - that slot is tracked through it.
+static void MTLVertexBufferLayoutArrayRetireItem(MTLVertexBufferLayoutDescriptorArray* array, MTLVertexBufferLayoutDescriptor* item)
+{
+ if (!item || !array->_owner) return;
+ if (!MTLVertexBufferLayoutItemIsExactClass(item)) return;
+ if (item->_dirtyWord != &array->_owner->_layoutDirty) return;
+ for (int i = 0; i < 31; ++i) if (array->_items[i] == item) return;
+ item->_dirtyWord = NULL; item->_dirtyBit = 0;
+}
+static void MTLVertexAttributeArrayRetireItem(MTLVertexAttributeDescriptorArray* array, MTLVertexAttributeDescriptor* item)
+{
+ if (!item || !array->_owner) return;
+ if (!MTLVertexAttributeItemIsExactClass(item)) return;
+ if (item->_dirtyWord != &array->_owner->_attrDirty) return;
+ for (int i = 0; i < 31; ++i) if (array->_items[i] == item) return;
+ item->_dirtyWord = NULL; item->_dirtyBit = 0;
+}
+// The arrays are released with the descriptor, but a caller may hold an array (or an item) past
+// it: the array must then not reach the dead descriptor again, and an item that still points at
+// the descriptor's mask must not mark it. Both are cleared here, while the items are alive (the
+// descriptor keeps the arrays, and they keep the items).
+static void MTLVertexBufferLayoutArrayRevokeItems(MTLVertexBufferLayoutDescriptorArray* array, uint32_t* word)
+{
+ if (!array) return;
+ array->_owner = nil;
+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) {
+ const int i = __builtin_ctz(bits);
+ MTLVertexBufferLayoutDescriptor* item = array->_items[i];
+ if (item && MTLVertexBufferLayoutItemIsExactClass(item) && item->_dirtyWord == word) {
+ item->_dirtyWord = NULL; item->_dirtyBit = 0;
+ }
+ }
+}
+static void MTLVertexAttributeArrayRevokeItems(MTLVertexAttributeDescriptorArray* array, uint32_t* word)
+{
+ if (!array) return;
+ array->_owner = nil;
+ for (uint32_t bits = array->_present; bits; bits &= bits - 1) {
+ const int i = __builtin_ctz(bits);
+ MTLVertexAttributeDescriptor* item = array->_items[i];
+ if (item && MTLVertexAttributeItemIsExactClass(item) && item->_dirtyWord == word) {
+ item->_dirtyWord = NULL; item->_dirtyBit = 0;
+ }
+ }
+}
+// The arrays used to be NSMutableDictionaries keyed by boxed NSNumbers, and converting a descriptor created all 31 entries;
+// under a game that builds vertex descriptors constantly that was ~30% of the render thread. Fixed storage, entries created lazily.
+@implementation MTLVertexBufferLayoutDescriptorArray
+- (void)dealloc { for (int i = 0; i < 31; ++i) [_items[i] release]; [super dealloc]; }
+- (MTLVertexBufferLayoutDescriptor*)objectAtIndexedSubscript:(NSUInteger)index {
+ if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index];
+ if (!_items[index]) { _items[index] = [MTLVertexBufferLayoutDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; }
+ if (!_items[index]) {
+ _items[index] = [MTLVertexBufferLayoutDescriptor new]; _present |= 1u << index; _concrete |= 1u << index;
+ // a fresh item holds the defaults, so nothing is marked until it is written; the slot is
+ // what a later setter marks
+ if (_owner) { _items[index]->_dirtyWord = &_owner->_layoutDirty; _items[index]->_dirtyBit = 1u << index; }
+ }
+ return _items[index];
+}
+- (MTLVertexBufferLayoutDescriptor*)existingObjectAtIndex:(NSUInteger)index { return index < 31 ? _items[index] : nil; }
+// reset in place: the game calls -[MTLVertexDescriptor reset] for every draw and then refills it, so keep the objects
+- (void)resetAll { MTLVertexBufferLayoutArrayResetItems(self); }
+- (void)resetAll {
+ MTLVertexBufferLayoutArrayResetSlots(self, _present);
+ if (_owner) _owner->_layoutDirty &= ~_present;
+}
+- (void)setObject:(MTLVertexBufferLayoutDescriptor*)descriptor atIndexedSubscript:(NSUInteger)index {
+ if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index];
+ id old = _items[index];
+ MTLVertexBufferLayoutDescriptor* old = _items[index];
+ _items[index] = [descriptor retain];
+ [old release];
+ MTLVertexBufferLayoutArrayRetireItem(self, old);
+ const uint32_t bit = 1u << index;
+ if (descriptor) _present |= bit; else _present &= ~bit;
+ if (descriptor && object_getClass(descriptor) == [MTLVertexBufferLayoutDescriptor class]) _concrete |= bit; else _concrete &= ~bit;
+ // The next reset must write this slot unless its item is one this descriptor tracks at exactly
+ // this slot: a subclass with its own setters, an item that belongs to another descriptor or
+ // another slot, or an item with no slot at all is marked "always" instead, which keeps the
+ // reset exact (over-marking never changes what a reset writes).
+ if (_owner) {
+ if (!descriptor) {
+ _owner->_layoutDirty &= ~bit;
+ _owner->_layoutPinned &= ~bit;
+ } else {
+ const int tracking = MTLVertexBufferLayoutItemIsExactClass(descriptor)
+ ? MTLVertexSlotTrackingFor(descriptor->_dirtyWord, descriptor->_dirtyBit, &_owner->_layoutDirty, bit)
+ : 2;
+ if (tracking == 1) _owner->_layoutDirty |= bit; else _owner->_layoutPinned |= bit;
+ }
+ }
+}
+- (id)copyWithZone:(NSZone*)zone {
+ MTLVertexBufferLayoutDescriptorArray* copy = [[[self class] allocWithZone:zone] init];
+ for (int i = 0; i < 31; ++i) if (_items[i]) copy->_items[i] = [_items[i] copy];
+ copy->_present = _present;
+ copy->_concrete = _concrete; // -copy of a concrete item returns the same class
+ // the copied items are new objects and are not tracked; the descriptor that adopts this array
+ // binds them and marks their slots (see MTLVertexDescriptor -copyWithZone:)
+ return copy;
+}
+@end
@@ -2782,25 +2953,50 @@ index cde968d..a3819d0 100644
+@synthesize format = _format;
+@synthesize offset = _offset;
+@synthesize bufferIndex = _bufferIndex;
+// Hand-written for the same reason as MTLVertexBufferLayoutDescriptor's: the setter marks the
+// descriptor slot this item fills.
+- (void)setFormat:(MTLVertexFormat)format { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _format = format; }
+- (void)setOffset:(NSUInteger)offset { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _offset = offset; }
+- (void)setBufferIndex:(NSUInteger)bufferIndex { MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _bufferIndex = bufferIndex; }
+- (id)copyWithZone:(NSZone*)zone { MTLVertexAttributeDescriptor* copy = [[[self class] allocWithZone:zone] init]; copy.format = _format; copy.offset = _offset; copy.bufferIndex = _bufferIndex; return copy; }
+@end
+@implementation MTLVertexAttributeDescriptorArray
+- (void)dealloc { for (int i = 0; i < 31; ++i) [_items[i] release]; [super dealloc]; }
+- (MTLVertexAttributeDescriptor*)objectAtIndexedSubscript:(NSUInteger)index {
+ if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index];
+ if (!_items[index]) { _items[index] = [MTLVertexAttributeDescriptor new]; _present |= 1u << index; _concrete |= 1u << index; }
+ if (!_items[index]) {
+ _items[index] = [MTLVertexAttributeDescriptor new]; _present |= 1u << index; _concrete |= 1u << index;
+ // a fresh item holds the defaults; the slot is what a later setter marks
+ if (_owner) { _items[index]->_dirtyWord = &_owner->_attrDirty; _items[index]->_dirtyBit = 1u << index; }
+ }
+ return _items[index];
+}
+- (MTLVertexAttributeDescriptor*)existingObjectAtIndex:(NSUInteger)index { return index < 31 ? _items[index] : nil; }
+- (void)resetAll { MTLVertexAttributeArrayResetItems(self); }
+- (void)resetAll {
+ MTLVertexAttributeArrayResetSlots(self, _present);
+ if (_owner) _owner->_attrDirty &= ~_present;
+}
+- (void)setObject:(MTLVertexAttributeDescriptor*)descriptor atIndexedSubscript:(NSUInteger)index {
+ if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index];
+ id old = _items[index];
+ MTLVertexAttributeDescriptor* old = _items[index];
+ _items[index] = [descriptor retain];
+ [old release];
+ MTLVertexAttributeArrayRetireItem(self, old);
+ const uint32_t bit = 1u << index;
+ if (descriptor) _present |= bit; else _present &= ~bit;
+ if (descriptor && object_getClass(descriptor) == [MTLVertexAttributeDescriptor class]) _concrete |= bit; else _concrete &= ~bit;
+ // see MTLVertexBufferLayoutDescriptorArray -setObject:atIndexedSubscript: for the rule
+ if (_owner) {
+ if (!descriptor) {
+ _owner->_attrDirty &= ~bit;
+ _owner->_attrPinned &= ~bit;
+ } else {
+ const int tracking = MTLVertexAttributeItemIsExactClass(descriptor)
+ ? MTLVertexSlotTrackingFor(descriptor->_dirtyWord, descriptor->_dirtyBit, &_owner->_attrDirty, bit)
+ : 2;
+ if (tracking == 1) _owner->_attrDirty |= bit; else _owner->_attrPinned |= bit;
+ }
+ }
+}
+- (id)copyWithZone:(NSZone*)zone {
+ MTLVertexAttributeDescriptorArray* copy = [[[self class] allocWithZone:zone] init];
@@ -2810,31 +3006,65 @@ index cde968d..a3819d0 100644
+ return copy;
+}
+@end
+// The reset work itself, kept in a static so the host model can run the shipping code (see
+// tools/riot-metal-bind-model/extract-vertex-reset.py): the mask words name the slots whose values
+// may differ from the defaults, and the helpers write exactly those, in ascending slot order.
+static void MTLVertexDescriptorResetMarks(MTLVertexDescriptor* descriptor)
+{
+ const uint32_t layouts = descriptor->_layoutDirty | descriptor->_layoutPinned;
+ const uint32_t attributes = descriptor->_attrDirty | descriptor->_attrPinned;
+ descriptor->_layoutDirty = descriptor->_layoutPinned;
+ descriptor->_attrDirty = descriptor->_attrPinned;
+ MTLVertexBufferLayoutArrayResetSlots(descriptor->_layouts, layouts);
+ MTLVertexAttributeArrayResetSlots(descriptor->_attributes, attributes);
+ if (MTLVertexDescriptorResetCounters()) {
+ MTLVertexDescriptorResetCount(__builtin_popcount(layouts) + __builtin_popcount(attributes),
+ __builtin_popcount(descriptor->_layouts->_present & ~layouts) + __builtin_popcount(descriptor->_attributes->_present & ~attributes));
+ }
+}
+// Guest diagnosis, same shape as the bind counters in MTLRenderCommandEncoder.mm: with
+// /tmp/metal-debug-binds (or the trace flag) present every reset reports how many slots it had to
+// write and how many it left at the defaults, so the next match measures what the marks save.
+// The flag is tested inline at the call site, so an ordinary match pays one load and a branch.
+@implementation MTLVertexDescriptor
+@synthesize layouts = _layouts;
+@synthesize attributes = _attributes;
++ (instancetype)vertexDescriptor { return [[[self alloc] init] autorelease]; }
+- (instancetype)init { self = [super init]; if (self) [self reset]; return self; }
+- (void)dealloc { [_layouts release]; [_attributes release]; [super dealloc]; }
+- (void)dealloc {
+ // an item (or an array) a caller kept must not reach this descriptor's storage after this
+ MTLVertexBufferLayoutArrayRevokeItems(_layouts, &_layoutDirty);
+ MTLVertexAttributeArrayRevokeItems(_attributes, &_attrDirty);
+ [_layouts release]; [_attributes release]; [super dealloc];
+}
+- (void)reset {
+ if (_layouts && _attributes) {
+ // Direct call instead of [_layouts resetAll]/[_attributes resetAll]: the game
+ // resets a vertex descriptor for every draw. Both arrays are only ever created
+ // by -init/-reset/-copyWithZone: in this file and the two properties are
+ // readonly, so their storage is the concrete class. The per-entry subclass
+ // case below keeps using the property setters through their own message sends.
+ MTLVertexBufferLayoutArrayResetItems(_layouts);
+ MTLVertexAttributeArrayResetItems(_attributes);
+ // Only the slots whose values may differ from the defaults are written, in the same
+ // ascending order as before; the rest were left at the defaults by the previous reset (or
+ // hold a fresh item), so writing them would change nothing. The dirty marks are cleared as
+ // the slots are written; the pinned marks stay (their items are written every time).
+ MTLVertexDescriptorResetMarks(self);
+ return;
+ }
+ [_layouts release]; [_attributes release];
+ _layouts = [MTLVertexBufferLayoutDescriptorArray new];
+ _attributes = [MTLVertexAttributeDescriptorArray new];
+ ((MTLVertexBufferLayoutDescriptorArray*)_layouts)->_owner = self;
+ ((MTLVertexAttributeDescriptorArray*)_attributes)->_owner = self;
+ _attrDirty = 0; _attrPinned = 0; _layoutDirty = 0; _layoutPinned = 0;
+}
+- (id)copyWithZone:(NSZone*)zone {
+ MTLVertexDescriptor* copy = [[[self class] allocWithZone:zone] init];
+ [copy->_layouts release]; [copy->_attributes release];
+ copy->_layouts = [_layouts copy]; copy->_attributes = [_attributes copy];
+ // the copied items are new objects nothing tracks; bind them to the copy and let its resets
+ // write every slot they occupy (a copy is rare, and this keeps it exact without tracking it)
+ MTLVertexBufferLayoutDescriptorArray* layouts = copy->_layouts;
+ MTLVertexAttributeDescriptorArray* attributes = copy->_attributes;
+ layouts->_owner = copy;
+ attributes->_owner = copy;
+ copy->_layoutPinned = layouts->_present;
+ copy->_attrPinned = attributes->_present;
+ return copy;
+}
+#if DARLING_METAL_ENABLED
+3 -3
View File
@@ -1,7 +1,7 @@
#!/bin/sh
# Install the Metal wrapper and Indium overlays built by build-riot-metal-runtime.py / build-riot-indium-runtime.py
# as a pair (the Indium blit interface they share changed), with a backup of the files they replace.
# --rollback puts the backed-up pair back. Neither stops a process: exit the game and both clients, then
# RIOT_METAL_ARTIFACT / RIOT_INDIUM_ARTIFACT select other builds. --rollback puts the backed-up pair back. Neither stops a process: exit the game and both clients, then
# `darling shutdown` and `sh scripts/run.sh` to activate.
set -eu
project_dir=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
@@ -13,8 +13,8 @@ if [ "${1:-}" = "--rollback" ]; then
source_metal="$backups/Metal"
source_indium="$backups/libindium.dylib"
else
source_metal="$project_dir/build/riot-Metal"
source_indium="$project_dir/build/riot-libindium.dylib"
source_metal=${RIOT_METAL_ARTIFACT:-$project_dir/build/riot-Metal}
source_indium=${RIOT_INDIUM_ARTIFACT:-$project_dir/build/riot-libindium.dylib}
mkdir -p "$backups"
# a repeated install must not replace the backup with the pair it installed last time
cmp -s "$metal" "$source_metal" || cp -p "$metal" "$backups/Metal"
@@ -0,0 +1,120 @@
// Host model of the render-encoder resource arguments (Metal wrapper + Indium).
//
// The wrapper used to hand each resource to Indium by value, `setFragmentTexture(std::shared_ptr<
// Texture>, size_t)`: the caller built a temporary shared_ptr for the argument (one atomic
// increment), stored it so the callee could move it into the recorded op, and the callee's
// parameter handling zeroed that temporary again; the deferred encoder then had to load back what
// the caller had just stored (the store-to-load chain the profile shows in
// DeferredRenderCommandEncoder::setFragmentTexture). The interface now takes
// `const std::shared_ptr<Texture>&`, and the op takes exactly one reference into the record that
// keeps the resource alive until the worker replays it.
//
// What a model can prove here is the lifetime rule that makes both shapes correct - and that had
// to survive the change: the op owns a strong reference, so the resource outlives the game's own
// references and is still alive when the worker replays the op (this is what makes the round-2
// bind record's raw Indium address meaningful: a recorded address cannot be reused while the op
// holds the resource). The model runs both call shapes over a counting reference count and checks
// that property, plus that the reference is released exactly once when the op is.
//
// The cost side (what the change removed) is measured in the report from the two builds'
// disassembly, not from this model: the by-value shape compiles an increment plus a stack copy in
// the caller and a store-to-load chain in the callee, the reference shape neither.
//
// Compile with: c++ -std=c++17 -O2 -Wall -Wextra -o bind-refcount-model bind-refcount-model.cpp
#include <cstdio>
#include <string>
#include <vector>
static unsigned long g_increments = 0, g_decrements = 0, g_destructions = 0, g_replays = 0;
struct Resource {
int refs = 0;
bool alive = false;
};
static void IncRef(Resource* r) { ++g_increments; ++r->refs; r->alive = true; }
static void DecRef(Resource* r)
{
++g_decrements;
if (--r->refs == 0) { ++g_destructions; r->alive = false; }
}
// a shared_ptr<Resource> in shape: a strong reference, moved without touching the count
struct Shared {
Resource* p = nullptr;
Shared() {}
explicit Shared(Resource* r) : p(r) { if (p) IncRef(p); }
Shared(const Shared& o) : p(o.p) { if (p) IncRef(p); }
Shared(Shared&& o) noexcept : p(o.p) { o.p = nullptr; }
Shared& operator=(const Shared& o) { if (o.p) IncRef(o.p); if (p) DecRef(p); p = o.p; return *this; }
Shared& operator=(Shared&& o) noexcept { if (p) DecRef(p); p = o.p; o.p = nullptr; return *this; }
~Shared() { if (p) DecRef(p); }
};
struct Op { Shared ref; }; // one recorded call: root of the resource's life
struct DeferredEncoder {
std::vector<Op> ops;
// the shipping addRef(): `o.p = ref` takes one reference into the op
void record(const Shared& reference) { ops.emplace_back(); ops.back().ref = reference; }
// the old by-value signature: the parameter is a shared_ptr the callee moves into the op
void recordByValue(Shared parameter) { ops.emplace_back(); ops.back().ref = std::move(parameter); }
void replay() { for (auto& op: ops) if (op.ref.p && op.ref.p->alive) ++g_replays; ops.clear(); }
};
struct Wrapper { Shared resource; }; // the game's MTLTexture/MTLBuffer object
// the shipping shape (from the wrapper's setFragmentTexture:atIndex: and the deferred encoder's
// setFragmentTexture(const std::shared_ptr<Texture>&)): the wrapper passes its own reference
static void BindByReference(DeferredEncoder& encoder, const Wrapper& wrapper)
{
encoder.record(wrapper.resource);
}
// the pre-change shape: the by-value argument is a caller-side temporary of the resource (the
// builds without the change compile its store plus an increment here) and the callee moves it
// into the op, so the temporary is empty afterwards and only the *store-to-load* chain - not a
// second reference - separates the two shapes
static void BindByValue(DeferredEncoder& encoder, const Wrapper& wrapper)
{
Shared argument(wrapper.resource); // the caller's temporary: one increment
encoder.recordByValue(std::move(argument)); // the callee moves it into the op
}
static bool Run(const char* name, bool byValue)
{
g_increments = g_decrements = g_destructions = g_replays = 0;
Resource resource;
Shared gameReference(&resource); // the game's reference to the Indium object
bool aliveAtReplay = false;
unsigned long incrementsDuringBind = 0, decrementsDuringBind = 0;
{
Wrapper wrapper;
wrapper.resource = gameReference; // the wrapper holds its own reference
DeferredEncoder encoder;
const unsigned long inc0 = g_increments, dec0 = g_decrements;
if (byValue) BindByValue(encoder, wrapper); else BindByReference(encoder, wrapper);
incrementsDuringBind = g_increments - inc0;
decrementsDuringBind = g_decrements - dec0;
// the game drops every reference of its own before the worker replays the op
wrapper.resource = Shared();
gameReference = Shared();
aliveAtReplay = !encoder.ops.empty() && encoder.ops.front().ref.p && encoder.ops.front().ref.p->alive;
encoder.replay();
}
printf("bind-refcount %-10s inc=%lu dec=%lu duringBind (inc=%lu dec=%lu) resourceAliveAtReplay=%s replaySeesResource=%s releasedAfterReplay=%s\n",
name, g_increments, g_decrements, incrementsDuringBind, decrementsDuringBind,
aliveAtReplay ? "yes" : "no", g_replays == 1 ? "yes" : "no", g_destructions == 1 ? "yes" : "no");
// the rule both shapes must keep: the op's reference outlives the game's own references, the
// replay still sees the resource, and the reference is released exactly once
return aliveAtReplay && g_replays == 1 && g_destructions == 1;
}
int main()
{
const bool oldOk = Run("by-value", true);
const bool newOk = Run("by-ref", false);
printf("BIND_REFCOUNT_MODEL by-value=%s by-ref=%s checks=%s\n", oldOk ? "ok" : "FAIL", newOk ? "ok" : "FAIL",
(oldOk && newOk) ? "pass" : "FAIL");
return (oldOk && newOk) ? 0 : 1;
}
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""Extract the shipping vertex-descriptor reset/marking helpers verbatim, for the host model.
The model (vertex-reset-model.cpp) must exercise the code that ships, not a copy of it: this
pulls three regions out of src/Metal/MTLVertexDescriptor.mm - the item marking and the
slot-tracking decision, the reset/revoke helpers, and the reset-marks function - and writes them
as a C++ include fragment with the source line numbers in its header. It fails loudly if the
anchor comments move.
Run it before compiling the model (tools/riot-metal-bind-model/run.sh does).
"""
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
ROOT = HERE.parent.parent
SOURCE = ROOT / 'build/darling-src/src/external/metal/src/Metal/MTLVertexDescriptor.mm'
OUT = ROOT / 'build/riot-metal-bind-model/vertex-reset.extract.inc'
OUT.parent.mkdir(parents=True, exist_ok=True)
# (name, start anchor, end anchor); the end anchor is exclusive
CHUNKS = [
('item-marks-and-tracking', '// Marking an untouched item is harmless', 'static bool MTLVertexDescriptorResetCounters()'),
('reset-slots-and-revoke', 'static void MTLVertexBufferLayoutArrayResetSlots', '// The arrays used to be NSMutableDictionaries'),
('reset-marks', '// The reset work itself, kept in a static', '// Guest diagnosis, same shape as the bind counters'),
]
text = SOURCE.read_text().splitlines(True)
out = ['// Generated by extract-vertex-reset.py from %s. Do not edit.\n' % SOURCE.name]
for name, start_anchor, end_anchor in CHUNKS:
start = next((i for i, line in enumerate(text) if line.startswith(start_anchor)), None)
end = next((i for i, line in enumerate(text) if i > (start or 0) and line.startswith(end_anchor)), None)
if start is None or end is None or end <= start:
sys.exit('vertex-reset markers not found for %s in %s' % (name, SOURCE))
out.append('// ---- %s (MTLVertexDescriptor.mm:%d-%d, verbatim) ----\n' % (name, start + 1, end))
out.extend(text[start:end])
region = ''.join(out).rstrip() + '\n'
OUT.write_text(region)
print('wrote %s (%d lines)' % (OUT, region.count('\n')))
+8 -2
View File
@@ -1,11 +1,17 @@
#!/bin/sh
# Host models of the Metal wrapper's bind deduplication (the shipped helpers, extracted verbatim) and of the Indium
# blit cast change. Linux only; never runs a Mach-O binary.
# Host models of the Metal wrapper's bind deduplication (the shipped helpers, extracted verbatim), of the
# vertex-descriptor reset marks (ditto), of the Indium blit cast change and of the render-encoder resource
# arguments (const references + the deferred op's single reference). Linux only; never runs a Mach-O binary.
set -e
here=$(cd "$(dirname "$0")" && pwd)
out=$(cd "$here/../.." && pwd)/build/riot-metal-bind-model
python3 "$here/extract-bind-tracking.py"
python3 "$here/extract-vertex-reset.py"
c++ -std=c++17 -O2 -Wall -I"$out" -o "$out/bind-dedup-model" "$here/bind-dedup-model.cpp"
c++ -std=c++17 -O2 -Wall -o "$out/blit-cast-model" "$here/blit-cast-model.cpp"
c++ -std=c++17 -O2 -Wall -I"$out" -o "$out/vertex-reset-model" "$here/vertex-reset-model.cpp"
c++ -std=c++17 -O2 -Wall -o "$out/bind-refcount-model" "$here/bind-refcount-model.cpp"
"$out/bind-dedup-model"
"$out/blit-cast-model"
"$out/vertex-reset-model"
"$out/bind-refcount-model"
@@ -0,0 +1,671 @@
// Host model of the MTLVertexDescriptor reset marks (Metal wrapper, MTLVertexDescriptor.mm).
//
// The wrapper now writes the defaults only into the slots whose values may differ from them:
// every item carries the mask word and bit of the descriptor slot it fills, its setters mark that
// slot, and the -setObject:/copy paths pin the slots whose items cannot be tracked. Over-marking
// is the only safe direction, so the invariant this model checks after every operation is:
// * the values of the descriptor's items after reset are the defaults (like the pre-change
// "reset everything" behaviour), and
// * a slot whose item holds non-default values is marked (dirty or pinned).
// Two worlds are driven with the same operations: `shipped` runs the shipping helpers (extracted
// verbatim into vertex-reset.extract.inc) and `reference` uses the pre-change reset (every present
// item). Any missed mark makes the two worlds' values differ after a reset, which is counted as a
// divergence and printed.
//
// Compile with: c++ -std=c++17 -O2 -Wall -Wextra -o vertex-reset-model vertex-reset-model.cpp
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <random>
#include <string>
#include <vector>
// ---------------------------------------------------------------------------------------------
// The model world: the same shapes as the wrapper's classes (ivar names match the shipping code),
// plus one flag the wrapper does not have (`_subclass`: an item a caller installed that is not the
// bridge class, i.e. an object whose setters this file does not control).
// ---------------------------------------------------------------------------------------------
using NSUInteger = unsigned long;
#define nil nullptr // the shipping helpers assign nil to mean "no owner/descriptor any more"
enum { MTLVertexFormatInvalid = 0, MTLVertexStepFunctionPerVertex = 1, MTLVertexStepFunctionPerInstance = 2 };
enum { MTLVertexFormatFloat4 = 31 }; // the game's setup fills Float4 into every attribute
// defined by the extracted shipping helpers (vertex-reset.extract.inc, included below)
static inline void MTLVertexDescriptorItemTouched(uint32_t* word, uint32_t bit);
struct MTLVertexDescriptor;
struct MTLVertexBufferLayoutDescriptor {
// ivars, in the shipping order (MTLVertexDescriptor.mm, MTLVertexBufferLayoutDescriptor ())
NSUInteger _stride = 0;
NSUInteger _stepFunction = MTLVertexStepFunctionPerVertex;
NSUInteger _stepRate = 1;
uint32_t* _dirtyWord = nullptr;
uint32_t _dirtyBit = 0;
bool _subclass = false; // model only: setters do not mark (the wrapper cannot see them)
// the shipping setters: they mark the slot (MTLVertexDescriptor.mm). The model adds the
// _subclass case: an item a caller installed is not the bridge class, so its own setters are
// the ones that run and they may not mark at all - that is what the pinned mask is for.
void setStride(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stride = v; }
void setStepFunction(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepFunction = v; }
void setStepRate(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _stepRate = v; }
};
struct MTLVertexAttributeDescriptor {
NSUInteger _format = MTLVertexFormatInvalid;
NSUInteger _offset = 0;
NSUInteger _bufferIndex = 0;
uint32_t* _dirtyWord = nullptr;
uint32_t _dirtyBit = 0;
bool _subclass = false;
void setFormat(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _format = v; }
void setOffset(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _offset = v; }
void setBufferIndex(NSUInteger v) { if (!_subclass) MTLVertexDescriptorItemTouched(_dirtyWord, _dirtyBit); _bufferIndex = v; }
};
struct MTLVertexBufferLayoutDescriptorArray {
MTLVertexBufferLayoutDescriptor* _items[31] = {};
uint32_t _present = 0;
uint32_t _concrete = 0;
MTLVertexDescriptor* _owner = nullptr;
};
struct MTLVertexAttributeDescriptorArray {
MTLVertexAttributeDescriptor* _items[31] = {};
uint32_t _present = 0;
uint32_t _concrete = 0;
MTLVertexDescriptor* _owner = nullptr;
};
struct MTLVertexDescriptor {
MTLVertexBufferLayoutDescriptorArray* _layouts = nullptr;
MTLVertexAttributeDescriptorArray* _attributes = nullptr;
uint32_t _attrDirty = 0, _attrPinned = 0, _layoutDirty = 0, _layoutPinned = 0;
};
// the hooks the extracted helpers use (the shipping file defines them with object_getClass)
static inline bool MTLVertexBufferLayoutItemIsExactClass(const MTLVertexBufferLayoutDescriptor* item) { return !item->_subclass; }
static inline bool MTLVertexAttributeItemIsExactClass(const MTLVertexAttributeDescriptor* item) { return !item->_subclass; }
// the model has no trace log to write to
static bool MTLVertexDescriptorResetCounters() { return false; }
static void MTLVertexDescriptorResetCount(unsigned long, unsigned long) {}
// the item's own setters (the shipping SetDefaults helpers call the public property setters)
static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item);
static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item);
// the shipping helpers, verbatim (extract-vertex-reset.py)
#include "vertex-reset.extract.inc"
// The shipping MTLVertexBufferLayoutSetDefaults/MTLVertexAttributeSetDefaults (MTLVertexDescriptor.mm)
// write the defaults through the item's own property setters, which is what an item that is not the
// bridge class needs; the same three values here.
static void MTLVertexBufferLayoutSetDefaults(MTLVertexBufferLayoutDescriptor* item)
{
item->setStride(0); item->setStepFunction(MTLVertexStepFunctionPerVertex); item->setStepRate(1);
}
static void MTLVertexAttributeSetDefaults(MTLVertexAttributeDescriptor* item)
{
item->setFormat(MTLVertexFormatInvalid); item->setOffset(0); item->setBufferIndex(0);
}
// ---------------------------------------------------------------------------------------------
// World: a descriptor plus the items the driver holds on to (to graft them, as the game may hold
// an item across a reset and install it elsewhere), and a write counter so a scenario can report
// how many items a reset touched.
// ---------------------------------------------------------------------------------------------
struct World {
MTLVertexDescriptor* d = nullptr;
std::vector<MTLVertexBufferLayoutDescriptor*> heldLayouts;
std::vector<MTLVertexAttributeDescriptor*> heldAttrs;
void alloc() // -reset's fresh path
{
d = new MTLVertexDescriptor;
d->_layouts = new MTLVertexBufferLayoutDescriptorArray;
d->_attributes = new MTLVertexAttributeDescriptorArray;
d->_layouts->_owner = d;
d->_attributes->_owner = d;
d->_attrDirty = d->_attrPinned = d->_layoutDirty = d->_layoutPinned = 0;
}
// the model's -objectAtIndexedSubscript: (creation; a fresh item is at the defaults, so it is
// not marked, but it knows its slot)
MTLVertexBufferLayoutDescriptor* layoutAt(NSUInteger index)
{
MTLVertexBufferLayoutDescriptorArray* a = d->_layouts;
if (!a->_items[index]) {
a->_items[index] = new MTLVertexBufferLayoutDescriptor;
a->_present |= 1u << index;
a->_concrete |= 1u << index;
if (a->_owner) { a->_items[index]->_dirtyWord = &a->_owner->_layoutDirty; a->_items[index]->_dirtyBit = 1u << index; }
}
return a->_items[index];
}
MTLVertexAttributeDescriptor* attrAt(NSUInteger index)
{
MTLVertexAttributeDescriptorArray* a = d->_attributes;
if (!a->_items[index]) {
a->_items[index] = new MTLVertexAttributeDescriptor;
a->_present |= 1u << index;
a->_concrete |= 1u << index;
if (a->_owner) { a->_items[index]->_dirtyWord = &a->_owner->_attrDirty; a->_items[index]->_dirtyBit = 1u << index; }
}
return a->_items[index];
}
// the tracking half of -setObject:atIndexedSubscript: / removal with nil
void setLayout(NSUInteger index, MTLVertexBufferLayoutDescriptor* item)
{
MTLVertexBufferLayoutDescriptorArray* a = d->_layouts;
const uint32_t bit = 1u << index;
MTLVertexBufferLayoutDescriptor* old = a->_items[index];
a->_items[index] = item;
MTLVertexBufferLayoutArrayRetireItem(a, old); // -setObject: retires the replaced item
if (item) a->_present |= bit; else a->_present &= ~bit;
if (item && !item->_subclass) a->_concrete |= bit; else a->_concrete &= ~bit;
if (a->_owner) {
if (!item) {
a->_owner->_layoutDirty &= ~bit;
a->_owner->_layoutPinned &= ~bit;
} else {
const int tracking = MTLVertexBufferLayoutItemIsExactClass(item)
? MTLVertexSlotTrackingFor(item->_dirtyWord, item->_dirtyBit, &a->_owner->_layoutDirty, bit)
: 2;
if (tracking == 1) a->_owner->_layoutDirty |= bit; else a->_owner->_layoutPinned |= bit;
}
}
}
void setAttr(NSUInteger index, MTLVertexAttributeDescriptor* item)
{
MTLVertexAttributeDescriptorArray* a = d->_attributes;
const uint32_t bit = 1u << index;
MTLVertexAttributeDescriptor* old = a->_items[index];
a->_items[index] = item;
MTLVertexAttributeArrayRetireItem(a, old); // -setObject: retires the replaced item
if (item) a->_present |= bit; else a->_present &= ~bit;
if (item && !item->_subclass) a->_concrete |= bit; else a->_concrete &= ~bit;
if (a->_owner) {
if (!item) {
a->_owner->_attrDirty &= ~bit;
a->_owner->_attrPinned &= ~bit;
} else {
const int tracking = MTLVertexAttributeItemIsExactClass(item)
? MTLVertexSlotTrackingFor(item->_dirtyWord, item->_dirtyBit, &a->_owner->_attrDirty, bit)
: 2;
if (tracking == 1) a->_owner->_attrDirty |= bit; else a->_owner->_attrPinned |= bit;
}
}
}
// the shipping reset (marked path); the fresh path is alloc()
void reset() { if (d->_layouts && d->_attributes) MTLVertexDescriptorResetMarks(d); }
// -dealloc: revoke the items that point at this descriptor's masks
void revoke() { MTLVertexBufferLayoutArrayRevokeItems(d->_layouts, &d->_layoutDirty); MTLVertexAttributeArrayRevokeItems(d->_attributes, &d->_attrDirty); }
// -copyWithZone: the arrays are copied (new items), then the copy binds its arrays and pins
// every present slot: nothing tracks the copies
World copy() const
{
World c;
c.d = new MTLVertexDescriptor;
MTLVertexBufferLayoutDescriptorArray* layouts = new MTLVertexBufferLayoutDescriptorArray;
MTLVertexAttributeDescriptorArray* attributes = new MTLVertexAttributeDescriptorArray;
for (int i = 0; i < 31; ++i) {
if (d->_layouts->_items[i]) layouts->_items[i] = new MTLVertexBufferLayoutDescriptor(*d->_layouts->_items[i]);
if (d->_attributes->_items[i]) attributes->_items[i] = new MTLVertexAttributeDescriptor(*d->_attributes->_items[i]);
if (layouts->_items[i]) { layouts->_items[i]->_dirtyWord = nullptr; layouts->_items[i]->_dirtyBit = 0; }
if (attributes->_items[i]) { attributes->_items[i]->_dirtyWord = nullptr; attributes->_items[i]->_dirtyBit = 0; }
}
layouts->_present = d->_layouts->_present;
layouts->_concrete = d->_layouts->_concrete;
attributes->_present = d->_attributes->_present;
attributes->_concrete = d->_attributes->_concrete;
c.d->_layouts = layouts;
c.d->_attributes = attributes;
layouts->_owner = c.d;
attributes->_owner = c.d;
c.d->_layoutPinned = layouts->_present;
c.d->_attrPinned = attributes->_present;
return c;
}
};
// reference implementation: the pre-change reset, i.e. write the defaults into every present item
static void ReferenceReset(MTLVertexDescriptor* d)
{
for (uint32_t bits = d->_layouts->_present; bits; bits &= bits - 1) {
const int i = __builtin_ctz(bits);
MTLVertexBufferLayoutDescriptor* item = d->_layouts->_items[i];
if (!item) continue;
item->_stride = 0; item->_stepFunction = MTLVertexStepFunctionPerVertex; item->_stepRate = 1;
}
for (uint32_t bits = d->_attributes->_present; bits; bits &= bits - 1) {
const int i = __builtin_ctz(bits);
MTLVertexAttributeDescriptor* item = d->_attributes->_items[i];
if (!item) continue;
item->_format = MTLVertexFormatInvalid; item->_offset = 0; item->_bufferIndex = 0;
}
}
static bool IsDefault(const MTLVertexBufferLayoutDescriptor* item)
{
return item->_stride == 0 && item->_stepFunction == MTLVertexStepFunctionPerVertex && item->_stepRate == 1;
}
static bool IsDefault(const MTLVertexAttributeDescriptor* item)
{
return item->_format == MTLVertexFormatInvalid && item->_offset == 0 && item->_bufferIndex == 0;
}
// every present slot whose item is not at the defaults must be marked (dirty or pinned); a slot
// holding a subclass instance must be pinned, because its setters cannot be seen
static bool Invariant(const World& w, std::string& why)
{
for (uint32_t bits = w.d->_layouts->_present; bits; bits &= bits - 1) {
const int i = __builtin_ctz(bits);
MTLVertexBufferLayoutDescriptor* item = w.d->_layouts->_items[i];
if (!item) continue;
const uint32_t bit = 1u << i;
const bool marked = (w.d->_layoutDirty | w.d->_layoutPinned) & bit;
if (item->_subclass && !(w.d->_layoutPinned & bit)) { why = "subclass layout slot not pinned"; return false; }
if (!IsDefault(item) && !marked) { why = "layout slot " + std::to_string(i) + " not marked"; return false; }
}
for (uint32_t bits = w.d->_attributes->_present; bits; bits &= bits - 1) {
const int i = __builtin_ctz(bits);
MTLVertexAttributeDescriptor* item = w.d->_attributes->_items[i];
if (!item) continue;
const uint32_t bit = 1u << i;
const bool marked = (w.d->_attrDirty | w.d->_attrPinned) & bit;
if (item->_subclass && !(w.d->_attrPinned & bit)) { why = "subclass attribute slot not pinned"; return false; }
if (!IsDefault(item) && !marked) { why = "attribute slot " + std::to_string(i) + " not marked"; return false; }
}
return true;
}
// the observable state: present flags and every item's values
static bool SameValues(const World& a, const World& b, std::string& why)
{
if (a.d->_layouts->_present != b.d->_layouts->_present) { why = "layout present differs"; return false; }
if (a.d->_attributes->_present != b.d->_attributes->_present) { why = "attribute present differs"; return false; }
for (int i = 0; i < 31; ++i) {
MTLVertexBufferLayoutDescriptor* la = a.d->_layouts->_items[i];
MTLVertexBufferLayoutDescriptor* lb = b.d->_layouts->_items[i];
if ((la == nullptr) != (lb == nullptr)) { why = "layout presence differs at " + std::to_string(i); return false; }
if (la && lb && (la->_stride != lb->_stride || la->_stepFunction != lb->_stepFunction || la->_stepRate != lb->_stepRate)) {
why = "layout value differs at " + std::to_string(i); return false;
}
MTLVertexAttributeDescriptor* aa = a.d->_attributes->_items[i];
MTLVertexAttributeDescriptor* ab = b.d->_attributes->_items[i];
if ((aa == nullptr) != (ab == nullptr)) { why = "attribute presence differs at " + std::to_string(i); return false; }
if (aa && ab && (aa->_format != ab->_format || aa->_offset != ab->_offset || aa->_bufferIndex != ab->_bufferIndex)) {
why = "attribute value differs at " + std::to_string(i); return false;
}
}
return true;
}
// ---------------------------------------------------------------------------------------------
// Driver: two worlds (shipped / reference) get the same operations; after every one the values
// must match and the shipped world's marks must cover every non-default slot.
// ---------------------------------------------------------------------------------------------
struct Op {
enum Kind { Reset, CreateLayout, CreateAttr, ValueLayout, ValueAttr, GraftLayout, GraftAttr,
GraftSubclassLayout, GraftSubclassAttr, RemoveLayout, RemoveAttr, Copy, RevokeUse } kind;
int slot = 0;
int held = 0;
NSUInteger value = 0;
};
struct Counters {
unsigned long ops = 0, resets = 0, written = 0, atDefaults = 0, divergences = 0;
};
static unsigned long MarkedSlots(const World& w)
{
const uint32_t l = (w.d->_layoutDirty | w.d->_layoutPinned) & w.d->_layouts->_present;
const uint32_t a = (w.d->_attrDirty | w.d->_attrPinned) & w.d->_attributes->_present;
return __builtin_popcount(l) + __builtin_popcount(a);
}
static std::vector<std::string> g_history; // the last ops, for a divergence report
static std::string Describe(const Op& op)
{
static const char* kinds[] = {"reset", "create-layout", "create-attr", "value-layout", "value-attr",
"graft-layout", "graft-attr", "graft-subclass-layout", "graft-subclass-attr",
"remove-layout", "remove-attr", "copy", "revoke+use"};
std::string s = kinds[(int) op.kind];
if (op.kind == Op::ValueLayout || op.kind == Op::ValueAttr)
s += " held=" + std::to_string(op.held) + " field=" + std::to_string(op.slot) + " value=" + std::to_string(op.value);
else if (op.kind == Op::GraftLayout || op.kind == Op::GraftAttr)
s += std::string(" slot=") + std::to_string(op.slot) + " held=" + std::to_string(op.held);
else if (op.kind == Op::GraftSubclassLayout || op.kind == Op::GraftSubclassAttr)
s += std::string(" slot=") + std::to_string(op.slot) + " (caller-made item)";
else if (op.kind == Op::CreateLayout || op.kind == Op::CreateAttr || op.kind == Op::RemoveLayout || op.kind == Op::RemoveAttr)
s += " slot=" + std::to_string(op.slot);
return s;
}
// applies one op to both worlds and checks the invariants; returns false on a divergence
static bool Apply(std::vector<World>& worlds, const Op& op, Counters& c, std::string& why)
{
g_history.push_back(Describe(op));
if (g_history.size() > 40) g_history.erase(g_history.begin());
World& shipped = worlds[0];
World& reference = worlds[1];
c.ops++;
switch (op.kind) {
case Op::Reset: {
const unsigned long marked = MarkedSlots(shipped);
shipped.reset();
ReferenceReset(reference.d);
c.resets++;
c.written += marked;
c.atDefaults += __builtin_popcount(shipped.d->_layouts->_present) + __builtin_popcount(shipped.d->_attributes->_present) - marked;
break;
}
case Op::CreateLayout:
shipped.heldLayouts.push_back(shipped.layoutAt(op.slot));
reference.heldLayouts.push_back(reference.layoutAt(op.slot));
break;
case Op::CreateAttr:
shipped.heldAttrs.push_back(shipped.attrAt(op.slot));
reference.heldAttrs.push_back(reference.attrAt(op.slot));
break;
case Op::ValueLayout: {
MTLVertexBufferLayoutDescriptor* a = shipped.heldLayouts.at(op.held);
MTLVertexBufferLayoutDescriptor* b = reference.heldLayouts.at(op.held);
switch (op.slot & 3) {
case 0: a->setStride(op.value); b->setStride(op.value); break;
case 1: a->setStepFunction(op.value); b->setStepFunction(op.value); break;
default: a->setStepRate(op.value); b->setStepRate(op.value); break;
}
break;
}
case Op::ValueAttr: {
MTLVertexAttributeDescriptor* a = shipped.heldAttrs.at(op.held);
MTLVertexAttributeDescriptor* b = reference.heldAttrs.at(op.held);
switch (op.slot & 3) {
case 0: a->setFormat(op.value); b->setFormat(op.value); break;
case 1: a->setOffset(op.value); b->setOffset(op.value); break;
default: a->setBufferIndex(op.value); b->setBufferIndex(op.value); break;
}
break;
}
case Op::GraftLayout:
shipped.setLayout(op.slot, shipped.heldLayouts.at(op.held));
reference.setLayout(op.slot, reference.heldLayouts.at(op.held));
break;
case Op::GraftAttr:
shipped.setAttr(op.slot, shipped.heldAttrs.at(op.held));
reference.setAttr(op.slot, reference.heldAttrs.at(op.held));
break;
case Op::GraftSubclassLayout: {
// a caller-made layout descriptor (not the bridge class): its setters are not ours, so
// the slot has to be pinned
shipped.heldLayouts.push_back(new MTLVertexBufferLayoutDescriptor);
shipped.heldLayouts.back()->_subclass = true;
reference.heldLayouts.push_back(new MTLVertexBufferLayoutDescriptor);
reference.heldLayouts.back()->_subclass = true;
shipped.setLayout(op.slot, shipped.heldLayouts.back());
reference.setLayout(op.slot, reference.heldLayouts.back());
break;
}
case Op::GraftSubclassAttr: {
shipped.heldAttrs.push_back(new MTLVertexAttributeDescriptor);
shipped.heldAttrs.back()->_subclass = true;
reference.heldAttrs.push_back(new MTLVertexAttributeDescriptor);
reference.heldAttrs.back()->_subclass = true;
shipped.setAttr(op.slot, shipped.heldAttrs.back());
reference.setAttr(op.slot, reference.heldAttrs.back());
break;
}
case Op::RemoveLayout:
shipped.setLayout(op.slot, nullptr);
reference.setLayout(op.slot, nullptr);
break;
case Op::RemoveAttr:
shipped.setAttr(op.slot, nullptr);
reference.setAttr(op.slot, nullptr);
break;
case Op::Copy: {
// -[MTLVertexDescriptor copyWithZone:]: the copy holds new items (nothing tracks them,
// so the descriptor pins every present slot). The model re-seats its held pointers to the
// copies' items so later operations can still mutate through a held item. The assign keeps
// the vector elements - and the references above - in place.
for (size_t w = 0; w < 2; ++w) {
World& old = worlds[w];
World copy = old.copy();
std::vector<MTLVertexBufferLayoutDescriptor*> heldLayouts;
std::vector<MTLVertexAttributeDescriptor*> heldAttrs;
for (auto* item: old.heldLayouts)
for (int i = 0; i < 31; ++i) if (old.d->_layouts->_items[i] == item && copy.d->_layouts->_items[i]) { heldLayouts.push_back(copy.d->_layouts->_items[i]); break; }
for (auto* item: old.heldAttrs)
for (int i = 0; i < 31; ++i) if (old.d->_attributes->_items[i] == item && copy.d->_attributes->_items[i]) { heldAttrs.push_back(copy.d->_attributes->_items[i]); break; }
copy.heldLayouts = heldLayouts;
copy.heldAttrs = heldAttrs;
old = copy;
}
break;
}
case Op::RevokeUse: {
// a caller keeps one of the items past the descriptor's death: the revocation (and the
// retirement an earlier -setObject: did) must have cleared the item's pointer, so a
// later mutation cannot write into the dead descriptor's storage - modelled as masks
// reset to zero, which any write would set a bit of
shipped.revoke();
reference.revoke();
shipped.d->_attrDirty = shipped.d->_attrPinned = shipped.d->_layoutDirty = shipped.d->_layoutPinned = 0;
if (!shipped.heldLayouts.empty()) {
shipped.heldLayouts[0]->setStride(7);
reference.heldLayouts[0]->setStride(7);
}
if (!shipped.heldAttrs.empty()) {
shipped.heldAttrs[0]->setFormat(MTLVertexFormatFloat4);
reference.heldAttrs[0]->setFormat(MTLVertexFormatFloat4);
}
if (shipped.d->_attrDirty != 0 || shipped.d->_layoutDirty != 0 || shipped.d->_attrPinned != 0 || shipped.d->_layoutPinned != 0) {
why = "revoked item still marks the descriptor";
return false;
}
break;
}
}
if (op.kind == Op::RevokeUse) {
// the descriptor is dead (and its masks are poisoned below): the values of a dead
// descriptor are not comparable to the reference, only the poison check above matters
return true;
}
std::string reason;
if (!Invariant(shipped, reason)) {
why = reason;
for (const auto& line: g_history) why += "\n " + line;
c.divergences++;
return false;
}
if (!SameValues(shipped, reference, reason)) {
why = reason;
for (const auto& line: g_history) why += "\n " + line;
c.divergences++;
return false;
}
return true;
}
static void Report(const char* name, const Counters& c, bool ok, const std::string& why)
{
printf("scenario %-34s ops=%6lu resets=%6lu slotsWritten=%7lu slotsAtDefaults=%7lu %s\n",
name, c.ops, c.resets, c.written, c.atDefaults, ok ? "ok" : ("DIVERGED: " + why).c_str());
}
static bool RunDirected(const char* name, const std::vector<Op>& ops)
{
std::vector<World> worlds(2);
for (World& w: worlds) w.alloc();
Counters c;
std::string why;
bool ok = true;
g_history.clear();
for (const Op& op: ops) {
if (!Apply(worlds, op, c, why)) { ok = false; break; }
}
Report(name, c, ok, why);
return ok;
}
int main()
{
unsigned failures = 0;
// 1. the game's descriptor: a setup that fills 16 attributes and 16 layouts once, then rebuilds
// a few slots per pass. The reset must write only the slots the rebuild touched.
{
std::vector<Op> ops;
for (int i = 0; i < 16; ++i) {
ops.push_back({Op::CreateAttr, i, 0, MTLVertexFormatFloat4});
ops.push_back({Op::ValueAttr, 2, i, (NSUInteger) i}); // bufferIndex = i
ops.push_back({Op::CreateLayout, i, 0, 0});
ops.push_back({Op::ValueLayout, 0, i, 16}); // stride = 16
}
for (int pass = 0; pass < 8; ++pass) {
ops.push_back({Op::Reset, 0, 0, 0});
for (int i = 0; i < 4; ++i) { // the pass's elements
const int slot = (pass + i) % 16;
ops.push_back({Op::ValueAttr, 0, slot, 3});
ops.push_back({Op::ValueAttr, 1, slot, (NSUInteger) (4 * i)});
ops.push_back({Op::ValueAttr, 2, slot, (NSUInteger) (30 - i)});
ops.push_back({Op::CreateLayout, 30 - i, 0, 0});
ops.push_back({Op::ValueLayout, 0, 15 - i, 32});
}
}
if (!RunDirected("game-rebuild", ops)) failures++;
}
// 2. reset with nothing ever mutated: only the fresh items exist, so a reset writes nothing
{
std::vector<Op> ops;
for (int i = 0; i < 8; ++i) ops.push_back({Op::CreateAttr, i, 0, 0});
for (int i = 0; i < 4; ++i) ops.push_back({Op::Reset, 0, 0, 0});
if (!RunDirected("never-mutated", ops)) failures++;
}
// 3. a subclass instance a caller installed keeps its own setters: the slot must stay pinned,
// and a mutation through it (which cannot mark) must still be reset
{
std::vector<Op> ops;
ops.push_back({Op::CreateAttr, 3, 0, 0});
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::CreateAttr, 5, 0, 0});
ops.push_back({Op::RemoveAttr, 5, 0, 0});
ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4}); // through the held item
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::GraftAttr, 5, 0, 0}); // install the held item
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::GraftSubclassAttr, 6, 0, 0}); // a caller-made item
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::ValueAttr, 0, 1, MTLVertexFormatFloat4}); // through the subclass item
ops.push_back({Op::Reset, 0, 0, 0});
if (!RunDirected("graft-held-and-subclass", ops)) failures++;
}
// 4. the same item in two slots: the second slot cannot be tracked by the item, so it must be
// pinned (the reset still writes it)
{
std::vector<Op> ops;
ops.push_back({Op::CreateLayout, 1, 0, 0});
ops.push_back({Op::ValueLayout, 0, 0, 64});
ops.push_back({Op::GraftLayout, 2, 0, 0}); // layout[2] = the item of 1
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::ValueLayout, 1, 0, 5}); // mutate through slot 1's item
ops.push_back({Op::Reset, 0, 0, 0});
if (!RunDirected("item-in-two-slots", ops)) failures++;
}
// 5. remove and re-create a slot between resets
{
std::vector<Op> ops;
ops.push_back({Op::CreateAttr, 7, 0, 0});
ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4});
ops.push_back({Op::RemoveAttr, 7, 0, 0});
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::CreateAttr, 7, 0, 0});
ops.push_back({Op::ValueAttr, 1, 1, 12});
ops.push_back({Op::Reset, 0, 0, 0});
if (!RunDirected("remove-recreate", ops)) failures++;
}
// 6. copy the descriptor in the middle (the copy pins everything it holds), keep mutating both
{
std::vector<Op> ops;
for (int i = 0; i < 5; ++i) { ops.push_back({Op::CreateAttr, i, 0, 0}); ops.push_back({Op::ValueAttr, 0, i, MTLVertexFormatFloat4}); }
ops.push_back({Op::Copy, 0, 0, 0});
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::ValueAttr, 1, 2, 8});
ops.push_back({Op::Reset, 0, 0, 0});
if (!RunDirected("copy-descriptor", ops)) failures++;
}
// 7. a caller keeps an item past the descriptor's death: the revocation must clear its slot
// pointer, so the later mutation cannot write into the dead descriptor's masks
{
std::vector<Op> ops;
ops.push_back({Op::CreateLayout, 0, 0, 0});
ops.push_back({Op::CreateAttr, 0, 0, 0});
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::RevokeUse, 0, 0, 0});
if (!RunDirected("revoke-then-mutate", ops)) failures++;
}
// 7b. an item a caller removed and kept, then the descriptor dies: the removal must have
// cleared its mask pointer (the revocation only walks the slots the array still holds)
{
std::vector<Op> ops;
ops.push_back({Op::CreateAttr, 0, 0, 0});
ops.push_back({Op::ValueAttr, 0, 0, MTLVertexFormatFloat4});
ops.push_back({Op::RemoveAttr, 0, 0, 0});
ops.push_back({Op::Reset, 0, 0, 0});
ops.push_back({Op::RevokeUse, 0, 0, 0});
if (!RunDirected("remove-then-revoke", ops)) failures++;
}
// 8. randomised: every operation class, random slots and values
{
std::mt19937 rng(20261014);
std::vector<World> worlds(2);
for (World& w: worlds) w.alloc();
Counters c;
std::string why;
bool ok = true;
g_history.clear();
for (int i = 0; i < 200000 && ok; ++i) {
Op op;
const unsigned pick = rng() % 100;
if (pick < 18) op.kind = Op::Reset;
else if (pick < 30) { op.kind = (rng() & 1) ? Op::CreateLayout : Op::CreateAttr; op.slot = rng() % 12; }
else if (pick < 50 && !worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty()) {
op.kind = (rng() & 1) ? Op::ValueLayout : Op::ValueAttr;
op.held = rng() % (op.kind == Op::ValueLayout ? worlds[0].heldLayouts.size() : worlds[0].heldAttrs.size());
op.slot = rng() % 3; op.value = rng() % 40;
} else if (pick < 72 && !worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty()) {
op.slot = rng() % 12;
const unsigned flavour = rng() % 6;
op.kind = flavour == 0 ? Op::GraftSubclassLayout
: flavour == 1 ? Op::GraftSubclassAttr
: (flavour & 1) ? Op::GraftLayout : Op::GraftAttr;
op.held = rng() % (op.kind == Op::GraftLayout ? worlds[0].heldLayouts.size() : worlds[0].heldAttrs.size());
} else if (pick < 88) { op.kind = (rng() & 1) ? Op::RemoveLayout : Op::RemoveAttr; op.slot = rng() % 12; }
else op.kind = Op::Copy;
if (!Apply(worlds, op, c, why)) { ok = false; break; }
if (!worlds[0].heldLayouts.empty() && !worlds[0].heldAttrs.empty() && (rng() % 4096) == 0) {
// occasionally end a world's life (the revocation path) and start a fresh one
Op revoke {Op::RevokeUse, 0, 0, 0};
if (!Apply(worlds, revoke, c, why)) { ok = false; break; }
for (World& w: worlds) { w = World(); w.alloc(); }
}
}
Report("randomized", c, ok, why);
if (!ok) failures++;
}
printf("VERTEX_RESET_MODEL failures=%u\n", failures);
return failures == 0 ? 0 : 1;
}
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
# Build and run the host-side tests of the Indium deferred hand-off (Linux host, no guest, no GPU).
#
# tools/test-deferred-handoff.cpp the ordered worker's FIFO, the present-dispatch token, the wait counters
# tools/test-deferred-encoder-model.cpp the deferred blit/render op hand-off: order, resource lifetime, drop safety
#
# Both compile the real headers from deps/indium/private-include with the host compiler. The counter test writes its
# summary to /tmp/opencode/waits-test.log (DARLING_WAITS_LOG) and needs /tmp/metal-debug-waits for the length of the
# run; it removes the flag again if it created it (and first checks, in a forked child, that the counters stay off and
# write nothing without it).
#
# In the runtime the mechanisms exercised here are opt-in experiments (round-3 behaviour is the default): the narrow
# present-dispatch wait needs /tmp/metal-present-dispatch-wait, the deferred blit list /tmp/metal-deferred-blit, the
# counters /tmp/metal-debug-waits. The tests exercise the mechanisms themselves, independent of those flag files.
set -eu
project_dir=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
out_dir=${RIOT_TEST_OUT:-/tmp/opencode}
include="$project_dir/build/darling-src/src/external/metal/deps/indium/private-include"
mkdir -p "$out_dir"
for name in test-deferred-handoff test-deferred-encoder-model; do
c++ -std=c++17 -O1 -pthread -I "$include" "$project_dir/tools/$name.cpp" -o "$out_dir/$name"
echo "== $name"
"$out_dir/$name"
done
echo "all host deferred-hand-off tests passed"
+286
View File
@@ -0,0 +1,286 @@
// Host test for the deferred blit/render hand-off model (Indium, Linux build of the real worker header).
//
// The deferred blit encoder (deferred-blit-encoder.private.hpp) records its calls into an op list that owns the
// resources it mentions and posts the list to the ordered worker; the real encoder is created, fed and ended there.
// What has to hold for that to be correct is not the Vulkan part (which needs the guest GPU) but the hand-off:
//
// 1. everything a command buffer defers -- render chunks (posted while the pass is recorded), a blit list (posted at
// endEncoding), the commit and the present -- is applied in command-buffer order, because one FIFO worker runs it;
// 2. the ops own their resources: a texture/buffer the game releases right after the call stays alive until the
// worker replayed the op, and is released afterwards;
// 3. a chunk posted after the commit is dropped and the deferred count is balanced, so waitDeferred() cannot hang.
//
// This is a model: it uses the real SubmitThread/Token and the real waits header, and mirrors the structure of the two
// facades (op list owning shared_ptr, deferredBegin/End around the posted replay, replayAllowed() check at post time).
//
// Build (Linux host):
// g++ -std=c++17 -O1 -pthread -I build/darling-src/src/external/metal/deps/indium/private-include \
// tools/test-deferred-encoder-model.cpp -o /tmp/opencode/test-deferred-encoder-model
// Run: /tmp/opencode/test-deferred-encoder-model
#include <indium/submit-thread.private.hpp>
#include <atomic>
#include <cassert>
#include <chrono>
#include <condition_variable>
#include <cstdio>
#include <memory>
#include <mutex>
#include <string>
#include <thread>
#include <vector>
namespace {
std::mutex logMutex;
std::vector<std::string> applied; // what the worker "recorded into the Vulkan command buffer", in order
std::atomic<int> droppedChunks {0};
void append(const std::string& entry) {
std::lock_guard<std::mutex> lock(logMutex);
applied.push_back(entry);
}
std::vector<std::string> snapshot() {
std::lock_guard<std::mutex> lock(logMutex);
return applied;
}
struct Gate {
std::mutex mutex;
std::condition_variable condvar;
bool open = false;
void wait() {
std::unique_lock<std::mutex> lock(mutex);
condvar.wait(lock, [this] { return open; });
}
void release() {
{
std::lock_guard<std::mutex> lock(mutex);
open = true;
}
condvar.notify_all();
}
};
// the parts of PrivateCommandBuffer the facades depend on
struct ModelCommandBuffer {
std::mutex mutex;
std::atomic<bool> committed {false};
unsigned deferredPending = 0;
std::mutex deferredMutex;
std::condition_variable deferredCondvar;
Gate* replayGate = nullptr; // test hook: holds the worker inside a replay
bool replayAllowed() const { return !committed.load(std::memory_order_acquire); }
void deferredBegin() {
std::lock_guard<std::mutex> lock(deferredMutex);
++deferredPending;
}
void deferredEnd() {
{
std::lock_guard<std::mutex> lock(deferredMutex);
--deferredPending;
}
deferredCondvar.notify_all();
}
void waitDeferred() {
std::unique_lock<std::mutex> lock(deferredMutex);
Indium::Waits::blockOn("model.waitDeferred", deferredCondvar, lock, [this] { return deferredPending == 0; });
}
void commit() {
committed.store(true, std::memory_order_release);
Indium::SubmitThread::post([this]() { append("commit"); });
}
void present() { Indium::SubmitThread::post([this]() { append("present"); }); }
};
// stands in for a Buffer/Texture: the op list owns the last reference
struct Resource {
std::string name;
explicit Resource(std::string name_): name(std::move(name_)) {}
};
// mirrors DeferredBlitCommandEncoder: record, own the resources, post once at endEncoding
class ModelBlitEncoder {
std::shared_ptr<ModelCommandBuffer> _commandBuffer;
std::vector<std::shared_ptr<Resource>> _resources;
std::vector<std::string> _ops;
bool _begun = false;
bool _ended = false;
void flush() {
auto cb = _commandBuffer;
if (!_begun) { _begun = true; cb->deferredBegin(); }
auto ops = std::make_shared<std::vector<std::string>>(std::move(_ops));
auto resources = std::make_shared<std::vector<std::shared_ptr<Resource>>>(std::move(_resources));
if (!cb->replayAllowed()) {
++droppedChunks;
cb->deferredEnd();
return;
}
Indium::SubmitThread::post([cb, ops, resources]() {
if (cb->replayGate) cb->replayGate->wait();
for (const auto& op: *ops) append(op);
append("blit-end"); // the real encoder's endEncoding(): the end-of-encoder barrier
// the ops (and the resources they own) die here, on the worker
resources->clear();
cb->deferredEnd();
});
}
public:
explicit ModelBlitEncoder(std::shared_ptr<ModelCommandBuffer> cb): _commandBuffer(std::move(cb)) {}
~ModelBlitEncoder() { endEncoding(); }
void copy(const std::shared_ptr<Resource>& source, const std::string& label) {
_resources.push_back(source);
_ops.push_back("blit:" + label);
}
void endEncoding() {
if (_ended) return;
_ended = true;
flush();
}
};
// mirrors DeferredRenderCommandEncoder's chunking: one op per chunk here, posted while the pass is recorded
class ModelRenderEncoder {
std::shared_ptr<ModelCommandBuffer> _commandBuffer;
bool _begun = false;
bool _ended = false;
void flush(const char* entry) {
auto cb = _commandBuffer;
if (!_begun) { _begun = true; cb->deferredBegin(); }
if (!cb->replayAllowed()) {
++droppedChunks;
cb->deferredEnd();
return;
}
Indium::SubmitThread::post([cb, entry]() {
if (cb->replayGate) cb->replayGate->wait();
append(entry);
});
}
public:
explicit ModelRenderEncoder(std::shared_ptr<ModelCommandBuffer> cb): _commandBuffer(std::move(cb)) {}
~ModelRenderEncoder() { endEncoding(); }
void draw() { flush("render"); }
void endEncoding() {
if (_ended) return;
_ended = true;
flush("render-end"); // the last chunk also ends the real encoder (end-of-encoder barrier)
}
};
void expectOrder(const std::vector<std::string>& expected, const char* what) {
auto actual = snapshot();
bool same = actual.size() == expected.size();
for (size_t i = 0; same && i < actual.size(); ++i) same = actual[i] == expected[i];
if (!same) {
printf("FAIL %s: expected", what);
for (const auto& entry: expected) printf(" %s", entry.c_str());
printf("; got");
for (const auto& entry: actual) printf(" %s", entry.c_str());
printf("\n");
fflush(stdout);
assert(false);
}
}
void testOrder() {
applied.clear();
auto cb = std::make_shared<ModelCommandBuffer>();
{
ModelRenderEncoder render(cb);
render.draw(); // render chunk 1
render.draw(); // render chunk 2
ModelBlitEncoder blit(cb);
blit.copy(std::make_shared<Resource>("upload"), "upload");
blit.endEncoding(); // blit list posted at endEncoding
render.draw(); // render chunk 3 (after the blit)
render.endEncoding();
}
cb->commit();
cb->present();
Indium::SubmitThread::drain("model.drain");
std::vector<std::string> expected {"render", "render", "blit:upload", "blit-end", "render", "render-end", "commit", "present"};
expectOrder(expected, "hand-off order");
printf("PASS hand-off order: render chunks, blit list, later render chunk, commit, present\n");
}
void testResourceLifetime() {
applied.clear();
auto cb = std::make_shared<ModelCommandBuffer>();
Gate gate;
cb->replayGate = &gate;
auto resource = std::make_shared<Resource>("texture");
{
ModelBlitEncoder blit(cb);
blit.copy(resource, "toDrawable");
blit.endEncoding();
}
resource.reset(); // the game drops its reference right after the call
// hold the worker inside the replay, then check that only the posted op keeps the resource alive
gate.release();
Indium::SubmitThread::drain("model.drain");
cb->replayGate = nullptr;
bool replayed = false;
for (const auto& entry: snapshot()) if (entry == "blit:toDrawable") replayed = true;
assert(replayed);
assert(resource.use_count() == 0); // released on the worker after the replay
printf("PASS resource lifetime: op owned the resource until its replay, released afterwards\n");
}
void testDropAfterCommit() {
applied.clear();
auto cb = std::make_shared<ModelCommandBuffer>();
cb->commit(); // the command buffer is closed
{
ModelBlitEncoder blit(cb);
blit.copy(std::make_shared<Resource>("late"), "late");
blit.endEncoding(); // must be dropped, not recorded into a closed command buffer
}
Indium::SubmitThread::drain("model.drain");
assert(droppedChunks.load() == 1);
for (const auto& entry: snapshot()) assert(entry != "blit:late");
cb->waitDeferred(); // balanced deferred count: returns immediately
printf("PASS drop-after-commit: chunk dropped, deferred count balanced\n");
}
void testWaitDeferred() {
applied.clear();
auto cb = std::make_shared<ModelCommandBuffer>();
Gate gate;
cb->replayGate = &gate;
{
ModelBlitEncoder blit(cb);
blit.copy(std::make_shared<Resource>("held"), "held");
blit.endEncoding();
}
std::atomic<bool> returned {false};
std::thread waiter([&]() {
cb->waitDeferred();
returned.store(true);
});
std::this_thread::sleep_for(std::chrono::milliseconds(50));
assert(!returned.load()); // still replaying: the waiter must block
gate.release();
waiter.join();
assert(returned.load());
cb->replayGate = nullptr;
cb->waitDeferred(); // and be a no-op once the replay ended
printf("PASS waitDeferred blocks until the deferred op was replayed\n");
}
}
int main() {
testOrder();
testResourceLifetime();
testDropAfterCommit();
testWaitDeferred();
printf("PASS all deferred-encoder model tests\n");
return 0;
}
+214
View File
@@ -0,0 +1,214 @@
// Host test for the ordered submit worker's hand-off primitives (Indium, Linux build of the real headers).
//
// The game's thread and the submit worker exchange work through submit-thread.private.hpp; the two things that decide
// frame pacing are (a) FIFO order of everything posted, and (b) how long a caller waits at the frame end. This test
// exercises the real header on the host (no Vulkan, no guest):
//
// 1. post() runs tasks on one worker thread, in post order;
// 2. the present-dispatch Token returns as soon as *its own* task ran, even with a later task still queued, while
// drain() waits for everything;
// 3. the wait-site counters (waits.private.hpp) count only waits that actually blocked, and produce the documented
// summary line (the test points DARLING_WAITS_LOG at a temporary file and creates /tmp/metal-debug-waits);
// 4. enabling them writes exactly one "waits enabled" line at the first check, and without the flag (a child
// process forked before the flag file is created) they stay off and write nothing, even for a blocking wait.
//
// Build (Linux host):
// g++ -std=c++17 -O1 -pthread -I build/darling-src/src/external/metal/deps/indium/private-include \
// tools/test-deferred-handoff.cpp -o /tmp/opencode/test-deferred-handoff
// Run: /tmp/opencode/test-deferred-handoff
#include <indium/submit-thread.private.hpp>
#include <atomic>
#include <cassert>
#include <chrono>
#include <condition_variable>
#include <cstdio>
#include <cstdlib>
#include <fstream>
#include <mutex>
#include <string>
#include <thread>
#include <sys/wait.h>
#include <unistd.h>
#include <vector>
namespace {
std::mutex logMutex;
std::vector<std::string> log;
std::atomic<bool> sawWorkerThread {false};
void append(const std::string& entry) {
std::lock_guard<std::mutex> lock(logMutex);
log.push_back(entry);
}
bool contains(const std::string& entry) {
std::lock_guard<std::mutex> lock(logMutex);
for (const auto& item: log) if (item == entry) return true;
return false;
}
std::vector<std::string> snapshot() {
std::lock_guard<std::mutex> lock(logMutex);
return log;
}
void sleepMs(int ms) {
std::this_thread::sleep_for(std::chrono::milliseconds(ms));
}
// a task that only runs once the test opens its gate
struct Gate {
std::mutex mutex;
std::condition_variable condvar;
bool open = false;
void wait() {
std::unique_lock<std::mutex> lock(mutex);
condvar.wait(lock, [this] { return open; });
}
void release() {
{
std::lock_guard<std::mutex> lock(mutex);
open = true;
}
condvar.notify_all();
}
};
void testFifo() {
constexpr int count = 200;
for (int i = 0; i < count; ++i) {
Indium::SubmitThread::post([i]() {
sawWorkerThread = Indium::SubmitThread::onWorker();
append("t" + std::to_string(i));
});
}
sleepMs(100); // let the worker finish, so the drain below is the idle (uncounted) kind
Indium::SubmitThread::drain("test.fifo.idle");
auto entries = snapshot();
assert(entries.size() == (size_t) count);
for (int i = 0; i < count; ++i) assert(entries[i] == "t" + std::to_string(i));
assert(sawWorkerThread.load());
assert(!Indium::SubmitThread::onWorker()); // the test thread is the caller, not the worker
printf("PASS fifo-order (%d tasks, post order preserved)\n", count);
}
void testNarrowToken() {
Gate gate;
Indium::SubmitThread::Token token;
// this commit's task: the "present dispatch" happens here, then the token signals
Indium::SubmitThread::post([&token]() {
sleepMs(2);
append("present");
token.signal();
});
// a later task (another frame's upload) that the caller does not depend on
Indium::SubmitThread::post([&gate]() {
gate.wait();
append("upload");
});
token.wait("test.token");
assert(contains("present"));
assert(!contains("upload")); // the narrow wait must not wait for tasks queued after its own
printf("PASS present-dispatch token (returned while a later task was still queued)\n");
gate.release();
Indium::SubmitThread::drain("test.gated");
assert(contains("upload")); // drain does wait for it
printf("PASS drain-waits-for-later-tasks\n");
}
void testIdleDrainIsNotCounted() {
Indium::SubmitThread::drain("test.idle");
printf("PASS idle drain returns immediately\n");
}
std::vector<std::string> waitsLines() {
std::ifstream file(getenv("DARLING_WAITS_LOG"));
std::vector<std::string> lines;
std::string line;
while (std::getline(file, line)) if (line.compare(0, 6, "waits@") == 0) lines.push_back(line);
return lines;
}
size_t enabledLines() {
std::ifstream file(getenv("DARLING_WAITS_LOG"));
size_t count = 0;
std::string line;
while (std::getline(file, line)) if (line.compare(0, 14, "waits enabled@") == 0) ++count;
return count;
}
bool anyLineHas(const std::vector<std::string>& lines, const std::string& needle) {
for (const auto& line: lines) if (line.find(needle) != std::string::npos) return true;
return false;
}
void testCounters() {
// a blocking drain: one task sleeps, so the caller really blocks
Indium::SubmitThread::post([]() { sleepMs(20); append("slow"); });
Indium::SubmitThread::drain("test.blocking");
assert(contains("slow"));
// a fresh dump window, then one idle drain (which must not be counted) and a probe record
sleepMs(5200);
Indium::SubmitThread::drain("test.idle2");
Indium::Waits::record("test.probe", 1000);
auto lines = waitsLines();
assert(!lines.empty());
assert(enabledLines() == 1); // the first check announced the counters, exactly once
assert(anyLineHas(lines, "test.blocking[n=1 "));
assert(anyLineHas(lines, "test.probe[n=1 "));
assert(anyLineHas(lines, "test.token[n=1 "));
assert(anyLineHas(lines, "test.gated[n=1 "));
// every wait that found its predicate already true: never recorded
assert(!anyLineHas(lines, "test.idle"));
printf("PASS counters: only blocking waits recorded, summary format ok\n");
{
std::ifstream file(getenv("DARLING_WAITS_LOG"));
std::string line;
while (std::getline(file, line)) if (line.compare(0, 14, "waits enabled@") == 0) printf(" %s\n", line.c_str());
}
for (const auto& line: lines) printf(" %s\n", line.c_str());
}
}
int main() {
// The counters latch their flag at the first use; point the summary at a temporary file and enable the counters.
std::string logPath = "/tmp/opencode/waits-test.log";
unlink(logPath.c_str());
setenv("DARLING_WAITS_LOG", logPath.c_str(), 1);
bool flagExisted = access("/tmp/metal-debug-waits", F_OK) == 0;
if (!flagExisted) {
// default (no flag file): a child that has never checked the flag must see the counters off, and a blocking
// wait must leave the log untouched (no "waits enabled" line, no summary)
pid_t child = fork();
assert(child >= 0);
if (child == 0) {
if (Indium::Waits::enabled()) _exit(2);
Indium::SubmitThread::post([]() { sleepMs(20); });
Indium::SubmitThread::drain("test.disabled");
{ Indium::Waits::Block block("test.disabled.block"); sleepMs(1); }
_exit(access(logPath.c_str(), F_OK) == 0 ? 3 : 0);
}
int status = 0;
assert(waitpid(child, &status, 0) == child);
assert(WIFEXITED(status) && WEXITSTATUS(status) == 0);
printf("PASS counters off without the flag file (nothing written)\n");
FILE* flag = fopen("/tmp/metal-debug-waits", "w");
assert(flag != nullptr);
fclose(flag);
}
assert(Indium::Waits::enabled());
testIdleDrainIsNotCounted();
testFifo();
testNarrowToken();
testCounters();
if (!flagExisted) unlink("/tmp/metal-debug-waits");
printf("PASS all deferred hand-off tests\n");
return 0;
}
+3 -1
View File
@@ -279,6 +279,7 @@
"modified_files": 36,
"new_files": [
"private-include/indium/counters.private.hpp",
"private-include/indium/deferred-blit-encoder.private.hpp",
"private-include/indium/deferred-render-encoder.private.hpp",
"private-include/indium/gpu-timing.private.hpp",
"private-include/indium/memory-pool.private.hpp",
@@ -288,7 +289,8 @@
"private-include/indium/retirement-queue.private.hpp",
"private-include/indium/submission-batch.private.hpp",
"private-include/indium/submit-batch.private.hpp",
"private-include/indium/submit-thread.private.hpp"
"private-include/indium/submit-thread.private.hpp",
"private-include/indium/waits.private.hpp"
]
}
],