mirror of
https://github.com/lucannez64/darling-lol.git
synced 2026-10-26 19:47:42 +00:00
Cut Darling overhead in League fights and add passive match capture
Game process (ARAM Hexgate: 84 -> 127 FPS average, slowest stretches ~110): - Local pthread mutexes, condition variables and Mach semaphores are on by default in the game process only, with /tmp/rclol-no-local-* opt-outs. The main thread's darlingserver waits fell from 1.36 to 0.10 ms per frame. - Local semaphores: the count and the slot generation change in one compare-exchange, a destroy leaves a negative marker, the table is reset after fork, and names the table did not create go to the server. - Metal: drop identical re-binds (keyed on the Indium object, depth-stencil retained), pass blit resources by reference, and stop resetting vertex descriptors twice. - Core isolation of the main thread is opt-in (/tmp/rclol-core-isolation); it lowered FPS in its only match. - Optimized libc/libc++/Foundation overlays, AVX2 string primitives, real CPU topology and their installers and offline checks. Tools and docs: - tools/riot-match-capture: passive /proc sampler, user-space IP profiler and reports, plus a comparison against the game's netstats. - tools/riot-lsem-model and tools/riot-metal-bind-model: host models that extract the shipped code and check it. - tools/install-riot-metal-pair.sh installs Metal and Indium together. - Status, setup, testing and troubleshooting docs updated with the measured matches and the switch table; manifest file counts match the patches. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
lucannez64andClaude Opus 5.5 committed
1 parent
5609d96f16
commit
4370f375cd
51 files changed
+8817
-498
No files matched your search
+65
-1
@@ -32,14 +32,78 @@ These commands assume the pinned, patched source tree and its configured Makefil
|
||||
|
||||
```sh
|
||||
cmake --build build/darling-src/build --target darlingserver system_kernel --parallel 1
|
||||
python3 build/runtime/tools/build-riot-kqueue-runtime.py
|
||||
sudo python3 tools/install-riot-darling-server.py
|
||||
python3 tools/install-riot-memory-runtime.py
|
||||
```
|
||||
|
||||
These two Python installers derive paths from the checkout/configuration and may be run directly from `tools/`; tokenized framework builders still require their rendered copies. The server installer defaults to `/usr/local/bin/darlingserver`, saves a checksum-named backup under `build/backups/darlingserver`, and preserves active server mappings. The memory installer updates the guest `RuntimeLibraries` and per-prefix bootstrap copy of `libsystem_kernel.dylib`, plus `build/riot-libsystem_kernel.dylib` if present. It leaves the system SDK alone. Each replacement is staged and verified before installation; the memory installer prepares every destination before replacing any. Replacement is atomic per file, not a transaction across all files.
|
||||
These two Python installers derive paths from the checkout/configuration and may be run directly from `tools/`; tokenized framework builders still require their rendered copies. The server installer defaults to `/usr/local/bin/darlingserver`, saves a checksum-named backup under `build/backups/darlingserver`, and preserves active server mappings. The CMake tree is configured without an optimization level, so `build-riot-kqueue-runtime.py` relinks `build/riot-libsystem_kernel.dylib` with the per-call paths rebuilt at `-O2`; the memory installer installs that relink by default into the guest `RuntimeLibraries` and the per-prefix bootstrap copy, and never overwrites it with a different library. It leaves the system SDK alone. Each replacement is staged and verified before installation; the memory installer prepares every destination before replacing any. Replacement is atomic per file, not a transaction across all files.
|
||||
|
||||
Memory backups default to **`build/memory-runtime-backups`**, separate from the potentially root-owned directory created by the sudo server installer. If a custom backup directory is unwritable, choose another with `--backup-dir /path/to/writable/backups`. See [permission troubleshooting](TROUBLESHOOTING.md#memory-installer-permission-denied). Neither installer stops or starts any process.
|
||||
|
||||
### Real CPU topology and the Foundation overlay
|
||||
|
||||
The kernel relink also reports the host's real CPU counts for `hw.*`/`machdep.cpu.*` (previously a fixed 8 logical / 8 physical MacBookPro16,1 configuration, which made the game create 7 worker threads on a 12-thread host). The memory installer above installs that library; no extra step is needed for the sysctl side. The rebuild also produces the commpage fix (`kNumCPUs`, physical/logical counts) as `build/riot-mldr`, whose SDK copy is root-owned:
|
||||
|
||||
```sh
|
||||
sudo install -m755 build/riot-mldr /usr/local/libexec/darling/usr/libexec/darling/mldr
|
||||
```
|
||||
|
||||
The optimized Foundation overlay is built and installed per user:
|
||||
|
||||
```sh
|
||||
python3 build/runtime/tools/build-riot-foundation-runtime.py
|
||||
python3 tools/install-riot-foundation-runtime.py
|
||||
```
|
||||
|
||||
The installer seeds `~/.darling/Users/<user>/RuntimeFrameworks/Foundation.framework` from the SDK (the same overlay mechanism CoreFoundation/AppKit use), replaces `Versions/C/Foundation` atomically after staging a checksum-named backup in `build/foundation-runtime-backups`, and prints JSON; `--binary`/`--backup-dir` override the defaults. Do not run it while a client is mid-update; a restart is required for activation. To A/B the old fixed CPU counts without reinstalling, run `touch /tmp/rclol-fixed-cpu-count` before starting a guest process and `rm /tmp/rclol-fixed-cpu-count` to go back to the real topology (the flag is read once per process). **The in-game FPS effect of both changes is unmeasured.** The topology change can add worker threads to the clients as well; if clients become busier during matches, use the opt-out flag for a controlled comparison. Rollbacks: `build/memory-runtime-backups` and `build/foundation-runtime-backups` hold the previous libraries; the pre-change kernel artifact is `build/perf-research/impl-cpu/riot-libsystem_kernel.dylib.before`.
|
||||
|
||||
### Optimized libc, libm, libc++ and string-primitive overlays
|
||||
|
||||
Four more libraries have optimized overlays. They are built from the configured tree without a full CMake rebuild:
|
||||
|
||||
```sh
|
||||
sh build/runtime/tools/build-riot-libs-runtime.sh
|
||||
python3 build/runtime/tools/install-riot-libs-runtime.py
|
||||
```
|
||||
|
||||
The builders recompile libsystem_c (618 of its 641 objects), libc++ and libc++abi (38 and 18) and the libsystem_platform string primitives at `-O2`, each object with its own configured defines, includes and per-file options, and relink with the configured link line. `-fno-strict-aliasing` is added for this old BSD C and no `-DNDEBUG`, `-ffast-math` or LTO is. Files whose correctness depends on frame layout, on running in a forked child, or on the floating-point environment keep their configured flags: the fork paths and the stack guard handler in libc and the frame-pointer backtrace walker. libsystem_m is not rebuilt: its sources are built at `-O0` upstream, and an `-O2` rebuild made the Riot client UI render a black window. The libkqueue objects libsystem_c also links belong to the kernel library's target and are left alone. Each builder stops while a League game process is running, because compiling would skew the FPS the user measures.
|
||||
|
||||
The installer writes the four files into `~/.darling/Users/<user>/RuntimeLibraries`, the directory the launcher puts in `DYLD_LIBRARY_PATH`; dyld searches those directories by leaf name before the install path, which is why an overlay there replaces the prefix and SDK copies of `libsystem_c.dylib`, `libc++.1.dylib`, `libc++abi.dylib` and `libsystem_platform.dylib` without touching either. Every replaced file is backed up first (`build/libs-runtime-backups`), the staged file is verified against the built artifact, and the rename is atomic. The installer never stops a process, so a running guest keeps the library it already loaded; exit the game and both clients and restart the namespace for activation. One command restores the previous state:
|
||||
|
||||
```sh
|
||||
python3 build/runtime/tools/install-riot-libs-runtime.py --rollback
|
||||
```
|
||||
|
||||
`--rollback` restores every file this installer replaced and removes the ones it created, so the SDK copy is used again. The policy and the rollback are checked offline by `python3 tools/riot-libs-installer-offline-check.py`, and each artifact is compared with the library it replaces by `python3 tools/riot_overlay_verify.py <artifact> <effective>` (same install name, versions, dependencies, re-exports, segment protections and exported symbols, no new undefined symbol). **The in-game FPS effect of these overlays is unmeasured.** The per-call effect of the string primitives is measured on the host (`build/riot-platform-memory-bench`); see [testing](TESTING.md#optimized-library-overlays-and-string-primitives).
|
||||
|
||||
### Metal and Indium overlays
|
||||
|
||||
The Metal wrapper and Indium share a private blit interface, so install them as a pair:
|
||||
|
||||
```sh
|
||||
python3 build/runtime/tools/build-riot-indium-runtime.py
|
||||
python3 build/runtime/tools/build-riot-metal-runtime.py
|
||||
sh build/runtime/tools/install-riot-metal-pair.sh
|
||||
```
|
||||
|
||||
The installer copies `build/riot-Metal` and `build/riot-libindium.dylib` into the guest `RuntimeFrameworks/Metal.framework` and `RuntimeLibraries`. It backs up the pair it replaces in `build/metal-pair-backups`; a repeated install keeps the older backup. `sh build/runtime/tools/install-riot-metal-pair.sh --rollback` restores that backup. `tools/install-riot-gpu-runtime.sh` still builds and installs the whole GPU set, including these two. Activation needs the namespace restart below.
|
||||
|
||||
### Game-process switches
|
||||
|
||||
The kernel library decides these once when a process starts, and only for the League game process (`.../LeagueofLegends.app/Contents/MacOS/LeagueofLegends`). The Riot and League clients keep the server path. Each switch is a host file, so it takes effect at the next game start without reinstalling:
|
||||
|
||||
| Path | Default in the game | Host file |
|
||||
|---|---|---|
|
||||
| Local pthread mutexes (first-fit) | on | `/tmp/rclol-no-local-mutex` turns it off |
|
||||
| Local condition variables | on | `/tmp/rclol-no-local-condvars` turns it off |
|
||||
| Local Mach semaphores | on | `/tmp/rclol-no-local-semaphores` turns it off |
|
||||
| Core isolation of the main thread | off | `/tmp/rclol-core-isolation` turns it on (it lowered FPS) |
|
||||
| Region-query pacing (50 µs) | on | `/tmp/rclol-region-pace-us` overrides the interval |
|
||||
| Main-thread RPC reply polling | off | `/tmp/rclol-rpc-spin-us` (µs, at most 500) turns it on |
|
||||
|
||||
The former opt-in files `/tmp/rclol-local-mutex`, `/tmp/rclol-local-condvars` and `/tmp/rclol-local-semaphores` are ignored. Each game start appends its decisions to `/tmp/rclol-game-decisions.log`, for example `local_semaphores=1` and `core_isolation=0 errno=-1`. `-all` variants (`/tmp/rclol-local-mutex-all` and the like) enable a local path in every process; they exist for probe tests and are not for normal play.
|
||||
|
||||
An on-disk update is not activation. Existing processes keep their old executable/library mappings, and Darling's active overlay can expose an old library even to a new guest process. After manually exiting the game and both Riot/League clients, restart the entire namespace:
|
||||
|
||||
```sh
|
||||
|
||||
+28
-2
@@ -1,7 +1,33 @@
|
||||
# Snapshot status — 2026-10-13
|
||||
# Snapshot status — 2026-10-14
|
||||
|
||||
This repository preserves a development runtime, including historical experiments. All 29 patches apply to their pinned revisions and the exported source matches the development tree for the patched files. A fresh checkout/build/install has not been validated end to end on another machine. The normal workflow uses manual client installation and play, without debugger attachment, client API calls or input automation.
|
||||
|
||||
## Game FPS, 2026-10-13/14
|
||||
|
||||
All matches were ARAM, played manually with unchanged game settings; FPS is the game's own 10-second average (`netstats.csv`). Bilgewater runs a few percent below Hexgate on the same build.
|
||||
|
||||
| Match | Map | Runtime | Avg FPS | Slowest quarter |
|
||||
|---|---|---|---|---|
|
||||
| 10-13 12:51 | Hexgate | before this work | 83.8 | — |
|
||||
| 10-13 13:14 | Bilgewater | before this work | 76.9 | — |
|
||||
| 10-13 16:54 | Hexgate | first kernel batch | 110.7 | — |
|
||||
| 10-13 20:36 | Bilgewater | + `-O2` libc/libc++, AVX2 string primitives, real CPU count | 109.7 | — |
|
||||
| 10-14 13:04 | Hexgate | lock switches off (lost to a reboot) | 119.2 | 104.2 |
|
||||
| 10-14 14:18 | Hexgate | + core isolation (since made opt-in) | 102.9 | 92.0 |
|
||||
| 10-14 14:42 | Bilgewater | local mutexes, condition variables and semaphores | 119.5 | 103.4 |
|
||||
| 10-14 16:26 | Hexgate | + Metal bind deduplication and cheaper blits | 126.7 | 109.6 |
|
||||
|
||||
The 13:04 and 16:26 Hexgate matches were captured with the passive sampler (`tools/riot-match-capture`). Per frame, the game's main thread waited 1.36 ms on darlingserver in the first and 0.10 ms in the second. It blocked 39 times per frame in the first and 14 in the second. darlingserver CPU fell from 82% to 61% of a core. GPU utilization stayed at 70–74%: the main thread is the limit and is now about 90% busy running code. Frame times within each 10-second window vary by about 1–1.4 ms (standard deviation) and never more than 2.7 ms, versus hitch windows of 77–168 ms in the matches of 10-13 morning.
|
||||
|
||||
What changed in the runtime (all in the patches):
|
||||
|
||||
- **Process-local pthread mutexes and condition variables** (`psynch_mutexwait.c`, `bsdthread_register.c`): contended first-fit mutexes and their condition variables sleep on futexes inside the game instead of making two darlingserver round trips. On by default in the League game process only.
|
||||
- **Process-local Mach semaphores** (`mach_traps.c`): semaphores the game creates are counted and waited on in the process. A destroyed or reused slot can no longer satisfy a wait or take another semaphore's signal: the count and the slot generation change in one compare-exchange, and a destroy leaves a negative marker. A forked child resets the table, and every name the table did not create goes to the server. The game's worker threads wake 380 times a second instead of 940. On by default in the game process only. The 2026-10-10 client crash happened while the switch was on in every process; the clients still use the server path. A local wait cannot be interrupted by `thread_abort`, and a semaphore shared with another task is not supported locally.
|
||||
- **Metal render encoder** (`MTLRenderCommandEncoder.mm`): re-binding the identical texture, sampler, buffer and offset, pipeline or depth-stencil state at the same slot is dropped before any work. The comparison key is the Indium object, which the encoder's binding tables keep alive, so a reused wrapper address cannot alias. The depth-stencil state, which Indium does not retain, is retained by the record. `endEncoding` clears the records. Blit encoder calls pass the resources by reference: 5 atomic operations per copy become 1. `-[MTLVertexDescriptor reset]` no longer resets every attribute twice.
|
||||
- **Core isolation** (`mach_traps.c`, opt-in with `/tmp/rclol-core-isolation`): gives the game's main thread its own Linux core-scheduling cookie. In its one match the main thread waited for a CPU 12.6% of the time instead of 1%, and FPS fell from 118 to 103 over the same minutes. It stays in the tree for experiments only.
|
||||
|
||||
Where the main thread's time goes now: about 72% of its user time is in the game binary, mostly in memory-latency-bound code. The game binary runs natively and selects its AVX paths with `cpuid`. Objective-C dispatch, the Metal wrapper, Indium, the allocator and the Vulkan driver take most of the rest. The remaining Darling-side levers are smaller: more of the heap on transparent huge pages (35% of anonymous memory is), and the per-draw Metal/Objective-C path.
|
||||
|
||||
## New PC: verified runtime changes
|
||||
|
||||
The current machine uses a Ryzen 5 5500, RX 6600 and Linux 7.2.9. The source snapshot includes the build corrections needed with its newer Clang/LLVM: x86-64-only OpenDirectory, consistent objc4 debug macros, older Objective-C collection code generation and LLVM 22/23 branch opcode compatibility.
|
||||
@@ -22,7 +48,7 @@ Server-only CPU samples identified worker context switching and receive/message
|
||||
|
||||
The receive queue now retains unused receive slots, transfers real messages into the existing queue and adopts every returned control header before potentially throwing. This preserves received descriptor ownership even if deque allocation fails. Disabled debug logging no longer prepares PID/TID strings. Independent host socket checks reduce warmed empty receive allocations from 32 to zero and single-packet allocations from 64 to two. Normal and address/undefined-behavior sanitizer checks passed, including exception cleanup and descriptor ownership.
|
||||
|
||||
The normal launcher no longer enables a client inspector, starts scheduling/update/link helpers or forces unlimited UI frames. The user installed the new server atomically while the old server remained active. **No post-activation FPS gain has been measured.** Validate through manually played matches after a full Darling restart when the game and clients are closed.
|
||||
The normal launcher no longer enables a client inspector, starts scheduling/update/link helpers or forces unlimited UI frames. The user installed the new server atomically while the old server remained active. Later matches measured the gains of this and the following changes; see [Game FPS, 2026-10-13/14](#game-fps-2026-10-1314).
|
||||
|
||||
## Memory reclamation and activation
|
||||
|
||||
|
||||
+171
-6
@@ -9,11 +9,13 @@ After preparing the pinned source tree:
|
||||
```sh
|
||||
python3 scripts/prepare.py --check-only
|
||||
python3 tools/riot-launcher-offline-check.py
|
||||
python3 tools/riot-memory-installer-offline-check.py
|
||||
python3 tools/riot-foundation-installer-offline-check.py
|
||||
node tools/riot-league-update-check.js
|
||||
node tools/riot-open-url-check.js
|
||||
```
|
||||
|
||||
The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. The JavaScript checks use mock Electron/filesystem/HTTP objects; they do not enable the legacy hooks in a real client. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads.
|
||||
The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. The memory-installer check imports the installer and exercises its policy (default source is the prepared artifact; the artifact is never a destination of a different library) in temporary directories. The Foundation-installer check does the same for the overlay installer: it seeds a temporary framework from a stub SDK, replaces `Versions/C/Foundation`, keeps the standard symlinks, stages a backup, treats a re-run as current and refuses a non-Mach-O source. The JavaScript checks use mock Electron/filesystem/HTTP objects; they do not enable the legacy hooks in a real client. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads.
|
||||
|
||||
## Standalone guest probes
|
||||
|
||||
@@ -62,16 +64,53 @@ Expected: `warm_empty_allocations=0 single_packet_allocations=2`, correct creden
|
||||
|
||||
The same source passed AddressSanitizer/UndefinedBehaviorSanitizer and leak detection. Allocation counts depend on this fixture/compiler/library; preserve the no-wasted-slot property rather than interpreting them as game performance. See [setup](SETUP.md#updating-an-existing-runtime) for server build/install/activation.
|
||||
|
||||
## Host reply path regression
|
||||
|
||||
The direct-reply checks run entirely on independent host sockets and exercise the real `src/message.cpp` (`MessageQueue::pushAndSend`, `sendMany`, `empty`):
|
||||
|
||||
```sh
|
||||
c++ -std=c++17 -O2 -pthread \
|
||||
-I build/darling-src/src/external/darlingserver/internal-include \
|
||||
tools/riot-host-reply-check.cpp \
|
||||
build/darling-src/src/external/darlingserver/src/message.cpp \
|
||||
-o build/riot-host-reply-check
|
||||
build/riot-host-reply-check
|
||||
```
|
||||
|
||||
Expected: `HOST_REPLY pass=1 direct_send=ok eagain_retry=ok fifo=ok concurrency=ok notify=ok descriptors=ok dead_peer=ok`. It covers: a direct send that drains the queue does not invoke the wakeup callback while a queued message does; EAGAIN leaves the queue intact and the retry path delivers every message in order; four concurrent pushers plus a draining "main loop" preserve per-pusher order without loss or duplication; descriptors are delivered exactly once, closed on the sender side right after the send and released on the receiver side; and a dead peer (ENOENT after unlink, ECONNREFUSED after close) drops only its own message without wedging the queue or leaking descriptors. Compile the same way with `-fsanitize=address,undefined` and `-fsanitize=thread`; the ThreadSanitizer run is the important one.
|
||||
|
||||
A same-process microbenchmark compares the direct path with the old `push` + eventfd + main-loop `sendMany` hand-off:
|
||||
|
||||
```sh
|
||||
c++ -std=c++17 -O2 -pthread \
|
||||
-I build/darling-src/src/external/darlingserver/internal-include \
|
||||
tools/riot-host-reply-bench.cpp \
|
||||
build/darling-src/src/external/darlingserver/src/message.cpp \
|
||||
-o build/riot-host-reply-bench
|
||||
build/riot-host-reply-bench 5000
|
||||
```
|
||||
|
||||
Expected: a `HOST_REPLY_BENCH same_process=1 ...` line; the direct path is a few microseconds faster on average. Both endpoints are threads of one process, so this isolates the hand-off, not real guest scheduling.
|
||||
|
||||
## Memory reclamation regression
|
||||
|
||||
The memory-reclamation check links the actual Darling `sys_madvise` implementation to host syscall hooks and uses only its own 4 MiB anonymous allocation:
|
||||
The memory-reclamation check links the actual Darling `sys_madvise` source to host syscall hooks and a `read_string` stub that selects the host's transparent-huge-page policy, and uses only its own 32 MiB anonymous allocation:
|
||||
|
||||
```sh
|
||||
python3 tools/build-riot-host-madvise-check.py
|
||||
build/logs/madvise-fix/host-madvise-check
|
||||
```
|
||||
|
||||
Expected: `reclaimable_KiB=4096 reuse_reclaimable_KiB=0 contents=preserved`. The old implementation returns success without marking any pages reclaimable. Darwin `MADV_FREE` and `MADV_FREE_REUSABLE` now map to Linux `MADV_FREE`; Darwin `MADV_FREE_REUSE` uses `MADV_POPULATE_WRITE` to cancel lazy reclamation while preserving surviving contents. The reuse operation requires Linux 5.14 or newer; an unsupported or invalid operation returns the translated kernel error rather than reporting false success. Reclamation is lazy and does not promise an immediate RSS decrease when RAM is available.
|
||||
The build script runs the check once per policy. Expected:
|
||||
|
||||
```
|
||||
HOST_MADVISE pass=1 mode=always madvise_calls=6 sub_page_syscalls=0 reclaimable_KiB=32768 reuse_reclaimable_KiB=0 anon_huge_KiB=32768/32768 contents=preserved
|
||||
HOST_MADVISE pass=1 mode=never ... reclaimable_KiB=32768 reuse_reclaimable_KiB=0 ...
|
||||
HOST_MADVISE pass=1 mode=madvise ... reclaimable_KiB=32768 reuse_reclaimable_KiB=0 ...
|
||||
```
|
||||
|
||||
`mode=always` covers the deployment host: sub-huge-page hints reach no `madvise` syscall and do not reduce the mapping's `AnonHugePages` (read from `/proc/self/smaps`), the 2 MiB-aligned region is marked reclaimable, an unaligned large range is rounded inward to the contained whole huge pages, and reuse cancels reclamation with contents preserved. `mode=never`/`mode=madvise` prove the full range is still forwarded when the host does not give this layer's mappings huge pages. The old implementation returned success without marking any pages.
|
||||
|
||||
Darwin `MADV_FREE` and `MADV_FREE_REUSABLE` map to Linux `MADV_FREE` for whole 2 MiB huge pages only; Darwin `MADV_FREE_REUSE` uses `MADV_POPULATE_WRITE` over the full requested range to cancel lazy reclamation while preserving surviving contents. The reuse operation requires Linux 5.14 or newer; an unsupported or invalid operation returns the translated kernel error rather than reporting false success. Reclamation is lazy and does not promise an immediate RSS decrease when RAM is available.
|
||||
|
||||
For the actual guest ABI, `tools/build-riot-madvise-check.py` builds an independent executable. After a full restart has activated the installed library, run it as follows; replace `me` and the checkout path:
|
||||
|
||||
@@ -82,9 +121,135 @@ darling shell env \
|
||||
/Volumes/SystemRoot/path/to/darling-lol/build/logs/madvise-fix/riot-madvise-check
|
||||
```
|
||||
|
||||
Expected: `DARLING_MADVISE pass=1 reclaimable_KiB=4096 reuse_reclaimable_KiB=0 contents=preserved`. To test a candidate before installation, place only its `libsystem_kernel.dylib` in a private directory and prepend its guest-visible path to `DYLD_LIBRARY_PATH`. The candidate passed; the previous installed implementation failed with zero reclaimable pages. The probe uses only its own allocation and Linux memory counters, and never contacts a client/game. It tests free/reuse hints directly, not complete Apple malloc or client behavior under memory pressure.
|
||||
Expected: `DARLING_MADVISE pass=1 reclaimable_KiB=4096 reuse_reclaimable_KiB=0 contents=preserved`. The probe marks a 2 MiB-aligned 4 MiB region inside a larger mapping, because only whole huge pages are forwarded; it uses only its own allocation and Linux memory counters, and never contacts a client/game.
|
||||
|
||||
Install the tested library with `python3 tools/install-riot-memory-runtime.py`. It uses user-owned `build/memory-runtime-backups` instead of the shared `build/backups` directory, which an earlier sudo server install may have made root-owned. It atomically replaces the two per-user kernel-library copies and the prepared runtime artifact, without stopping any process or changing the system SDK. The active Darling overlay can retain the old file even for new guest processes after an update to its backing directory; this was verified by differing host versus namespace checksums. After exiting the game and both clients manually, run `darling shutdown`, then `sh scripts/run.sh` to start a new namespace. Do not restart Darling during a match. Verify reclamation and client memory use after that restart.
|
||||
## Host vDSO clock regression
|
||||
|
||||
The clock check compiles the actual `emulation/src/linux_premigration/ext/clock_gettime.c` against a stub ELF bridge backed by the host's `dlopen`/`dlsym`:
|
||||
|
||||
```sh
|
||||
python3 tools/build-riot-host-vdso-clock-check.py
|
||||
build/logs/vdso-clock/host-vdso-clock-check default
|
||||
build/logs/vdso-clock/host-vdso-clock-check no-symbol
|
||||
build/logs/vdso-clock/host-vdso-clock-check race
|
||||
```
|
||||
|
||||
Expected (costs vary with machine load):
|
||||
|
||||
```
|
||||
HOST_VDSO_CLOCK pass=1 mode=default syscall_ns=528 vdso_ns=24 dlopen_calls=1 dlsym_calls=1 million_reads=1 threads=8 fork=preserved
|
||||
HOST_VDSO_CLOCK pass=1 mode=no-symbol ... (syscall fallback, correct values)
|
||||
HOST_VDSO_CLOCK pass=1 mode=race ... dlopen_calls=1 ...
|
||||
```
|
||||
|
||||
`default` reads the clock before the bridge exists (syscall fallback, not cached), resolves the vDSO entry once the bridge appears, checks a million reads per clock id against the raw syscall and across eight threads, and checks that a forked child inherits the resolved pointer. `no-symbol` proves the fallback when the bridge has no entry. `race` starts eight threads before resolution and requires that only one resolution ran.
|
||||
|
||||
Install the tested library with `python3 tools/install-riot-memory-runtime.py`. Its default source is the optimized relink `build/riot-libsystem_kernel.dylib`; if it is missing, build it first with `python3 build/runtime/tools/build-riot-kqueue-runtime.py`. The installer uses user-owned `build/memory-runtime-backups` instead of the shared `build/backups` directory, which an earlier sudo server install may have made root-owned. It atomically replaces the two per-user kernel-library copies, without stopping any process or changing the system SDK. The prepared artifact is never a destination of a different library: a run with an explicit `--binary` replaces the two prefix copies and leaves `build/riot-libsystem_kernel.dylib` untouched (the check `tools/riot-memory-installer-offline-check.py` covers this). The active Darling overlay can retain the old file even for new guest processes after an update to its backing directory; this was verified by differing host versus namespace checksums. After exiting the game and both clients manually, run `darling shutdown`, then `sh scripts/run.sh` to start a new namespace. Do not restart Darling during a match. Verify reclamation and client memory use after that restart.
|
||||
|
||||
## CPU topology regression
|
||||
|
||||
The hw.* / machdep.cpu.* handlers are checked on the host against the host's real topology. The check links the actual `sysctl_hw.c` and `sysctl_unspec.c` to host shims (real `/sys` reads, forwarded syscalls, sentinel Mach replies) and drives each handler directly plus `sysctlbyname("hw.*")` through the real name-to-oid lookup:
|
||||
|
||||
```sh
|
||||
python3 tools/build-riot-host-sysctl-cpu-check.py
|
||||
```
|
||||
|
||||
Expected on this 6-core/12-thread host:
|
||||
|
||||
```
|
||||
HOST_SYSCTL_CPU pass=1 mode=normal ncpu=12 active=12 logicalcpu=12 logicalcpu_max=12 physicalcpu=6 physicalcpu_max=6 model=MacBookPro16,1
|
||||
HOST_SYSCTL_CPU pass=1 mode=affinity ncpu=12 active=4 logicalcpu=4 logicalcpu_max=12 physicalcpu=4 physicalcpu_max=6 model=MacBookPro16,1
|
||||
HOST_SYSCTL_CPU pass=1 mode=no-detect ncpu=8 active=8 logicalcpu=8 logicalcpu_max=8 physicalcpu=8 physicalcpu_max=8 model=MacBookPro16,1
|
||||
HOST_SYSCTL_CPU pass=1 mode=optout ncpu=8 active=8 logicalcpu=8 logicalcpu_max=8 physicalcpu=8 physicalcpu_max=8 model=MacBookPro16,1
|
||||
```
|
||||
|
||||
`normal` is the deployment case: hw.ncpu/hw.logicalcpu_max/hw.physicalcpu_max are the host's online counts, hw.activecpu/hw.availcpu/hw.logicalcpu/hw.physicalcpu follow this process's affinity mask. `affinity` restricts the check to four CPUs first, so avail/active fall to 4 while the max values stay at 12 (and 6). `no-detect` fails every `/sys` CPU read and `sched_getaffinity`, `optout` sets the flag file (a check-local path compiled into that binary); both must fall back to the old fixed 8s. The emulated `hw.model` must stay `MacBookPro16,1` in every mode. The production opt-out flag is `/tmp/rclol-fixed-cpu-count`; the host check overrides only the path, never touches the real file.
|
||||
|
||||
The commpage fields the same change publishes (`_COMM_PAGE_NCPUS`, `_COMM_PAGE_ACTIVE_CPUS`, `_COMM_PAGE_PHYSICAL_CPUS`, `_COMM_PAGE_LOGICAL_CPUS` and the `kNumCPUs` capability bits) come from mldr. `build/riot-mldr` is the rebuilt loader; activating it means replacing the root-owned SDK copy (`/usr/local/libexec/darling/usr/libexec/darling/mldr`) after exiting clients and shutting down Darling, as in the setup notes.
|
||||
|
||||
## Optimized library overlays and string primitives
|
||||
|
||||
The string primitives the guest resolves to are checked on the host against two copies of the same libplatform sources, a byte-loop reference, and an unmapped page on each side of every buffer:
|
||||
|
||||
```sh
|
||||
python3 tools/build-riot-platform-memory-check.py
|
||||
```
|
||||
|
||||
Expected (five runs: the generic path, the vector path, the vector plus REP path, then the same check built with AddressSanitizer and UndefinedBehaviorSanitizer):
|
||||
|
||||
```
|
||||
PLATFORM_MEMORY_CHECK pass=1 capabilities=0 cases=194680 mismatches=0 page_faults=0 variants=2
|
||||
PLATFORM_MEMORY_CHECK pass=1 capabilities=0x20000000 cases=194680 mismatches=0 page_faults=0 variants=2
|
||||
PLATFORM_MEMORY_CHECK pass=1 capabilities=0x28000000 cases=194680 mismatches=0 page_faults=0 variants=2
|
||||
PLATFORM_MEMORY_CHECK pass=1 capabilities=0 cases=194680 mismatches=0 page_faults=0 variants=2
|
||||
PLATFORM_MEMORY_CHECK pass=1 capabilities=0x28000000 cases=194680 mismatches=0 page_faults=0 variants=2
|
||||
```
|
||||
|
||||
`current` is the code and flags the installed `libsystem_platform.dylib` was built from and `new` is what `tools/build-riot-platform-strings.py` produces; every case runs against both and against the reference, for every size up to 512 with a subset of the 64x64 alignments, for random sizes and alignments, for both overlap directions, and for buffers that only just fit the guarded region. The same script builds `build/riot-platform-memory-bench`, which measures the same functions against the host's glibc for every size from 1 to 256 and at 512 B to 1 MiB (`BENCH size=... op=... current_ns=... new_ns=... glibc_ns=...`). That is a throughput measurement of the primitives, not an FPS measurement.
|
||||
|
||||
Which file the guest actually loads for a library, and how that file was compiled, is listed by:
|
||||
|
||||
```sh
|
||||
python3 tools/riot_overlay_audit.py
|
||||
```
|
||||
|
||||
An overlay is compared with the library it would replace by:
|
||||
|
||||
```sh
|
||||
python3 tools/riot_overlay_verify.py build/riot-libsystem_c.dylib /usr/local/libexec/darling/usr/lib/system/libsystem_c.dylib
|
||||
```
|
||||
|
||||
Expected `OVERLAY_VERIFY pass=1` plus the export counts and the stack-reference share before and after. The check requires the same install name and versions, the same dependent libraries, re-exports and segment protections, the same exported symbols (a weak definition the optimized build drops is reported as a note, because the ABI makes a weak definition optional), no new undefined symbol, and an improved optimization estimate. It never starts a guest process.
|
||||
|
||||
The batch installer's policy and rollback are exercised in temporary directories:
|
||||
|
||||
```sh
|
||||
python3 tools/riot-libs-installer-offline-check.py
|
||||
```
|
||||
|
||||
Expected: `LIBS_INSTALLER_OFFLINE pass=1 overlays=5 backup_before_replace=1 rollback_restores=1 rollback_removes=1 idempotent=1 rejects_non_macho=1`. It imports the installer, never executes it, and never touches the real prefix.
|
||||
|
||||
## Local semaphore model
|
||||
|
||||
```sh
|
||||
sh tools/riot-lsem-model/run.sh
|
||||
sh tools/riot-lsem-model/sensitivity-check.sh
|
||||
```
|
||||
|
||||
`extract-lsem.py` copies the shipped local-semaphore table and trap functions verbatim out of the patched `mach_traps.c`; it refuses to run if the generation-checked token compare-exchange or the negative destroy marker is missing. The model compiles that code with shims for the raw syscalls and server RPCs. It runs create/destroy churn with name reuse on 8 threads, waiter/signaler pairs, timed waits, `signal_all`, destroy while waiting, and the XNU ordering of `wait_signal`. Expected: `LSEM_MODEL checks=686 failed=0 spurious_success=0 lost_wakeup=0 steal=0 hang=0`. The sensitivity check re-breaks the protocol in a copy and must report failures: positive tombstone (`failed=5 spurious_success=1`), missing generation bump (`failed=2 spurious_success=400`). Outputs go to `build/riot-lsem-model`.
|
||||
|
||||
## Metal bind model
|
||||
|
||||
```sh
|
||||
sh tools/riot-metal-bind-model/run.sh
|
||||
```
|
||||
|
||||
The bind-tracking helpers are extracted verbatim from `MTLRenderCommandEncoder.mm`. The model drives two encoder states, one receiving every call and one only the forwarded calls, and requires them to be equal after every call. That covers directed scenarios, among them a freed wrapper whose address is reused by a different resource and depth-stencil retention, plus 119,401 random calls. Expected: `randomized calls=119401 forwarded=110294 dropped=9107 (7.6%)`, `BIND_DEDUP_MODEL failures=0` and `BLIT_CAST_MODEL failures=0`.
|
||||
|
||||
## Passive match capture
|
||||
|
||||
These tools observe a manually played match from the host. They wait for the game process, stop when it exits, and never attach to it, write to it, or read its memory.
|
||||
|
||||
```sh
|
||||
python3 tools/riot-match-capture/sample-match.py # start before the match; writes build/logs/host-sample-<stamp>.jsonl
|
||||
python3 tools/riot-match-capture/sample-report.py build/logs/host-sample-<stamp>.jsonl
|
||||
python3 tools/riot-match-capture/compare-matches.py \
|
||||
"before=build/logs/host-sample-A.jsonl:$HOME/.darling/Applications/League of Legends.app/Contents/LoL/Logs/GameLogs/<A>" \
|
||||
"after=build/logs/host-sample-B.jsonl:$HOME/.darling/Applications/League of Legends.app/Contents/LoL/Logs/GameLogs/<B>"
|
||||
```
|
||||
|
||||
`sample-match.py` reads only `/proc` and `/sys`. Per thread of the game and of darlingserver it records scheduler run, runqueue and wakeup counts every second, and the system call each thread is blocked in (50 Hz for the main thread, 10 Hz for the others). It also records GPU busy percentage, the game's huge-page share every 30 seconds, and whether another of our threads runs on the main thread's SMT sibling. `compare-matches.py` aligns the capture with the game's 10-second `netstats.csv` rows. It prints average FPS, the worst row, the slowest quarter and, per frame, the main thread's CPU time, server waits, futex waits and blocks.
|
||||
|
||||
For an instruction-address profile, which needs `perf_event_paranoid` ≤ 2 (user-space samples only), build the sampler once and run the wrapper instead:
|
||||
|
||||
```sh
|
||||
c++ -O2 -std=c++17 -o build/riot-ip-profiler tools/riot-match-capture/ip-profiler.cpp
|
||||
python3 tools/riot-match-capture/profile-match.py
|
||||
python3 tools/riot-match-capture/profile-report.py build/logs/host-profile-<stamp> 120 4
|
||||
python3 tools/riot-match-capture/profile-symbols.py build/logs/host-profile-<stamp> <main tid> Metal libobjc.A.dylib libindium.dylib
|
||||
```
|
||||
|
||||
The profiler samples each thread's user-space instruction pointer every 1 ms on the main thread and every 4 ms on the others, with no call stacks and no registers. The wrapper records the memory map every 15 seconds so the report can attribute samples to modules. Ask the person playing before profiling their match.
|
||||
|
||||
## Manual game observation
|
||||
|
||||
|
||||
@@ -78,6 +78,12 @@ A separate multi-window path stopped the active async presenter when another dif
|
||||
|
||||
## Lower fight FPS than Windows
|
||||
|
||||
Host observation found substantial Darling CPU/RPC costs and main-thread socket waits, with reported fight FPS of 70–80 and quieter moments at 100–120 on the Ryzen 5 5500 / RX 6600. GPU utilization averaged about 61%; the CPU reached 92–94°C. These observations identify areas to investigate, not the full cause of every slow frame.
|
||||
The game code itself runs natively under Darling: the x86-64 slice of the universal binary executes unchanged, picks its instruction-set paths with `cpuid` on the real CPU, and sizes its worker pool from the real logical CPU count. What differs from Windows is the path around it. Every Metal call goes through Objective-C, the Metal wrapper, Indium and Vulkan instead of a native Direct3D driver, and kernel services go through the emulation and darlingserver. In an ARAM fight the game's own code now takes most of a frame; see [status](STATUS.md#game-fps-2026-10-1314) for measurements.
|
||||
|
||||
The server now retains unused receive buffers and avoids preparing disabled debug strings. The host socket fixture reduces warmed empty receive allocations from 32 to zero and single-packet allocations from 64 to two. Normal launches also stop forcing unlimited UI frames and inspector scheduling hooks. An FPS gain has not yet been measured after activation. Compare manually played matches with the same graphics settings and record CPU/GPU utilization, temperature and memory pressure. Do not use legacy automated practice or profiling cycles.
|
||||
If FPS drops back towards the 10-13 morning numbers (75–85 on Hexgate):
|
||||
|
||||
1. Make sure the current kernel library, Metal and Indium overlays are installed and active: exit the game and both clients, then `darling shutdown` and `sh scripts/run.sh` ([setup](SETUP.md#game-process-switches)).
|
||||
2. Check the game's line in `/tmp/rclol-game-decisions.log`: it should read `local_semaphores=1` and `core_isolation=0`. No `/tmp/rclol-no-local-*` file should exist unless you created it on purpose.
|
||||
3. Capture a match with `tools/riot-match-capture/sample-match.py` and compare it with an earlier one using `compare-matches.py` ([testing](TESTING.md#passive-match-capture)). A server wait above about 0.5 ms per frame means one of the local paths is off. A runqueue wait above a few percent on the main thread means the CPU is oversubscribed, for example by `/tmp/rclol-core-isolation` or background load.
|
||||
|
||||
Compare matches on the same map: Bilgewater runs a few percent below Hexgate.
|
||||
Reference in new issue
Block a user