diff --git a/.github/workflows/source-check.yml b/.github/workflows/source-check.yml index 276adf0..d8de01d 100644 --- a/.github/workflows/source-check.yml +++ b/.github/workflows/source-check.yml @@ -12,9 +12,12 @@ jobs: python-version: '3.12' - run: python scripts/validate.py - run: python tools/riot-input-offline-check.py + - run: python tools/riot-launcher-offline-check.py - run: python tools/riot-runtime-cleanup-offline-check.py - run: python tools/riot-cdn-dns-offline-check.py - run: python -m unittest discover -s tests -p 'test_hidden_riot_recovery.py' - run: python -m unittest discover -s tests -p 'test_desktop_run.py' - run: python -m unittest discover -s tests -p 'test_urlsession_runner.py' - run: python -m unittest discover -s tests -p 'test_system_profiler.py' + - run: python -m unittest discover -s tests -p 'test_metal_device_info.py' + - run: python -m unittest discover -s tests -p 'test_exec_guest.py' diff --git a/docs/SETUP.md b/docs/SETUP.md index aa3d7db..4d85c6b 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -73,6 +73,17 @@ opts in to Hyprland-only recovery of a default-prefix Riot session with no visible window; it is off by default. See [troubleshooting](TROUBLESHOOTING.md#riot-exits-and-league-reports-that-it-lost-its-connection) for the observed failures, recovery guards and NSURLSession limitations. +`RCLOL_FILTER_HOST_ENV=1` opts in to removing host desktop-session variables before +the guest starts: `WAYLAND_DISPLAY`, `XDG_SESSION_*`, `XDG_CURRENT_DESKTOP`, +GNOME/KDE/GDM, systemd, SSH agent, terminal, toolkit (`QT_*`, `GTK_*`, `GDK_*`), +Flatpak/Snap/AppImage and exported shell functions, and `SHELL` becomes `/bin/zsh`. +It is off by default. Variables that Darling or the host graphics and audio libraries read +are always kept: `DISPLAY`, `XAUTHORITY`, `XDG_RUNTIME_DIR` and the XDG base directories, +D-Bus, PulseAudio/PipeWire/ALSA, `DRI_PRIME`, `MESA_*`, `__GL_*`, `__NV_*`, `VK_*`, +`LD_PRELOAD` and `LD_LIBRARY_PATH`. List names to keep anyway in `RCLOL_KEEP_HOST_ENV`, +for example `RCLOL_KEEP_HOST_ENV=WAYLAND_DISPLAY`. The launcher always closes inherited +file descriptors above stderr before the guest starts. + ## Updating an existing runtime `./scripts/update.sh` updates everything after `git pull`: `build.sh`, then the base install, then `install-runtime.sh`. When a pull changes a patch, `prepare.py` finds the earlier version applied in that component (its copy in `build/applied-patches`, or a version from this repository's history), reverts it, and applies the new one. It never resets a repository; if neither version matches, it stops and names the repository with other edits. The steps below update single components instead. diff --git a/docs/TESTING.md b/docs/TESTING.md index 8a544f9..2fdedef 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -23,11 +23,11 @@ c++ -std=c++17 -I build/darling-src/src/external/metal/private-include \ build/test-metal-device-info ``` -Expected: `METAL_DEVICE_INFO failures=0 pass=1`. It covers how `MTLDevice.name` turns Vulkan names into macOS style (`AMD Radeon RX 6800 XT (RADV NAVI21)` becomes `AMD Radeon RX 6800 XT`, `Intel(R) UHD Graphics 630 (CML GT2)` becomes `Intel UHD Graphics 630`, NVIDIA names stay as they are), the `DARLING_METAL_VRAM_MB` parser, and the working-set sum over Vulkan heaps. +Expected: `METAL_DEVICE_INFO failures=0 pass=1`. It covers how `MTLDevice.name` turns Vulkan names into macOS style (`AMD Radeon RX 6800 XT (RADV NAVI21)` becomes `AMD Radeon RX 6800 XT`, `Intel(R) UHD Graphics 630 (CML GT2)` becomes `Intel UHD Graphics 630`, NVIDIA names stay as they are), the `DARLING_METAL_VRAM_MB` parser, the working-set sum over Vulkan heaps, and a non-zero `registryID`. `python3 -m unittest tests.test_metal_device_info` runs the same check against the header in `patches/metal.patch`, so it also works without a prepared source tree; CI runs it. The LLVM version check compiles Iridium's actual dynamic loader against small stub libraries. It verifies that a matching major initializes, older/newer majors produce a version mismatch, and a missing library fails initialization. It does not start Darling. The native AIR translator (`python3 tools/build-riot-air-check.py --native`) selects both headers and the shared library from the same `llvm-config`; its build prints those paths so shader results can be attributed to the tested LLVM installation. -The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, and preserves Darling runtime library/framework paths. The memory-installer check imports the installer and exercises its policy (default source is the prepared artifact; the artifact is never a destination of a different library) in temporary directories. The Foundation-installer check does the same for the overlay installer: it seeds a temporary framework from a stub SDK, replaces `Versions/C/Foundation`, keeps the standard symlinks, stages a backup, treats a re-run as current and refuses a non-Mach-O source. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads. +The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, preserves Darling runtime library/framework paths, and checks that host file descriptors the launcher inherited (including multi-digit ones) are closed before the guest starts. The memory-installer check imports the installer and exercises its policy (default source is the prepared artifact; the artifact is never a destination of a different library) in temporary directories. The Foundation-installer check does the same for the overlay installer: it seeds a temporary framework from a stub SDK, replaces `Versions/C/Foundation`, keeps the standard symlinks, stages a backup, treats a re-run as current and refuses a non-Mach-O source. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads. ## NSURLSession lifecycle regression @@ -181,7 +181,7 @@ build/riot-host-receive-check python3 tools/riot-launcher-offline-check.py ``` -Expected: `warm_empty_allocations=0 single_packet_allocations=2`, correct credentials, descriptor ownership/CLOEXEC, sender addresses, multiple batches, truncation flags, callback reentrancy and exception cleanup. The old receive implementation makes 32 allocations for a warmed empty receive and 64 for a single-packet receive in the same fixture. These are allocation counts, not an FPS benchmark. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, and preserves Darling runtime library/framework paths. +Expected: `warm_empty_allocations=0 single_packet_allocations=2`, correct credentials, descriptor ownership/CLOEXEC, sender addresses, multiple batches, truncation flags, callback reentrancy and exception cleanup. The old receive implementation makes 32 allocations for a warmed empty receive and 64 for a single-packet receive in the same fixture. These are allocation counts, not an FPS benchmark. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, preserves Darling runtime library/framework paths, and checks that host file descriptors the launcher inherited (including multi-digit ones) are closed before the guest starts. The same source passed AddressSanitizer/UndefinedBehaviorSanitizer and leak detection. Allocation counts depend on this fixture/compiler/library; preserve the no-wasted-slot property rather than interpreting them as game performance. See [setup](SETUP.md#updating-an-existing-runtime) for server build/install/activation. diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index e9494f3..a6da2e1 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -375,6 +375,27 @@ Tested on a GTX 1060 6GB with driver 580.178.04 on NixOS, Hyprland and Xwayland: the client and the tutorial match render on the NVIDIA GPU. Newer drivers were not retested with the option off. +### Metal device name and memory budget + +`MTLDevice` reports the host GPU: its Vulkan name in macOS style (for example +`AMD Radeon RX 6800 XT (RADV NAVI21)` becomes `AMD Radeon RX 6800 XT`) and a working set +(`recommendedMaxWorkingSetSize`) from the GPU's Vulkan memory heaps. Apps size their GPU +allocations from the working set, so only override it to match the card's real memory. +Both can be overridden per launch: + +```sh +DARLING_METAL_GPU_NAME='AMD Radeon Pro 5500M' DARLING_METAL_VRAM_MB=4096 ./scripts/run.sh +``` + +The Riot launcher sets `DARLING_METAL_MAC_GPU_NAME=1`: the name becomes the GPU that +the MacBookPro16,1 IOKit reports shipped with, from the host's vendor: `AMD Radeon Pro 5500M` +on AMD and `Intel UHD Graphics 630` on Intel. NVIDIA GPUs keep their own name, so the game +never takes AMD or Intel driver paths on an NVIDIA card. The working set is still the host's. +`DARLING_METAL_MAC_GPU_NAME=0 ./scripts/run.sh` reports the host name instead; other Darling +apps report it unless they set the variable. `DARLING_METAL_GPU_NAME` wins over both. + +`registryID` is a fixed non-zero value, as on a real Mac. + ### Host-side checks These are host settings, not game settings. Run on AC power, and use a performance power profile (`powerprofilesctl set performance`, or `asusctl profile -P Performance` on ASUS laptops). On the tested desktop the game is limited mostly by its own main thread on the CPU, not by the GPU (see [status](STATUS.md#game-fps-2026-10-1314)), so laptop CPU power limits matter as much as the GPU choice. diff --git a/patches/metal.patch b/patches/metal.patch index adeb448..597b074 100644 --- a/patches/metal.patch +++ b/patches/metal.patch @@ -1134,7 +1134,7 @@ index 2a57796..0cbedf2 100644 - @end diff --git a/src/Metal/MTLDevice.mm b/src/Metal/MTLDevice.mm -index f58184a..a80a625 100644 +index f58184a..2c45608 100644 --- a/src/Metal/MTLDevice.mm +++ b/src/Metal/MTLDevice.mm @@ -2,14 +2,32 @@ @@ -1218,7 +1218,7 @@ index f58184a..a80a625 100644 } @synthesize device = _device; -@@ -116,6 +168,39 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -116,6 +168,50 @@ void MTLRemoveDeviceObserver(id observer) { [_threadExitCondition unlock]; } @@ -1228,16 +1228,27 @@ index f58184a..a80a625 100644 + // Overrides are read once per process. + static const char* const nameOverride = getenv("DARLING_METAL_GPU_NAME"); + static const uint64_t vramOverride = MTLDeviceInfo::parseMegabytes(getenv("DARLING_METAL_VRAM_MB")); ++ static const bool macName = [] { ++ const char* value = getenv("DARLING_METAL_MAC_GPU_NAME"); ++ return value && value[0] == '1' && value[1] == '\0'; ++ }(); + -+ std::string vulkanName = _device->name(); -+ _name = [[NSString alloc] initWithUTF8String: MTLDeviceInfo::resolveName(nameOverride, vulkanName.c_str()).c_str()]; ++ auto priv = dynamic_cast(_device.get()); ++ std::string vulkanName; ++ try { ++ vulkanName = _device ? _device->name() : ""; ++ } catch (...) { ++ vulkanName = ""; ++ } ++ const char* mac = macName && priv ? MTLDeviceInfo::macGpuName(priv->properties().vendorID) : nullptr; ++ _name = [[NSString alloc] initWithUTF8String: MTLDeviceInfo::resolveName(nameOverride, vulkanName.c_str(), mac).c_str()]; + if (_name == nil) { + _name = [[NSString alloc] initWithUTF8String: MTLDeviceInfo::kFallbackName]; + } + + uint64_t workingSet = 0; + _sharesSystemMemory = NO; -+ if (auto priv = dynamic_cast(_device.get())) { ++ if (priv) { + const VkPhysicalDeviceType type = priv->properties().deviceType; + _sharesSystemMemory = type == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU || type == VK_PHYSICAL_DEVICE_TYPE_CPU; + const VkPhysicalDeviceMemoryProperties memory = priv->memoryProperties(); @@ -1258,7 +1269,7 @@ index f58184a..a80a625 100644 - (instancetype)initWithDevice: (std::shared_ptr)device { self = [super init]; -@@ -123,6 +208,7 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -123,6 +219,7 @@ void MTLRemoveDeviceObserver(id observer) { _device = device; _threadExitCondition = [NSCondition new]; _threadIsRunning = NO; @@ -1266,7 +1277,7 @@ index f58184a..a80a625 100644 _pollingThread = [[NSThread alloc] initWithTarget: self selector: @selector(pollingLoop) object: nil]; [_pollingThread start]; } -@@ -133,6 +219,7 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -133,6 +230,7 @@ void MTLRemoveDeviceObserver(id observer) { { [_pollingThread release]; [_threadExitCondition release]; @@ -1274,7 +1285,7 @@ index f58184a..a80a625 100644 [super dealloc]; } -@@ -153,12 +240,60 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -153,12 +251,60 @@ void MTLRemoveDeviceObserver(id observer) { [_threadExitCondition unlock]; } @@ -1311,7 +1322,7 @@ index f58184a..a80a625 100644 +- (BOOL)isRemovable { return NO; } +- (BOOL)isHeadless { return NO; } +- (NSString*)name { return _name; } -+- (uint64_t)registryID { return 0; } ++- (uint64_t)registryID { return MTLDeviceInfo::kDefaultRegistryID; } +- (BOOL)supportsFamily: (NSUInteger)family { return YES; } +- (BOOL)supportsFeatureSet: (NSUInteger)featureSet { return YES; } +- (BOOL)supportsTextureSampleCount: (NSUInteger)sampleCount { return sampleCount <= 4; } @@ -1336,7 +1347,7 @@ index f58184a..a80a625 100644 if (!pso) { if (error) { // TODO: better error and/or match what the official Metal method does -@@ -166,6 +301,7 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -166,6 +312,7 @@ void MTLRemoveDeviceObserver(id observer) { } return nil; } @@ -1344,7 +1355,7 @@ index f58184a..a80a625 100644 return [[MTLComputePipelineStateInternal alloc] initWithState: pso device: self label: descriptor.label]; } -@@ -183,7 +319,13 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -183,7 +330,13 @@ void MTLRemoveDeviceObserver(id observer) { reflection: (MTLAutoreleasedComputePipelineReflection*)reflection error: (NSError**)error { @@ -1359,7 +1370,7 @@ index f58184a..a80a625 100644 if (!pso) { if (error) { // TODO: better error and/or match what the official Metal method does -@@ -191,13 +333,44 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -191,13 +344,44 @@ void MTLRemoveDeviceObserver(id observer) { } return nil; } @@ -1405,7 +1416,7 @@ index f58184a..a80a625 100644 if (!pso) { if (error) { // TODO: better error and/or match what the official Metal method does -@@ -205,7 +378,114 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -205,7 +389,118 @@ void MTLRemoveDeviceObserver(id observer) { } return nil; } @@ -1453,7 +1464,9 @@ index f58184a..a80a625 100644 +- (id)newDepthStencilStateWithDescriptor:(MTLDepthStencilDescriptor*)descriptor +{ + if (!descriptor) return nil; -+ auto state = _device->newDepthStencilState([descriptor asIndiumDescriptor]); ++ std::shared_ptr state; ++ try { state = _device->newDepthStencilState([descriptor asIndiumDescriptor]); } ++ catch (...) { return nil; } + if (!state) return nil; + return [[MTLDepthStencilStateInternal alloc] initWithState:state device:self label:descriptor.label]; +} @@ -1475,7 +1488,9 @@ index f58184a..a80a625 100644 + desc.lodMaxClamp = descriptor.lodMaxClamp; + desc.supportArgumentBuffers = descriptor.supportArgumentBuffers; + desc.compareFunction = static_cast(descriptor.compareFunction); -+ auto sampler = _device->newSamplerState(desc); ++ std::shared_ptr sampler; ++ try { sampler = _device->newSamplerState(desc); } ++ catch (...) { return nil; } + if (!sampler) return nil; + return [[MTLSamplerStateInternal alloc] initWithSampler:sampler device:self label:descriptor.label]; +} @@ -1521,7 +1536,7 @@ index f58184a..a80a625 100644 } - (id)newCommandQueue -@@ -275,8 +555,21 @@ void MTLRemoveDeviceObserver(id observer) { +@@ -275,8 +570,21 @@ void MTLRemoveDeviceObserver(id observer) { - (id)newLibraryWithData: (dispatch_data_t)data error: (NSError**)error { @@ -2581,7 +2596,7 @@ index 66a9056..f28235c 100644 - @end diff --git a/src/Metal/MTLTexture.mm b/src/Metal/MTLTexture.mm -index 6c71fb9..822c112 100644 +index 6c71fb9..186b8c0 100644 --- a/src/Metal/MTLTexture.mm +++ b/src/Metal/MTLTexture.mm @@ -1,3 +1,5 @@ @@ -2590,7 +2605,7 @@ index 6c71fb9..822c112 100644 // SPDX-FileCopyrightText: 2026 Darling Developers // SPDX-License-Identifier: MPL-2.0 -@@ -7,11 +9,37 @@ +@@ -7,11 +9,38 @@ #import #import @@ -2606,7 +2621,8 @@ index 6c71fb9..822c112 100644 + if (FILE* log = fopen("/tmp/metal-trace.log", "a")) { + const unsigned char* b = (const unsigned char*) bytes; + unsigned long nonZero = 0; -+ size_t probe = std::min(4096, (size_t) region.size.width * 4); ++ // Every pixel format takes at least one byte per pixel, so the first `width` bytes lie inside the first row. ++ size_t probe = std::min(4096, (size_t) region.size.width); + for (size_t i = 0; i < probe; ++i) if (b[i]) ++nonZero; + fprintf(log, "replaceRegion tex=%p %lux%lu fmt=%lu level=%lu region=(%lu,%lu %lux%lu) firstBytesNonZero=%lu/%zu took=%.1fms\n", texture, + (unsigned long) texture.width, (unsigned long) texture.height, (unsigned long) texture.pixelFormat, (unsigned long) level, @@ -2628,7 +2644,7 @@ index 6c71fb9..822c112 100644 @synthesize device = _device; @synthesize resourceOptions = _resourceOptions; @synthesize label = _label; -@@ -164,7 +192,9 @@ +@@ -164,7 +193,9 @@ withBytes: (const void*)pixelBytes bytesPerRow: (NSUInteger)bytesPerRow { @@ -2638,7 +2654,7 @@ index 6c71fb9..822c112 100644 } - (void)replaceRegion: (MTLRegion)region -@@ -174,7 +204,9 @@ +@@ -174,7 +205,9 @@ bytesPerRow: (NSUInteger)bytesPerRow bytesPerImage: (NSUInteger)bytesPerImage { @@ -2736,7 +2752,7 @@ index 37cccb3..79b2350 100644 - @end diff --git a/src/Metal/MTLVertexDescriptor.mm b/src/Metal/MTLVertexDescriptor.mm -index cde968d..aa7fe05 100644 +index cde968d..0a016c6 100644 --- a/src/Metal/MTLVertexDescriptor.mm +++ b/src/Metal/MTLVertexDescriptor.mm @@ -1,24 +1,435 @@ @@ -3015,8 +3031,8 @@ index cde968d..aa7fe05 100644 + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; + MTLVertexBufferLayoutDescriptor* old = _items[index]; + _items[index] = [descriptor retain]; -+ [old release]; + MTLVertexBufferLayoutArrayRetireItem(self, old); ++ [old release]; + const uint32_t bit = 1u << index; + if (descriptor) _present |= bit; else _present &= ~bit; + if (descriptor && object_getClass(descriptor) == [MTLVertexBufferLayoutDescriptor class]) _concrete |= bit; else _concrete &= ~bit; @@ -3077,8 +3093,8 @@ index cde968d..aa7fe05 100644 + if (index >= 31) [NSException raise:NSRangeException format:@"Vertex descriptor index %lu exceeds 30", (unsigned long)index]; + MTLVertexAttributeDescriptor* old = _items[index]; + _items[index] = [descriptor retain]; -+ [old release]; + MTLVertexAttributeArrayRetireItem(self, old); ++ [old release]; + const uint32_t bit = 1u << index; + if (descriptor) _present |= bit; else _present &= ~bit; + if (descriptor && object_getClass(descriptor) == [MTLVertexAttributeDescriptor class]) _concrete |= bit; else _concrete &= ~bit; @@ -3440,7 +3456,7 @@ diff --git a/private-include/Metal/MTLDepthStencilInternal.h b/private-include/M new file mode 100644 --- /dev/null +++ b/private-include/Metal/MTLDepthStencilInternal.h -@@ -0,0 +1,25 @@ +@@ -0,0 +1,27 @@ +// SPDX-FileCopyrightText: 2026 Darling Developers +// SPDX-License-Identifier: MPL-2.0 + @@ -3462,15 +3478,17 @@ new file mode 100644 +- (instancetype)initWithState:(std::shared_ptr)state device:(id)device label:(NSString*)label; +#endif +@end ++#if DARLING_METAL_ENABLED +// Direct access for the encoders: the property getter is a message send that returns the shared_ptr by value. +const std::shared_ptr& MTLDepthStencilGetIndium(id object); ++#endif + +#endif diff --git a/private-include/Metal/MTLDeviceInfo.h b/private-include/Metal/MTLDeviceInfo.h new file mode 100644 --- /dev/null +++ b/private-include/Metal/MTLDeviceInfo.h -@@ -0,0 +1,148 @@ +@@ -0,0 +1,166 @@ +// SPDX-FileCopyrightText: 2026 Darling Developers +// SPDX-License-Identifier: MPL-2.0 + @@ -3489,6 +3507,8 @@ new file mode 100644 + +// Used when neither the cleaned nor the raw Vulkan name has any characters. +static constexpr const char* kFallbackName = "Darling Indium Device"; ++// A real Mac never reports registry ID 0, and apps use it as a key for the device. ++static constexpr uint64_t kDefaultRegistryID = 0x100000439ULL; + +static inline bool isSpace(char c) { + return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v'; @@ -3568,10 +3588,26 @@ new file mode 100644 +} + +// DARLING_METAL_GPU_NAME (used verbatim when non-empty) wins over the Vulkan name. -+static inline std::string resolveName(const char* overrideName, const char* vulkanName) { ++// With DARLING_METAL_MAC_GPU_NAME=1: the GPU from the host's vendor that the MacBookPro16,1 ++// iokitd reports shipped with, by Vulkan vendor ID. nullptr for other vendors (NVIDIA), which ++// keep their own name so apps never take another vendor's driver paths. ++static inline const char* macGpuName(uint32_t vendorID) { ++ switch (vendorID) { ++ case 0x1002: return "AMD Radeon Pro 5500M"; ++ case 0x8086: return "Intel UHD Graphics 630"; ++ default: return nullptr; ++ } ++} ++ ++// DARLING_METAL_GPU_NAME wins, then the Mac GPU name (when enabled and known), then the ++// cleaned Vulkan name. ++static inline std::string resolveName(const char* overrideName, const char* vulkanName, const char* macName = nullptr) { + if (overrideName && overrideName[0] != '\0') { + return overrideName; + } ++ if (macName && macName[0] != '\0') { ++ return macName; ++ } + return metalNameFromVulkanName(vulkanName); +} + diff --git a/tests/test_exec_guest.py b/tests/test_exec_guest.py new file mode 100644 index 0000000..9582245 --- /dev/null +++ b/tests/test_exec_guest.py @@ -0,0 +1,90 @@ +"""exec-guest.py: inherited descriptors and the opt-in host environment filter.""" +import importlib.util +import json +import os +from pathlib import Path +import subprocess +import sys +import unittest + +ROOT = Path(__file__).resolve().parents[1] +HELPER = ROOT / 'tools/exec-guest.py' +spec = importlib.util.spec_from_file_location('exec_guest', HELPER) +exec_guest = importlib.util.module_from_spec(spec) +spec.loader.exec_module(exec_guest) + +# What the guest and the host libraries Darling loads still need. +REQUIRED = { + 'DISPLAY': ':0', 'XAUTHORITY': '/run/user/1000/.mutter-Xwaylandauth.A', 'XCURSOR_SIZE': '24', + 'XDG_RUNTIME_DIR': '/run/user/1000', 'XDG_DATA_DIRS': '/usr/share', 'XDG_CONFIG_HOME': '/home/u/.config', + 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'PULSE_SERVER': 'unix:/run/pulse', + 'PIPEWIRE_LATENCY': '128/48000', 'ALSA_CONFIG_PATH': '/usr/share/alsa/alsa.conf', + 'DRI_PRIME': '1', 'MESA_NO_ERROR': '1', 'mesa_glthread': 'false', '__GL_THREADED_OPTIMIZATIONS': '0', + '__NV_PRIME_RENDER_OFFLOAD': '1', '__EGL_VENDOR_LIBRARY_FILENAMES': '/usr/share/glvnd/egl_vendor.d/10_nvidia.json', + 'VK_ICD_FILENAMES': '/usr/share/vulkan/icd.d/nvidia_icd.json', 'LIBGL_ALWAYS_SOFTWARE': '0', + 'LD_PRELOAD': 'libgamemodeauto.so.0', 'LD_LIBRARY_PATH': '/run/opengl-driver/lib', + 'HOME': '/home/u', 'USER': 'u', 'PATH': '/usr/bin:/bin', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', + 'DPREFIX': '/home/u/.darling', 'RCLOL_REFRESH_HZ': '144', 'DYLD_LIBRARY_PATH': '/Users/u/RuntimeLibraries', +} +SESSION = { + 'WAYLAND_DISPLAY': 'wayland-0', 'XDG_CURRENT_DESKTOP': 'GNOME', 'XDG_SESSION_TYPE': 'wayland', + 'XDG_SESSION_ID': '2', 'XDG_SEAT': 'seat0', 'DESKTOP_SESSION': 'gnome', 'GDMSESSION': 'gnome', + 'GDM_LANG': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/host:@/tmp/.ICE-unix/1', 'GNOME_SETUP_DISPLAY': ':1', + 'KDE_FULL_SESSION': 'true', 'SSH_AUTH_SOCK': '/run/user/1000/ssh', 'SYSTEMD_EXEC_PID': '1', + 'INVOCATION_ID': 'abc', 'JOURNAL_STREAM': '8:1', 'MANAGERPID': '1', 'NOTIFY_SOCKET': '/run/notify', + 'LISTEN_FDS': '1', 'MEMORY_PRESSURE_WATCH': '/sys/fs/cgroup', 'QT_QPA_PLATFORM': 'wayland', + 'GTK_MODULES': 'canberra', 'GDK_BACKEND': 'wayland', 'GIO_LAUNCHED_DESKTOP_FILE': '/x.desktop', + 'FLATPAK_ID': 'x', 'SNAP': '/snap/x', 'APPIMAGE': '/x.AppImage', 'VIRTUAL_ENV': '/home/u/.venv', + 'COLORTERM': 'truecolor', 'VTE_VERSION': '7600', 'WINDOWID': '1', 'TERM_PROGRAM': 'vscode', + 'LS_COLORS': 'di=01;34', 'SHLVL': '2', 'BASH_FUNC_f%%': '() { :; }', +} + + +class FilterTests(unittest.TestCase): + def test_strips_session_metadata_and_keeps_runtime_variables(self): + environ = {**REQUIRED, **SESSION, 'SHELL': '/bin/fish'} + removed = exec_guest.filter_host_environment(environ) + self.assertEqual(set(removed), set(SESSION)) + self.assertEqual(environ, {**REQUIRED, 'SHELL': '/bin/zsh'}) + + def test_keep_list(self): + environ = {'WAYLAND_DISPLAY': 'wayland-0', 'SSH_AUTH_SOCK': '/s', 'SHELL': '/bin/fish', + 'RCLOL_KEEP_HOST_ENV': 'WAYLAND_DISPLAY, SHELL'} + self.assertEqual(exec_guest.filter_host_environment(environ), ['SSH_AUTH_SOCK']) + self.assertEqual(environ['WAYLAND_DISPLAY'], 'wayland-0') + self.assertEqual(environ['SHELL'], '/bin/fish') + + +class ExecTests(unittest.TestCase): + DUMP = ('import json, os; print(json.dumps({"env": dict(os.environ), "fds": sorted(' + 'fd for fd in map(int, os.listdir("/proc/self/fd")) if fd < 3 or os.path.exists("/proc/self/fd/%d" % fd))}))') + + def run_helper(self, environ): + leaked = list(os.pipe()) + leaked.append(os.dup2(leaked[0], 41, inheritable=False)) + try: + result = subprocess.run([sys.executable, str(HELPER), sys.executable, '-c', self.DUMP], + env=environ, pass_fds=leaked, capture_output=True, text=True, check=True) + finally: + for fd in leaked: + os.close(fd) + return json.loads(result.stdout) + + def test_closes_descriptors_and_keeps_environment_by_default(self): + environ = {**REQUIRED, **SESSION, 'SHELL': '/bin/fish'} + child = self.run_helper(environ) + self.assertEqual(child['fds'], [0, 1, 2]) + for name, value in environ.items(): + self.assertEqual(child['env'].get(name), value, name) + + def test_opt_in_filter(self): + child = self.run_helper({**REQUIRED, **SESSION, 'SHELL': '/bin/fish', 'RCLOL_FILTER_HOST_ENV': '1'}) + self.assertEqual(child['fds'], [0, 1, 2]) + self.assertFalse(set(SESSION) & set(child['env'])) + for name, value in REQUIRED.items(): + self.assertEqual(child['env'].get(name), value, name) + self.assertEqual(child['env']['SHELL'], '/bin/zsh') + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_metal_device_info.py b/tests/test_metal_device_info.py new file mode 100644 index 0000000..99e9b52 --- /dev/null +++ b/tests/test_metal_device_info.py @@ -0,0 +1,47 @@ +"""Compile tools/test-metal-device-info.cpp against MTLDeviceInfo.h as shipped in patches/metal.patch.""" +from pathlib import Path +import re +import shutil +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] +HEADER = 'private-include/Metal/MTLDeviceInfo.h' + + +def header_from_patch(): + """MTLDeviceInfo.h is a new file in the patch, so its text is the section's added lines.""" + patch = (ROOT / 'patches/metal.patch').read_text() + section = re.search(r'(?ms)^diff --git a/%s b/%s\n.*?^@@ [^\n]*\n(.*?)(?=^diff --git |\Z)' % (HEADER, HEADER), patch) + if not section: + raise AssertionError(HEADER + ' is not a new file in patches/metal.patch') + lines = section.group(1).splitlines(keepends=True) + if not all(line.startswith('+') for line in lines): + raise AssertionError(HEADER + ' section has context or removed lines') + return ''.join(line[1:] for line in lines) + + +class MetalDeviceInfoTests(unittest.TestCase): + def test_host_check_passes(self): + compiler = shutil.which('c++') + if not compiler: + self.skipTest('no C++ compiler') + with tempfile.TemporaryDirectory() as folder: + include = Path(folder) / 'include/Metal' + include.mkdir(parents=True) + (include / 'MTLDeviceInfo.h').write_text(header_from_patch()) + binary = Path(folder) / 'test-metal-device-info' + build = subprocess.run([compiler, '-std=c++17', '-Wall', '-Werror', '-I', str(include.parent), + str(ROOT / 'tools/test-metal-device-info.cpp'), '-o', str(binary)], + capture_output=True, text=True) + self.assertEqual(build.returncode, 0, build.stderr) + result = subprocess.run([str(binary)], capture_output=True, text=True, timeout=30) + self.assertEqual(result.returncode, 0, result.stdout) + self.assertIn('METAL_DEVICE_INFO failures=0 pass=1', result.stdout) + self.assertIn('PASS registry ID is non-zero', result.stdout) + self.assertIn('PASS Mac GPU name', result.stdout) + + +if __name__ == '__main__': + unittest.main() diff --git a/tools/exec-guest.py b/tools/exec-guest.py new file mode 100755 index 0000000..331f754 --- /dev/null +++ b/tools/exec-guest.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Run a command without the host file descriptors this launcher inherited (anything above stderr). + +With RCLOL_FILTER_HOST_ENV=1 it also drops host desktop-session variables first (see +host_session_variable). RCLOL_KEEP_HOST_ENV lists names to keep anyway, separated by spaces or commas. +""" +import os +import sys + +# Desktop-session, service-manager, terminal and packaging metadata that nothing in the guest reads. +# Everything Darling and the host libraries it loads need is kept: DISPLAY, XAUTHORITY, XCURSOR_*, +# XDG_RUNTIME_DIR, XDG_DATA_DIRS and the other XDG base directories (the Vulkan loader searches +# them), DBUS_SESSION_BUS_ADDRESS, PULSE_*, PIPEWIRE_*, ALSA_*, DRI_PRIME, MESA_*, mesa_glthread, +# __GL_*, __NV_*, __EGL_*, VK_*, LIBGL_*, LD_PRELOAD (gamemoderun) and LD_LIBRARY_PATH. +SESSION_PREFIXES = ( + 'XDG_SESSION_', 'XDG_SEAT', 'XDG_VTNR', 'XDG_CURRENT_DESKTOP', 'XDG_MENU_PREFIX', + 'GNOME_', 'KDE_', 'GDM', 'SSH_', 'SYSTEMD_', 'MEMORY_PRESSURE_', 'QT_', 'GTK_', 'GDK_', + 'GIO_', 'GSETTINGS_', 'GJS_', 'FLATPAK_', 'SNAP_', 'KONSOLE_', 'TERM_PROGRAM', 'BASH_FUNC_', +) +SESSION_NAMES = { + 'WAYLAND_DISPLAY', 'DESKTOP_SESSION', 'DESKTOP_STARTUP_ID', 'SESSION_MANAGER', + 'INVOCATION_ID', 'JOURNAL_STREAM', 'MANAGERPID', 'MANAGERPIDFDID', 'NOTIFY_SOCKET', + 'LISTEN_FDS', 'LISTEN_PID', 'LISTEN_FDNAMES', 'COLORTERM', 'VTE_VERSION', 'WINDOWID', + 'WINDOWPATH', 'CHROME_DESKTOP', 'NO_AT_BRIDGE', 'MOTD_SHOWN', 'SNAP', 'APPIMAGE', 'APPDIR', + 'OWD', 'VIRTUAL_ENV', 'DEBUGINFOD_URLS', 'LS_COLORS', 'COLORFGBG', 'OLDPWD', 'SHLVL', +} +GUEST_SHELL = '/bin/zsh' + + +def host_session_variable(name): + return name in SESSION_NAMES or name.startswith(SESSION_PREFIXES) + + +def filter_host_environment(environ): + """Return the names removed from environ (a mutable mapping), honoring RCLOL_KEEP_HOST_ENV.""" + keep = set(environ.get('RCLOL_KEEP_HOST_ENV', '').replace(',', ' ').split()) + removed = sorted(name for name in environ if host_session_variable(name) and name not in keep) + for name in removed: + del environ[name] + # The host login shell (for example /bin/fish) does not exist in the guest. + if 'SHELL' not in keep: + environ['SHELL'] = GUEST_SHELL + return removed + + +def close_inherited_fds(): + """A terminal, IDE or service manager can leak descriptors; the guest must not inherit them.""" + try: + names = os.listdir('/proc/self/fd') + except OSError: + os.closerange(3, 65536) + return + for name in names: + if name.isdigit() and int(name) > 2: + try: + os.close(int(name)) + except OSError: + pass # the descriptor listdir used is already closed + + +def main(argv): + if len(argv) < 2: + sys.exit('usage: exec-guest.py COMMAND [ARGUMENT]...') + if os.environ.get('RCLOL_FILTER_HOST_ENV') == '1': + filter_host_environment(os.environ) + close_inherited_fds() + os.execvp(argv[1], argv[1:]) + + +if __name__ == '__main__': + main(sys.argv) diff --git a/tools/riot-launcher-offline-check.py b/tools/riot-launcher-offline-check.py index 1254b7a..0280d4e 100644 --- a/tools/riot-launcher-offline-check.py +++ b/tools/riot-launcher-offline-check.py @@ -16,15 +16,28 @@ def check(): folder = Path(directory) launcher = folder / "launch.sh" launcher.write_text((ROOT / "tools/run-riot-client.sh").read_text().replace("@GUEST_HOME@", "/Users/offline-check")) + (folder / "exec-guest.py").write_text((ROOT / "tools/exec-guest.py").read_text()) binary_folder = folder / "bin" binary_folder.mkdir() log = folder / "calls.jsonl" stub = f'''#!{sys.executable} import json, os, sys record = {{"command": os.path.basename(sys.argv[0]), "args": sys.argv[1:]}} +# The launcher starts the guest through exec-guest.py; run it for real. +if record["command"] == "python3" and sys.argv[1:] and os.path.basename(sys.argv[1]) == "exec-guest.py": + os.execv(sys.executable, [sys.executable] + sys.argv[1:]) if record["command"] == "darling": + # Descriptors open now were inherited; the one listdir used is closed again before this check. + def is_open(fd): + try: + os.fstat(fd) + return True + except OSError: + return False + record["fds"] = sorted(fd for fd in map(int, os.listdir("/proc/self/fd")) if is_open(fd)) record["flags"] = {{key: os.environ.get(key) for key in - ("RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT", "RCLOL_UX_ARG1", "RCLOL_APP_ARGS", "RCLOL_APP_ARG2", "DYLD_INSERT_LIBRARIES", "DYLD_LIBRARY_PATH", "DYLD_FRAMEWORK_PATH", "DARLING_XSHM_LIBRARY")}} + ("RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT", "RCLOL_UX_ARG1", "RCLOL_APP_ARGS", "RCLOL_APP_ARG2", "DYLD_INSERT_LIBRARIES", "DYLD_LIBRARY_PATH", "DYLD_FRAMEWORK_PATH", "DARLING_XSHM_LIBRARY", + "WAYLAND_DISPLAY", "XAUTHORITY", "DRI_PRIME", "MESA_NO_ERROR", "__GL_THREADED_OPTIMIZATIONS", "SHELL")}} with open({str(log)!r}, "a") as output: output.write(json.dumps(record) + "\\n") ''' @@ -60,7 +73,15 @@ with open({str(log)!r}, "a") as output: "RCLOL_OPEN_URL_BRIDGE": "1", "RCLOL_DISABLE_GPU_VSYNC": "1", }) - subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True) + # Leaked host descriptors, including multi-digit ones, must not reach the guest. + leaked = list(os.pipe()) + for target in (12, 200): + leaked.append(os.dup2(leaked[0], target, inheritable=False)) + try: + subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True, pass_fds=leaked) + finally: + for fd in leaked: + os.close(fd) records = [json.loads(line) for line in log.read_text().splitlines()] maintenance = [r for r in records if r["command"] == "python3"] assert [Path(r["args"][0]).name for r in maintenance] == [ @@ -70,6 +91,7 @@ with open({str(log)!r}, "a") as output: launches = [r for r in records if r["command"] == "darling"] assert len(launches) == 1, "unexpected launch count" launch = launches[0] + assert launch["fds"] == [0, 1, 2], f"inherited descriptors reached the guest: {launch['fds']}" assert all(launch["flags"][key] is None for key in ( "RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT" )), "inherited inspector/unlimited-frame argument survived" @@ -95,7 +117,17 @@ with open({str(log)!r}, "a") as output: subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True) clean = json.loads(log.read_text().splitlines()[-1]) assert clean["flags"]["DYLD_INSERT_LIBRARIES"] is None - print("LAUNCHER_OFFLINE pass=1 installed=PASS installer=PASS inherited_injection=absent runtime=preserved") + # The opt-in host environment filter drops session metadata but never the launcher's GPU tuning. + env.update({"RCLOL_FILTER_HOST_ENV": "1", "WAYLAND_DISPLAY": "wayland-0", "XAUTHORITY": "/tmp/xauth", + "DRI_PRIME": "1", "SHELL": "/bin/fish"}) + env.pop("MESA_NO_ERROR", None) + env.pop("__GL_THREADED_OPTIMIZATIONS", None) + subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True) + filtered = json.loads(log.read_text().splitlines()[-1])["flags"] + assert filtered["WAYLAND_DISPLAY"] is None and filtered["SHELL"] == "/bin/zsh", "host session metadata survived" + assert (filtered["XAUTHORITY"], filtered["DRI_PRIME"]) == ("/tmp/xauth", "1"), "X11/GPU selection lost" + assert (filtered["MESA_NO_ERROR"], filtered["__GL_THREADED_OPTIMIZATIONS"]) == ("1", "0"), "launcher GPU tuning lost" + print("LAUNCHER_OFFLINE pass=1 installed=PASS installer=PASS inherited_injection=absent runtime=preserved inherited_fds=closed host_env_filter=PASS") if __name__ == "__main__": diff --git a/tools/run-riot-client.sh b/tools/run-riot-client.sh index 6889e0b..5fa75a5 100755 --- a/tools/run-riot-client.sh +++ b/tools/run-riot-client.sh @@ -29,7 +29,9 @@ unset RCLOL_APP_ARG3 RCLOL_APP_ARG3_SLOT RCLOL_UX_ARG1 RCLOL_UX_ARG1_SLOT unset RCLOL_UX_ARG2 RCLOL_UX_ARG2_SLOT # The paired runtime presents completed GPU textures without CPU readback. # Explicit 0 or an empty value selects the CPU fallback for diagnostics. -for option in RCLOL_GPU_DIRECT_PRESENT RCLOL_GPU_DIRECT_ASYNC RCLOL_GPU_DIRECT_VSYNC DARLING_CA_PRESENT_AT_COMMIT DARLING_CA_SKIP_LOCAL_IOSURFACE_RENDER RCLOL_FORWARD_MOUSE_MOVED RCLOL_MOUSE_CONFINEMENT; do +# DARLING_METAL_MAC_GPU_NAME reports the MacBookPro16,1 GPU of the host vendor (AMD or Intel) as +# MTLDevice.name, matching the model IOKit reports; memory figures stay the host's. +for option in RCLOL_GPU_DIRECT_PRESENT RCLOL_GPU_DIRECT_ASYNC RCLOL_GPU_DIRECT_VSYNC DARLING_CA_PRESENT_AT_COMMIT DARLING_CA_SKIP_LOCAL_IOSURFACE_RENDER RCLOL_FORWARD_MOUSE_MOVED RCLOL_MOUSE_CONFINEMENT DARLING_METAL_MAC_GPU_NAME; do eval 'value=${'"$option"'-1}' case "$value" in 0|'') unset "$option";; *) export "$option=$value";; esac done @@ -104,7 +106,9 @@ fi # On affected setups Chromium's default temp directory is unusable. Electron # can copy app.asar entries to temporary files for async reads and report a # failed copy as "not found". Use a writable guest directory by default. -exec env -u ELECTRON_RUN_AS_NODE -u DYLD_INSERT_LIBRARIES EGL_PLATFORM=x11 \ +# exec-guest.py closes host file descriptors above stderr before starting the guest. A shell +# loop cannot: dash only accepts single-digit descriptors in redirections. +exec python3 "$runner_dir/exec-guest.py" env -u ELECTRON_RUN_AS_NODE -u DYLD_INSERT_LIBRARIES EGL_PLATFORM=x11 \ MAC_CHROMIUM_TMPDIR="${MAC_CHROMIUM_TMPDIR:-/private/tmp}" \ LANG="${RCLOL_GUEST_LOCALE:-en_US.UTF-8}" LC_ALL="${RCLOL_GUEST_LOCALE:-en_US.UTF-8}" \ mesa_glthread="${RCLOL_MESA_GLTHREAD-${mesa_glthread-false}}" \ diff --git a/tools/test-metal-device-info.cpp b/tools/test-metal-device-info.cpp index 926dd75..45cd492 100644 --- a/tools/test-metal-device-info.cpp +++ b/tools/test-metal-device-info.cpp @@ -49,6 +49,13 @@ int main() { expectName("", MTLDeviceInfo::kFallbackName); expectName(nullptr, MTLDeviceInfo::kFallbackName); + // -[MTLDevice registryID]: never 0, like a real Mac. + { + bool ok = MTLDeviceInfo::kDefaultRegistryID != 0; + failures += !ok; + std::printf("%s registry ID is non-zero\n", ok ? "PASS" : "FAIL"); + } + // DARLING_METAL_GPU_NAME override. { bool ok = MTLDeviceInfo::resolveName("My GPU", "AMD Radeon RX 6800 XT (RADV NAVI21)") == "My GPU" @@ -58,6 +65,19 @@ int main() { std::printf("%s name override\n", ok ? "PASS" : "FAIL"); } + // DARLING_METAL_MAC_GPU_NAME: MacBookPro16,1 GPUs for AMD and Intel hosts; NVIDIA keeps its name. + { + using MTLDeviceInfo::macGpuName; + bool ok = std::string(macGpuName(0x1002)) == "AMD Radeon Pro 5500M" + && std::string(macGpuName(0x8086)) == "Intel UHD Graphics 630" + && macGpuName(0x10de) == nullptr + && MTLDeviceInfo::resolveName(nullptr, "AMD Radeon RX 6600 (RADV NAVI23)", macGpuName(0x1002)) == "AMD Radeon Pro 5500M" + && MTLDeviceInfo::resolveName(nullptr, "NVIDIA GeForce GTX 1060 6GB", macGpuName(0x10de)) == "NVIDIA GeForce GTX 1060 6GB" + && MTLDeviceInfo::resolveName("My GPU", "AMD Radeon RX 6600 (RADV NAVI23)", macGpuName(0x1002)) == "My GPU"; + failures += !ok; + std::printf("%s Mac GPU name\n", ok ? "PASS" : "FAIL"); + } + // DARLING_METAL_VRAM_MB parsing: positive decimal integer only. const unsigned long long MiB = 1024ull * 1024ull; expectBytes("vram \"8192\"", MTLDeviceInfo::parseMegabytes("8192"), 8192 * MiB);