Merge remote-tracking branch 'origin/main' into pr61/metal-device-fixes

# Conflicts:
#	.github/workflows/source-check.yml

.github/workflows/source-check.yml

@@ -12,9 +12,11 @@ jobs:
           python-version: '3.12'
       - run: python scripts/validate.py
       - run: python tools/riot-input-offline-check.py
+      - run: python tools/riot-launcher-offline-check.py
       - run: python tools/riot-runtime-cleanup-offline-check.py
       - run: python tools/riot-cdn-dns-offline-check.py
       - run: python -m unittest discover -s tests -p 'test_hidden_riot_recovery.py'
       - run: python -m unittest discover -s tests -p 'test_desktop_run.py'
       - run: python -m unittest discover -s tests -p 'test_urlsession_runner.py'
       - run: python -m unittest discover -s tests -p 'test_metal_device_info.py'
+      - run: python -m unittest discover -s tests -p 'test_exec_guest.py'

docs/SETUP.md

@@ -73,6 +73,17 @@ opts in to Hyprland-only recovery of a default-prefix Riot session with no
 visible window; it is off by default. See [troubleshooting](TROUBLESHOOTING.md#riot-exits-and-league-reports-that-it-lost-its-connection)
 for the observed failures, recovery guards and NSURLSession limitations.
 
+`RCLOL_FILTER_HOST_ENV=1` opts in to removing host desktop-session variables before
+the guest starts: `WAYLAND_DISPLAY`, `XDG_SESSION_*`, `XDG_CURRENT_DESKTOP`,
+GNOME/KDE/GDM, systemd, SSH agent, terminal, toolkit (`QT_*`, `GTK_*`, `GDK_*`),
+Flatpak/Snap/AppImage and exported shell functions, and `SHELL` becomes `/bin/zsh`.
+It is off by default. Variables that Darling or the host graphics and audio libraries read
+are always kept: `DISPLAY`, `XAUTHORITY`, `XDG_RUNTIME_DIR` and the XDG base directories,
+D-Bus, PulseAudio/PipeWire/ALSA, `DRI_PRIME`, `MESA_*`, `__GL_*`, `__NV_*`, `VK_*`,
+`LD_PRELOAD` and `LD_LIBRARY_PATH`. List names to keep anyway in `RCLOL_KEEP_HOST_ENV`,
+for example `RCLOL_KEEP_HOST_ENV=WAYLAND_DISPLAY`. The launcher always closes inherited
+file descriptors above stderr before the guest starts.
+
 ## Updating an existing runtime
 
 `./scripts/update.sh` updates everything after `git pull`: `build.sh`, then the base install, then `install-runtime.sh`. When a pull changes a patch, `prepare.py` finds the earlier version applied in that component (its copy in `build/applied-patches`, or a version from this repository's history), reverts it, and applies the new one. It never resets a repository; if neither version matches, it stops and names the repository with other edits. The steps below update single components instead.

docs/TESTING.md

@@ -27,7 +27,7 @@ Expected: `METAL_DEVICE_INFO failures=0 pass=1`. It covers how `MTLDevice.name`
 
 The LLVM version check compiles Iridium's actual dynamic loader against small stub libraries. It verifies that a matching major initializes, older/newer majors produce a version mismatch, and a missing library fails initialization. It does not start Darling. The native AIR translator (`python3 tools/build-riot-air-check.py --native`) selects both headers and the shared library from the same `llvm-config`; its build prints those paths so shader results can be attributed to the tested LLVM installation.
 
-The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, and preserves Darling runtime library/framework paths. The memory-installer check imports the installer and exercises its policy (default source is the prepared artifact; the artifact is never a destination of a different library) in temporary directories. The Foundation-installer check does the same for the overlay installer: it seeds a temporary framework from a stub SDK, replaces `Versions/C/Foundation`, keeps the standard symlinks, stages a backup, treats a re-run as current and refuses a non-Mach-O source. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads.
+The patch check uses temporary base indexes and does not reset or modify the source checkout. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, preserves Darling runtime library/framework paths, and checks that host file descriptors the launcher inherited (including multi-digit ones) are closed before the guest starts. The memory-installer check imports the installer and exercises its policy (default source is the prepared artifact; the artifact is never a destination of a different library) in temporary directories. The Foundation-installer check does the same for the overlay installer: it seeds a temporary framework from a stub SDK, replaces `Versions/C/Foundation`, keeps the standard symlinks, stages a backup, treats a re-run as current and refuses a non-Mach-O source. The installer handoff mock covers 28 scenarios, including HTTP 409, interrupted verification and preserving active downloads.
 
 ## NSURLSession lifecycle regression
 
@@ -181,7 +181,7 @@ build/riot-host-receive-check
 python3 tools/riot-launcher-offline-check.py
 ```
 
-Expected: `warm_empty_allocations=0 single_packet_allocations=2`, correct credentials, descriptor ownership/CLOEXEC, sender addresses, multiple batches, truncation flags, callback reentrancy and exception cleanup. The old receive implementation makes 32 allocations for a warmed empty receive and 64 for a single-packet receive in the same fixture. These are allocation counts, not an FPS benchmark. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, and preserves Darling runtime library/framework paths.
+Expected: `warm_empty_allocations=0 single_packet_allocations=2`, correct credentials, descriptor ownership/CLOEXEC, sender addresses, multiple batches, truncation flags, callback reentrancy and exception cleanup. The old receive implementation makes 32 allocations for a warmed empty receive and 64 for a single-packet receive in the same fixture. These are allocation counts, not an FPS benchmark. The launcher check uses stub commands and never starts or contacts Riot or Darling. It checks installed-client and first-time-installer launches, strips inherited `DYLD_INSERT_LIBRARIES` even with the old VPX opt-in set, preserves Darling runtime library/framework paths, and checks that host file descriptors the launcher inherited (including multi-digit ones) are closed before the guest starts.
 
 The same source passed AddressSanitizer/UndefinedBehaviorSanitizer and leak detection. Allocation counts depend on this fixture/compiler/library; preserve the no-wasted-slot property rather than interpreting them as game performance. See [setup](SETUP.md#updating-an-existing-runtime) for server build/install/activation.
 

tests/test_exec_guest.py

@@ -0,0 +1,90 @@
+"""exec-guest.py: inherited descriptors and the opt-in host environment filter."""
+import importlib.util
+import json
+import os
+from pathlib import Path
+import subprocess
+import sys
+import unittest
+
+ROOT = Path(__file__).resolve().parents[1]
+HELPER = ROOT / 'tools/exec-guest.py'
+spec = importlib.util.spec_from_file_location('exec_guest', HELPER)
+exec_guest = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(exec_guest)
+
+# What the guest and the host libraries Darling loads still need.
+REQUIRED = {
+    'DISPLAY': ':0', 'XAUTHORITY': '/run/user/1000/.mutter-Xwaylandauth.A', 'XCURSOR_SIZE': '24',
+    'XDG_RUNTIME_DIR': '/run/user/1000', 'XDG_DATA_DIRS': '/usr/share', 'XDG_CONFIG_HOME': '/home/u/.config',
+    'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'PULSE_SERVER': 'unix:/run/pulse',
+    'PIPEWIRE_LATENCY': '128/48000', 'ALSA_CONFIG_PATH': '/usr/share/alsa/alsa.conf',
+    'DRI_PRIME': '1', 'MESA_NO_ERROR': '1', 'mesa_glthread': 'false', '__GL_THREADED_OPTIMIZATIONS': '0',
+    '__NV_PRIME_RENDER_OFFLOAD': '1', '__EGL_VENDOR_LIBRARY_FILENAMES': '/usr/share/glvnd/egl_vendor.d/10_nvidia.json',
+    'VK_ICD_FILENAMES': '/usr/share/vulkan/icd.d/nvidia_icd.json', 'LIBGL_ALWAYS_SOFTWARE': '0',
+    'LD_PRELOAD': 'libgamemodeauto.so.0', 'LD_LIBRARY_PATH': '/run/opengl-driver/lib',
+    'HOME': '/home/u', 'USER': 'u', 'PATH': '/usr/bin:/bin', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm',
+    'DPREFIX': '/home/u/.darling', 'RCLOL_REFRESH_HZ': '144', 'DYLD_LIBRARY_PATH': '/Users/u/RuntimeLibraries',
+}
+SESSION = {
+    'WAYLAND_DISPLAY': 'wayland-0', 'XDG_CURRENT_DESKTOP': 'GNOME', 'XDG_SESSION_TYPE': 'wayland',
+    'XDG_SESSION_ID': '2', 'XDG_SEAT': 'seat0', 'DESKTOP_SESSION': 'gnome', 'GDMSESSION': 'gnome',
+    'GDM_LANG': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/host:@/tmp/.ICE-unix/1', 'GNOME_SETUP_DISPLAY': ':1',
+    'KDE_FULL_SESSION': 'true', 'SSH_AUTH_SOCK': '/run/user/1000/ssh', 'SYSTEMD_EXEC_PID': '1',
+    'INVOCATION_ID': 'abc', 'JOURNAL_STREAM': '8:1', 'MANAGERPID': '1', 'NOTIFY_SOCKET': '/run/notify',
+    'LISTEN_FDS': '1', 'MEMORY_PRESSURE_WATCH': '/sys/fs/cgroup', 'QT_QPA_PLATFORM': 'wayland',
+    'GTK_MODULES': 'canberra', 'GDK_BACKEND': 'wayland', 'GIO_LAUNCHED_DESKTOP_FILE': '/x.desktop',
+    'FLATPAK_ID': 'x', 'SNAP': '/snap/x', 'APPIMAGE': '/x.AppImage', 'VIRTUAL_ENV': '/home/u/.venv',
+    'COLORTERM': 'truecolor', 'VTE_VERSION': '7600', 'WINDOWID': '1', 'TERM_PROGRAM': 'vscode',
+    'LS_COLORS': 'di=01;34', 'SHLVL': '2', 'BASH_FUNC_f%%': '() { :; }',
+}
+
+
+class FilterTests(unittest.TestCase):
+    def test_strips_session_metadata_and_keeps_runtime_variables(self):
+        environ = {**REQUIRED, **SESSION, 'SHELL': '/bin/fish'}
+        removed = exec_guest.filter_host_environment(environ)
+        self.assertEqual(set(removed), set(SESSION))
+        self.assertEqual(environ, {**REQUIRED, 'SHELL': '/bin/zsh'})
+
+    def test_keep_list(self):
+        environ = {'WAYLAND_DISPLAY': 'wayland-0', 'SSH_AUTH_SOCK': '/s', 'SHELL': '/bin/fish',
+                   'RCLOL_KEEP_HOST_ENV': 'WAYLAND_DISPLAY, SHELL'}
+        self.assertEqual(exec_guest.filter_host_environment(environ), ['SSH_AUTH_SOCK'])
+        self.assertEqual(environ['WAYLAND_DISPLAY'], 'wayland-0')
+        self.assertEqual(environ['SHELL'], '/bin/fish')
+
+
+class ExecTests(unittest.TestCase):
+    DUMP = ('import json, os; print(json.dumps({"env": dict(os.environ), "fds": sorted('
+            'fd for fd in map(int, os.listdir("/proc/self/fd")) if fd < 3 or os.path.exists("/proc/self/fd/%d" % fd))}))')
+
+    def run_helper(self, environ):
+        leaked = list(os.pipe())
+        leaked.append(os.dup2(leaked[0], 41, inheritable=False))
+        try:
+            result = subprocess.run([sys.executable, str(HELPER), sys.executable, '-c', self.DUMP],
+                                    env=environ, pass_fds=leaked, capture_output=True, text=True, check=True)
+        finally:
+            for fd in leaked:
+                os.close(fd)
+        return json.loads(result.stdout)
+
+    def test_closes_descriptors_and_keeps_environment_by_default(self):
+        environ = {**REQUIRED, **SESSION, 'SHELL': '/bin/fish'}
+        child = self.run_helper(environ)
+        self.assertEqual(child['fds'], [0, 1, 2])
+        for name, value in environ.items():
+            self.assertEqual(child['env'].get(name), value, name)
+
+    def test_opt_in_filter(self):
+        child = self.run_helper({**REQUIRED, **SESSION, 'SHELL': '/bin/fish', 'RCLOL_FILTER_HOST_ENV': '1'})
+        self.assertEqual(child['fds'], [0, 1, 2])
+        self.assertFalse(set(SESSION) & set(child['env']))
+        for name, value in REQUIRED.items():
+            self.assertEqual(child['env'].get(name), value, name)
+        self.assertEqual(child['env']['SHELL'], '/bin/zsh')
+
+
+if __name__ == '__main__':
+    unittest.main()

tools/exec-guest.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""Run a command without the host file descriptors this launcher inherited (anything above stderr).
+
+With RCLOL_FILTER_HOST_ENV=1 it also drops host desktop-session variables first (see
+host_session_variable). RCLOL_KEEP_HOST_ENV lists names to keep anyway, separated by spaces or commas.
+"""
+import os
+import sys
+
+# Desktop-session, service-manager, terminal and packaging metadata that nothing in the guest reads.
+# Everything Darling and the host libraries it loads need is kept: DISPLAY, XAUTHORITY, XCURSOR_*,
+# XDG_RUNTIME_DIR, XDG_DATA_DIRS and the other XDG base directories (the Vulkan loader searches
+# them), DBUS_SESSION_BUS_ADDRESS, PULSE_*, PIPEWIRE_*, ALSA_*, DRI_PRIME, MESA_*, mesa_glthread,
+# __GL_*, __NV_*, __EGL_*, VK_*, LIBGL_*, LD_PRELOAD (gamemoderun) and LD_LIBRARY_PATH.
+SESSION_PREFIXES = (
+    'XDG_SESSION_', 'XDG_SEAT', 'XDG_VTNR', 'XDG_CURRENT_DESKTOP', 'XDG_MENU_PREFIX',
+    'GNOME_', 'KDE_', 'GDM', 'SSH_', 'SYSTEMD_', 'MEMORY_PRESSURE_', 'QT_', 'GTK_', 'GDK_',
+    'GIO_', 'GSETTINGS_', 'GJS_', 'FLATPAK_', 'SNAP_', 'KONSOLE_', 'TERM_PROGRAM', 'BASH_FUNC_',
+)
+SESSION_NAMES = {
+    'WAYLAND_DISPLAY', 'DESKTOP_SESSION', 'DESKTOP_STARTUP_ID', 'SESSION_MANAGER',
+    'INVOCATION_ID', 'JOURNAL_STREAM', 'MANAGERPID', 'MANAGERPIDFDID', 'NOTIFY_SOCKET',
+    'LISTEN_FDS', 'LISTEN_PID', 'LISTEN_FDNAMES', 'COLORTERM', 'VTE_VERSION', 'WINDOWID',
+    'WINDOWPATH', 'CHROME_DESKTOP', 'NO_AT_BRIDGE', 'MOTD_SHOWN', 'SNAP', 'APPIMAGE', 'APPDIR',
+    'OWD', 'VIRTUAL_ENV', 'DEBUGINFOD_URLS', 'LS_COLORS', 'COLORFGBG', 'OLDPWD', 'SHLVL',
+}
+GUEST_SHELL = '/bin/zsh'
+
+
+def host_session_variable(name):
+    return name in SESSION_NAMES or name.startswith(SESSION_PREFIXES)
+
+
+def filter_host_environment(environ):
+    """Return the names removed from environ (a mutable mapping), honoring RCLOL_KEEP_HOST_ENV."""
+    keep = set(environ.get('RCLOL_KEEP_HOST_ENV', '').replace(',', ' ').split())
+    removed = sorted(name for name in environ if host_session_variable(name) and name not in keep)
+    for name in removed:
+        del environ[name]
+    # The host login shell (for example /bin/fish) does not exist in the guest.
+    if 'SHELL' not in keep:
+        environ['SHELL'] = GUEST_SHELL
+    return removed
+
+
+def close_inherited_fds():
+    """A terminal, IDE or service manager can leak descriptors; the guest must not inherit them."""
+    try:
+        names = os.listdir('/proc/self/fd')
+    except OSError:
+        os.closerange(3, 65536)
+        return
+    for name in names:
+        if name.isdigit() and int(name) > 2:
+            try:
+                os.close(int(name))
+            except OSError:
+                pass  # the descriptor listdir used is already closed
+
+
+def main(argv):
+    if len(argv) < 2:
+        sys.exit('usage: exec-guest.py COMMAND [ARGUMENT]...')
+    if os.environ.get('RCLOL_FILTER_HOST_ENV') == '1':
+        filter_host_environment(os.environ)
+    close_inherited_fds()
+    os.execvp(argv[1], argv[1:])
+
+
+if __name__ == '__main__':
+    main(sys.argv)

tools/riot-launcher-offline-check.py

@@ -16,15 +16,28 @@ def check():
         folder = Path(directory)
         launcher = folder / "launch.sh"
         launcher.write_text((ROOT / "tools/run-riot-client.sh").read_text().replace("@GUEST_HOME@", "/Users/offline-check"))
+        (folder / "exec-guest.py").write_text((ROOT / "tools/exec-guest.py").read_text())
         binary_folder = folder / "bin"
         binary_folder.mkdir()
         log = folder / "calls.jsonl"
         stub = f'''#!{sys.executable}
 import json, os, sys
 record = {{"command": os.path.basename(sys.argv[0]), "args": sys.argv[1:]}}
+# The launcher starts the guest through exec-guest.py; run it for real.
+if record["command"] == "python3" and sys.argv[1:] and os.path.basename(sys.argv[1]) == "exec-guest.py":
+    os.execv(sys.executable, [sys.executable] + sys.argv[1:])
 if record["command"] == "darling":
+    # Descriptors open now were inherited; the one listdir used is closed again before this check.
+    def is_open(fd):
+        try:
+            os.fstat(fd)
+            return True
+        except OSError:
+            return False
+    record["fds"] = sorted(fd for fd in map(int, os.listdir("/proc/self/fd")) if is_open(fd))
     record["flags"] = {{key: os.environ.get(key) for key in
-        ("RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT", "RCLOL_UX_ARG1", "RCLOL_APP_ARGS", "RCLOL_APP_ARG2", "DYLD_INSERT_LIBRARIES", "DYLD_LIBRARY_PATH", "DYLD_FRAMEWORK_PATH", "DARLING_XSHM_LIBRARY")}}
+        ("RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT", "RCLOL_UX_ARG1", "RCLOL_APP_ARGS", "RCLOL_APP_ARG2", "DYLD_INSERT_LIBRARIES", "DYLD_LIBRARY_PATH", "DYLD_FRAMEWORK_PATH", "DARLING_XSHM_LIBRARY",
+         "WAYLAND_DISPLAY", "XAUTHORITY", "DRI_PRIME", "MESA_NO_ERROR", "__GL_THREADED_OPTIMIZATIONS", "SHELL")}}
 with open({str(log)!r}, "a") as output:
     output.write(json.dumps(record) + "\\n")
 '''
@@ -60,7 +73,15 @@ with open({str(log)!r}, "a") as output:
             "RCLOL_OPEN_URL_BRIDGE": "1",
             "RCLOL_DISABLE_GPU_VSYNC": "1",
         })
-        subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True)
+        # Leaked host descriptors, including multi-digit ones, must not reach the guest.
+        leaked = list(os.pipe())
+        for target in (12, 200):
+            leaked.append(os.dup2(leaked[0], target, inheritable=False))
+        try:
+            subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True, pass_fds=leaked)
+        finally:
+            for fd in leaked:
+                os.close(fd)
         records = [json.loads(line) for line in log.read_text().splitlines()]
         maintenance = [r for r in records if r["command"] == "python3"]
         assert [Path(r["args"][0]).name for r in maintenance] == [
@@ -70,6 +91,7 @@ with open({str(log)!r}, "a") as output:
         launches = [r for r in records if r["command"] == "darling"]
         assert len(launches) == 1, "unexpected launch count"
         launch = launches[0]
+        assert launch["fds"] == [0, 1, 2], f"inherited descriptors reached the guest: {launch['fds']}"
         assert all(launch["flags"][key] is None for key in (
             "RCLOL_APP_ARG3", "RCLOL_APP_ARG3_SLOT", "RCLOL_UX_ARG2", "RCLOL_UX_ARG2_SLOT"
         )), "inherited inspector/unlimited-frame argument survived"
@@ -95,7 +117,17 @@ with open({str(log)!r}, "a") as output:
         subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True)
         clean = json.loads(log.read_text().splitlines()[-1])
         assert clean["flags"]["DYLD_INSERT_LIBRARIES"] is None
-        print("LAUNCHER_OFFLINE pass=1 installed=PASS installer=PASS inherited_injection=absent runtime=preserved")
+        # The opt-in host environment filter drops session metadata but never the launcher's GPU tuning.
+        env.update({"RCLOL_FILTER_HOST_ENV": "1", "WAYLAND_DISPLAY": "wayland-0", "XAUTHORITY": "/tmp/xauth",
+                    "DRI_PRIME": "1", "SHELL": "/bin/fish"})
+        env.pop("MESA_NO_ERROR", None)
+        env.pop("__GL_THREADED_OPTIMIZATIONS", None)
+        subprocess.run(["sh", str(launcher)], env=env, check=True, capture_output=True, text=True)
+        filtered = json.loads(log.read_text().splitlines()[-1])["flags"]
+        assert filtered["WAYLAND_DISPLAY"] is None and filtered["SHELL"] == "/bin/zsh", "host session metadata survived"
+        assert (filtered["XAUTHORITY"], filtered["DRI_PRIME"]) == ("/tmp/xauth", "1"), "X11/GPU selection lost"
+        assert (filtered["MESA_NO_ERROR"], filtered["__GL_THREADED_OPTIMIZATIONS"]) == ("1", "0"), "launcher GPU tuning lost"
+        print("LAUNCHER_OFFLINE pass=1 installed=PASS installer=PASS inherited_injection=absent runtime=preserved inherited_fds=closed host_env_filter=PASS")
 
 
 if __name__ == "__main__":

tools/run-riot-client.sh

@@ -106,7 +106,9 @@ fi
 # On affected setups Chromium's default temp directory is unusable. Electron
 # can copy app.asar entries to temporary files for async reads and report a
 # failed copy as "not found". Use a writable guest directory by default.
-exec env -u ELECTRON_RUN_AS_NODE -u DYLD_INSERT_LIBRARIES EGL_PLATFORM=x11 \
+# exec-guest.py closes host file descriptors above stderr before starting the guest. A shell
+# loop cannot: dash only accepts single-digit descriptors in redirections.
+exec python3 "$runner_dir/exec-guest.py" env -u ELECTRON_RUN_AS_NODE -u DYLD_INSERT_LIBRARIES EGL_PLATFORM=x11 \
   MAC_CHROMIUM_TMPDIR="${MAC_CHROMIUM_TMPDIR:-/private/tmp}" \
   LANG="${RCLOL_GUEST_LOCALE:-en_US.UTF-8}" LC_ALL="${RCLOL_GUEST_LOCALE:-en_US.UTF-8}" \
   mesa_glthread="${RCLOL_MESA_GLTHREAD-${mesa_glthread-false}}" \