feat(setup): check host versions before the build and the runtime install

setup failures reported in #137, #138 and #143 happened only after the long
fetch/build or at Riot startup. scripts/preflight.py now runs from
scripts/build.sh (before configure.py, so an old interpreter gets this message
and not a traceback) and from scripts/install-runtime.sh, and checks the host
once:

- python 3.10+, cmake 3.20+ (the bundled indium CMakeLists requires it),
  git-lfs 2.13+, kernel 5.14+ and LLVM 16+ exit nonzero with one actionable
  line each. RCLOL_SKIP_PREFLIGHT=1 bypasses the gate.
- llvm-objdump and llvm-nm are required, and unresolved names are explained.
- Vulkan 1.3 and timeline semaphores are required when vulkaninfo is
  installed; they only warn when it is not, because a missing tool does not
  mean a missing driver.
- the X display is probed with xdpyinfo in the environment Darling inherits.
  It only warns: a build does not need a display.
- a prepared build/darling-src is scanned for small .dylib/.a/.so/.o files
  that still contain Git LFS pointer text, which links but crashes the guest.
- --json prints the same report for the diagnostic tools.

The installers no longer call hashlib.file_digest (Python 3.11+), so Ubuntu
22.04's Python 3.10 works instead of failing after the overlays are built
(#137). A streaming tools/file_digest.py replaces it.

LLVM command line tools are resolved by tools/llvm_tools.py: an unversioned
name on PATH first, then the highest llvm-<tool>-<N>, then llvm-config
--bindir. Debian/Ubuntu install llvm-objdump and llvm-nm only under
/usr/lib/llvm-<N>/bin, which broke the runtime validation (#138). Six tools
now use it. The LLVM 16 floor also stops the LLVM 15 runtime abort from #139
before anything is built.

Two test helpers used APIs added after Python 3.10; they are shimmed so the
same suite runs on 3.10, 3.11 and 3.14.

Adds tests/test_preflight.py (24 tests, fake host tools only).
This commit is contained in:
MintMachineHQ committed 2026-10-25 13:58:25 -06:00
1 parent 24276b91c6
commit fa18b2d04f
19 files changed
+756 -31

No files matched your search

+1 -1
View File
@@ -41,7 +41,7 @@ Game files, Riot installers, account databases, tokens, local prefixes, captures
## Requirements
- x86-64 Linux **5.14 or newer**, an X11 session or Xwayland, and a working hardware Vulkan/OpenGL driver. AMD RADV and the NVIDIA driver (proprietary and open kernel modules) have been exercised, see [NVIDIA](docs/TROUBLESHOOTING.md#nvidia); other drivers are not validated. Hybrid laptops usually render on the integrated GPU unless started with PRIME offload. See [hybrid graphics](docs/TROUBLESHOOTING.md#hybrid-graphics-laptops-amdintel-igpu--nvidia).
- Git + Git LFS, Bash, curl, unzip, CMake, Make, Clang/LLD, LLVM development files, a C/C++ compiler, Python 3.12+ and the Darling build dependencies. The patch includes LLVM 22/23 branch-opcode compatibility; the new PC used LLVM/Clang 23. Node.js is needed only for the offline JavaScript checks.
- Git + Git LFS (2.13 or newer), Bash, curl, unzip, CMake 3.20+, Make, Clang/LLD, LLVM development files (16 or newer), a C/C++ compiler, Python 3.10+ and the Darling build dependencies. `scripts/build.sh` and `scripts/install-runtime.sh` run `scripts/preflight.py` first and stop on a version that cannot build or run the result; `RCLOL_SKIP_PREFLIGHT=1` bypasses that gate. The patch includes LLVM 22/23 branch-opcode compatibility, and LLVM 22 or 23 is what the Metal overlays are tested with. Node.js is needed only for the offline JavaScript checks.
- X11/XCB/SHM development files for the current runtime helpers.
- Enough free disk for Darling sources/build, Git LFS assets and the game: budget at least 50 GB. Building is substantially larger than this source-only repository.
+2
View File
@@ -55,6 +55,8 @@ Linux 5.14 or newer is required for `MADV_POPULATE_WRITE`, used when Darwin's al
The runtime installer requires native `pkg-config` entries for `x11`, `x11-xcb`, `xcb-shm` and `xext`, plus the base Darling SDK.
`scripts/build.sh` and `scripts/install-runtime.sh` start with `scripts/preflight.py`, which checks the host versions the build and the runtime need: Python, CMake, Git LFS, the kernel, LLVM and its command line tools, the Vulkan API level, and the X display. A required check that fails stops immediately with one actionable line; the display check only warns, because a build does not need one. Run it directly with `python3 scripts/preflight.py` (`--json` for a machine-readable report), and set `RCLOL_SKIP_PREFLIGHT=1` to bypass the gate for one run.
Run configuration, builds and the per-user runtime installation as your ordinary account. Use sudo only for the base CMake installation and the incremental server installer. Running the whole build or per-user install under sudo can create root-owned files and configure the wrong guest home. `scripts/install-runtime.sh` initializes the default user prefix and creates `~/.darling/private/tmp` with mode `1777`, so the installer can write guest temporary files. It then shuts down Darling before replacing the paired runtime: first exit the game and both clients manually.
The normal launcher starts neither an Electron inspector nor client API/UI helpers. It clears inherited `DYLD_INSERT_LIBRARIES`, retains Darling runtime library/framework overrides, supplies an absolute log directory to the launcher, and selects the League product. The log-directory/Chromium argument shim, empty-autofill hook and optional native VPX bridge are no longer injected or built by runtime installation. Existing copies on disk are inactive and may remain. Child-process log-directory rewriting and Chromium feature overrides previously supplied by the shim are removed; client startup without those workarounds needs manual validation. The first-time installer uses `--skip-to-install` and omits the installed client's `--app-root`, avoiding an immediate exit or inaccessible intro screen. The project ships no inspector hooks, client API helpers, input automation or client binary patches. Keep account data, logs and captures private.
+23
View File
@@ -20,6 +20,29 @@ never delete a populated Riot/game prefix just to repair the launcher.
Use the [setup instructions](SETUP.md) from the repository root as your ordinary user. Exit the game and both clients manually before a full installation or Darling restart. Incremental server/memory installers can replace files without interrupting processes, but their changes need a later full restart to activate.
## Setup stops at preflight
`scripts/build.sh` and `scripts/install-runtime.sh` now run `scripts/preflight.py` before the source fetch, the build and the per-user install. Host versions that cannot build or run the result fail in seconds with one actionable line instead of after the long steps ([#137](https://github.com/lucannez64/darling-lol/issues/137), [#138](https://github.com/lucannez64/darling-lol/issues/138), [#143](https://github.com/lucannez64/darling-lol/issues/143)):
- **python** 3.10 or newer. The per-user installers hash files with a streaming helper instead of `hashlib.file_digest`, which only exists from Python 3.11, so the Ubuntu 22.04 default no longer fails at the end of the install.
- **cmake** 3.20 or newer. The bundled indium CMakeLists requires it.
- **git-lfs** 2.13 or newer. The pinned submodules ship libraries as LFS objects.
- **lfs-pointer**: when `build/darling-src` is already prepared, small `.dylib`, `.a`, `.so` and `.o` files are checked for Git LFS pointer text. A pointer file links but crashes the guest, so this is a required check rather than a warning. It passes before the first fetch, when there is nothing to inspect yet.
- **kernel** 5.14 or newer for `MADV_POPULATE_WRITE`.
- **llvm** 16 or newer; 22 or 23 is what the patches are tested with. Iridium calls `LLVMGetVersion`, which LLVM 15 does not export, so an LLVM 15 runtime aborts as soon as Riot creates a Metal device ([#139](https://github.com/lucannez64/darling-lol/issues/139)). LLVM 16-21 builds with a warning.
- **llvm-tools**: `llvm-objdump` and `llvm-nm`. They are now resolved as an unversioned name, as `llvm-<tool>-<N>`, or from `llvm-config --bindir`, so Ubuntu's `llvm-15-dev` no longer needs symlinks in `/usr/local/bin`.
- **vulkan** API 1.3 and timeline semaphores. Indium rejects older devices, so this stops the setup with the version `vulkaninfo` reports. Without `vulkaninfo` it only warns, because the tool may be missing even when the driver is fine.
- **x11** only warns: `DISPLAY` is probed with `xdpyinfo`, and the GL/EGL vendor is reported when a probe is installed. An inaccessible display is what made `iokitd` restart in a loop before [#140](https://github.com/lucannez64/darling-lol/issues/140); export `XAUTHORITY` from servers and SSH sessions. A build without a display is still valid.
Run the same checks by hand, from the checkout:
```sh
python3 scripts/preflight.py
python3 scripts/preflight.py --json
```
If a required check is stricter than your setup, `RCLOL_SKIP_PREFLIGHT=1 ./scripts/build.sh` (and the same for `scripts/install-runtime.sh`) skips the gate for that run. Include the preflight output when reporting a setup failure, and keep `--json` output when a tool needs to parse it.
## Memory installer: permission denied
An earlier sudo server installation can create root-owned `build/backups`. The old memory installer tried to create `build/backups/memory-runtime` there and failed at `backup_folder.mkdir(...)`. The corrected default is a separate user-owned `build/memory-runtime-backups` directory:
+257
View File
@@ -0,0 +1,257 @@
#!/usr/bin/env python3
"""Check host versions the build and the installed runtime need, before the expensive steps.
Required checks exit 1 with one actionable line each; optional checks only warn.
Set RCLOL_SKIP_PREFLIGHT=1 in the environment to bypass the gate. Nothing here
touches the prefix, the guest or the network.
"""
import argparse
import json
import os
import platform
import re
import shutil
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / 'tools'))
import llvm_tools
MIN_PYTHON = (3, 10)
MIN_LLVM = 16 # LLVMGetVersion, checked by Iridium at startup, appeared in LLVM 16.
TESTED_LLVM = 22 # The patch tracks LLVM 22/23.
MIN_CMAKE = (3, 20) # The bundled indium CMakeLists requires 3.20.
MIN_LFS = (2, 13) # Debian 11 ships 2.13, the oldest release the setup is tested with.
MIN_KERNEL = (5, 14) # MADV_POPULATE_WRITE.
MIN_VULKAN = (1, 3) # Indium rejects older devices and needs timeline semaphores.
SOURCE = ROOT / 'build/darling-src'
POINTER = b'version https://git-lfs.github.com/spec/v1'
LIBRARY_SUFFIXES = ('.dylib', '.a', '.so', '.o')
def version(text):
"""(major, minor) from the first x.y in text, or None."""
match = re.search(r'(\d+)\.(\d+)', text or '')
return (int(match.group(1)), int(match.group(2))) if match else None
def dotted(value):
return '.'.join(str(part) for part in value) if value else 'unknown'
def output(command, timeout=30):
"""stdout of a command, or None when it is missing, fails or times out."""
try:
result = subprocess.run(command, capture_output=True, text=True, errors='replace',
timeout=timeout)
except (OSError, subprocess.SubprocessError):
return None
return result.stdout if result.returncode == 0 else None
def check_python(report):
found = platform.python_version()
report('python', sys.version_info[:2] >= MIN_PYTHON, found,
f'Python {dotted(MIN_PYTHON)} or newer is required; older releases lack standard '
'library calls the helpers use. Ubuntu 22.04 ships 3.10, which is enough.')
def check_cmake(report):
text = output(['cmake', '--version'])
found = version(text.splitlines()[0]) if text else None
report('cmake', found is not None and found >= MIN_CMAKE, dotted(found),
f'CMake {dotted(MIN_CMAKE)} or newer is required by the configured build.')
def check_git_lfs(report):
text = output(['git', 'lfs', 'version'])
found = version(text)
report('git-lfs', found is not None and found >= MIN_LFS, dotted(found),
f'Git LFS {dotted(MIN_LFS)} or newer is required: the pinned submodules ship '
'libraries as LFS objects and pointer text crashes the guest.')
def lfs_pointers(source):
"""Small library-like files under source that are still Git LFS pointer text."""
found = []
for folder, directories, files in os.walk(source):
directories[:] = [name for name in directories if name not in ('.git', '__pycache__')]
if Path(folder) == source:
directories[:] = [name for name in directories if name != 'build']
for name in files:
if not name.endswith(LIBRARY_SUFFIXES):
continue
path = Path(folder) / name
# A pointer is ~130 bytes. Skip everything else before opening it, so a
# prepared checkout adds about a second instead of opening every file.
try:
if path.is_symlink() or path.stat().st_size > 1024:
continue
with path.open('rb') as handle:
if handle.read(len(POINTER)) == POINTER:
found.append(path)
except OSError:
continue
return found
def check_lfs_pointers(report):
if not SOURCE.is_dir():
# The first build runs this before prepare.py fetches the pinned sources.
report('lfs-pointer', True, 'no prepared checkout yet')
return
pointers = lfs_pointers(SOURCE)
shown = ', '.join(str(path.relative_to(SOURCE)) for path in pointers[:3])
if len(pointers) > 3:
shown += f' (+{len(pointers) - 3} more)'
report('lfs-pointer', not pointers,
f'{len(pointers)} library file(s) still contain pointer text: {shown}'
if pointers else 'no pointer text in the prepared libraries',
'Git LFS did not fetch these objects. Install git-lfs and re-run '
'./scripts/build.sh (or git lfs pull in that checkout); a pointer file links but '
'crashes the guest at run time.')
def check_kernel(report):
release = os.environ.get('RCLOL_PREFLIGHT_KERNEL_RELEASE') or platform.release()
found = version(release)
report('kernel', found is not None and found >= MIN_KERNEL, release,
f'Linux {dotted(MIN_KERNEL)} or newer is required for MADV_POPULATE_WRITE, which '
"Darwin's allocator uses when it reuses pages.")
def check_llvm(report):
configuration = llvm_tools.find('llvm-config')
text = output([configuration, '--version']) if configuration else None
found = version(text.splitlines()[0]) if text else None
major = found[0] if found else None
report('llvm', major is not None and major >= MIN_LLVM,
f'{dotted(found)} ({configuration or "llvm-config not found"})',
f'LLVM {MIN_LLVM} or newer is required: Iridium calls LLVMGetVersion at startup and '
'LLVM 15 does not export it. LLVM 22 or 23 is what the patches are tested with. '
'Install the development package (for example llvm-dev or llvm23-dev).')
if major is not None and MIN_LLVM <= major < TESTED_LLVM:
report('llvm-tested', False, dotted(found),
f'LLVM {major} builds, but the patch tracks LLVM {TESTED_LLVM}/23 and older '
'majors are untested; use 22 or 23 to be safe.', required=False)
def check_llvm_tools(report):
tools = {name: llvm_tools.find(name) for name in ('llvm-objdump', 'llvm-nm')}
missing = [name for name, path in tools.items() if not path]
present = ', '.join(f'{name}={path}' for name, path in tools.items() if path)
report('llvm-tools', not missing, present or 'none found',
f'Install the LLVM tools package (for example llvm-dev or llvm23-tools): {", ".join(missing)} '
'not found as an unversioned name, as llvm-<tool>-<N> or in `llvm-config --bindir`. '
'The runtime install validates Darwin binaries with them.')
def vulkan_devices():
"""(highest apiVersion, whether a device reports timelineSemaphore, device names)."""
summary = output(['vulkaninfo', '--summary'])
if summary is None:
return None
api = [version(found) for found in re.findall(r'apiVersion\s*=\s*(\d+\.\d+)', summary)]
api = [value for value in api if value]
names = re.findall(r'deviceName\s*=\s*(.+)', summary)
return (max(api) if api else None, None, [name.strip() for name in names])
def check_vulkan(report):
devices = vulkan_devices()
if devices is None:
report('vulkan', False, 'vulkaninfo not available',
'Install vulkan-tools to check the device. The runtime needs Vulkan 1.3 and '
'timeline semaphores; without a suitable device the game cannot render.',
required=False)
return
api, timeline, names = devices
detail = f'apiVersion {dotted(api)}' + (f' ({", ".join(names[:2])})' if names else '')
report('vulkan', api is not None and api >= MIN_VULKAN, detail,
f'Vulkan {dotted(MIN_VULKAN)} or newer is required: Indium rejects older devices. '
'Update Mesa or the vendor driver, or check `vulkaninfo --summary`.')
feature = output(['vulkaninfo'], timeout=60)
if feature is None:
return
values = re.findall(r'timelineSemaphore\s*=\s*(true|false)', feature)
if values and 'true' not in values:
report('vulkan-timeline', False, 'timelineSemaphore = false',
'No Vulkan device reports timeline semaphores, which the Metal wrapper needs; '
'update the driver or select another ICD.')
def check_display(report):
display = os.environ.get('DISPLAY')
if not display:
report('x11', False, 'DISPLAY unset',
'Without X11/Xwayland the runtime boots but the game cannot present; expected on '
'a headless host that is only used to build.', required=False)
return
if not shutil.which('xdpyinfo'):
report('x11', False, f'DISPLAY={display}, xdpyinfo not installed',
'Install x11-utils to check the display before a launch.', required=False)
return
if output(['xdpyinfo'], timeout=15) is None:
report('x11', False, f'DISPLAY={display}',
f'Cannot open {display}. On servers, SSH and sudo sessions export XAUTHORITY '
'with a cookie the user running Darling can read; otherwise iokitd runs without '
'X11/EGL and the game cannot render. Builds still work without it.',
required=False)
return
vendor = ''
for probe in (['glxinfo', '-B'], ['eglinfo']):
text = output(probe, timeout=20)
if text:
found = re.search(r'(?:OpenGL vendor string|EGL_VENDOR):\s*(.+)', text)
if found:
vendor = f', {probe[0]} vendor {found.group(1).strip()}'
break
report('x11', True, f'DISPLAY={display} (xdpyinfo){vendor}')
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--json', action='store_true', help='machine-readable output')
args = parser.parse_args()
checks = []
def report(name, ok, detail, hint=None, required=True):
checks.append({'check': name, 'ok': bool(ok), 'required': required,
'detail': detail, 'hint': None if ok else hint})
for check in (check_python, check_cmake, check_git_lfs, check_lfs_pointers, check_kernel,
check_llvm, check_llvm_tools, check_vulkan, check_display):
check(report)
failed = [check for check in checks if check['required'] and not check['ok']]
warned = [check for check in checks if not check['required'] and not check['ok']]
passed = [check for check in checks if check['ok']]
if args.json:
print(json.dumps({'ok': not failed, 'checks': checks}, indent=2))
else:
print('preflight: host checks the build and runtime need')
for check in checks:
state = 'ok ' if check['ok'] else ('FAIL' if check['required'] else 'warn')
print(f' {state} {check["check"]:<13} {check["detail"]}')
if check['hint'] and not check['ok']:
print(f' {check["hint"]}')
summary = f'preflight: {len(passed)} checks passed'
if warned:
summary += f', {len(warned)} warning' + ('s' if len(warned) != 1 else '')
if failed:
summary += (f', {len(failed)} required check'
+ ('s' if len(failed) != 1 else '') + ' failed')
print(summary)
if failed:
if not args.json:
print('preflight: fix the required checks above, or set RCLOL_SKIP_PREFLIGHT=1 to bypass',
file=sys.stderr)
raise SystemExit(1)
if __name__ == '__main__':
main()
+304
View File
@@ -0,0 +1,304 @@
"""Preflight and LLVM tool discovery, with fake host tools and no Darling runtime."""
import hashlib
import importlib.util
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
PREFLIGHT = ROOT / 'scripts/preflight.py'
KERNEL = '6.1.0-test'
def load(name, path):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class PreflightFixture(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory(prefix='preflight fixture ', dir=os.environ.get('TMPDIR'))
self.addCleanup(self.tmp.cleanup)
self.bin = Path(self.tmp.name) / 'bin'
self.bin.mkdir()
self.llvm_bindir = Path(self.tmp.name) / 'llvm bin'
self.llvm_bindir.mkdir()
self.good()
def good(self):
self.stub('llvm-config', '''case "$1" in
--version) echo 23.1.5;;
--bindir) printf '%s\\n' "LLVM_BINDIR";;
esac'''.replace('LLVM_BINDIR', str(self.llvm_bindir)))
self.stub('llvm-objdump', 'exit 0')
self.stub('llvm-nm', 'exit 0')
self.stub('cmake', 'echo "cmake version 3.31.6"')
self.stub('git', 'case "$*" in "lfs version") echo "git-lfs/3.7.0 (GitHub; linux amd64)";; esac')
self.stub('vulkaninfo', '''case "$*" in
--summary) printf 'apiVersion = 1.3.289\\ndeviceName = Test GPU\\n';;
*) printf 'timelineSemaphore = true\\n';;
esac''')
self.stub('xdpyinfo', 'exit 0')
def stub(self, name, body, folder=None):
path = (folder or self.bin) / name
path.write_text('#!/bin/sh\nset -eu\n' + body + '\n')
path.chmod(0o755)
return path
def preflight(self, *arguments, kernel=KERNEL, display=':0'):
env = dict(os.environ, PATH=str(self.bin), RCLOL_PREFLIGHT_KERNEL_RELEASE=kernel)
if display is None:
env.pop('DISPLAY', None)
else:
env['DISPLAY'] = display
return subprocess.run([sys.executable, str(PREFLIGHT), *arguments], env=env,
text=True, capture_output=True, timeout=120)
def test_all_required_checks_pass(self):
result = self.preflight()
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('preflight: ', result.stdout)
self.assertNotIn('FAIL', result.stdout)
self.assertIn('ok llvm-tools', result.stdout)
def test_llvm_15_stops_with_the_reason(self):
self.stub('llvm-config', 'echo 15.0.7')
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL llvm', result.stdout)
self.assertIn('LLVM 16 or newer is required', result.stdout)
self.assertIn('LLVMGetVersion', result.stdout)
self.assertIn('RCLOL_SKIP_PREFLIGHT=1', result.stderr)
def test_untested_llvm_major_warns_but_builds(self):
self.stub('llvm-config', 'case "$1" in --version) echo 17.0.6;; esac')
result = self.preflight()
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('warn llvm-tested', result.stdout)
self.assertIn('1 warning', result.stdout)
def test_versioned_llvm_tools_are_found(self):
(self.bin / 'llvm-objdump').unlink()
(self.bin / 'llvm-nm').unlink()
self.stub('llvm-objdump-15', 'exit 0')
self.stub('llvm-nm-15', 'exit 0')
result = self.preflight()
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('llvm-objdump-15', result.stdout)
self.assertIn('llvm-nm-15', result.stdout)
def test_llvm_tools_come_from_llvm_config_bindir(self):
(self.bin / 'llvm-objdump').unlink()
(self.bin / 'llvm-nm').unlink()
self.stub('llvm-objdump-23', 'exit 0', folder=self.llvm_bindir)
self.stub('llvm-nm-23', 'exit 0', folder=self.llvm_bindir)
result = self.preflight()
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('llvm bin', result.stdout)
def test_missing_llvm_tool_names_what_to_install(self):
(self.bin / 'llvm-nm').unlink()
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL llvm-tools', result.stdout)
self.assertIn('llvm-nm not found', result.stdout)
self.assertIn('llvm23-tools', result.stdout)
def test_missing_llvm_config_names_the_package(self):
(self.bin / 'llvm-config').unlink()
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL llvm', result.stdout)
self.assertIn('llvm-config not found', result.stdout)
self.assertIn('llvm-dev', result.stdout)
def test_old_cmake_stops_the_setup(self):
self.stub('cmake', 'echo "cmake version 3.16.3"')
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL cmake', result.stdout)
self.assertIn('CMake 3.20', result.stdout)
def test_old_git_lfs_stops_the_setup(self):
self.stub('git', 'case "$*" in "lfs version") echo "git-lfs/2.9.2 (GitHub; linux amd64)";; esac')
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL git-lfs', result.stdout)
self.assertIn('Git LFS 2.13', result.stdout)
def test_old_kernel_is_required(self):
result = self.preflight(kernel='5.10.0-28-amd64')
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL kernel', result.stdout)
self.assertIn('Linux 5.14', result.stdout)
def test_old_vulkan_stops_the_setup(self):
self.stub('vulkaninfo', 'echo "apiVersion = 1.2.170"')
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL vulkan', result.stdout)
self.assertIn('Vulkan 1.3 or newer is required', result.stdout)
def test_missing_timeline_semaphores_is_reported(self):
self.stub('vulkaninfo', '''case "$*" in
--summary) echo 'apiVersion = 1.3.289';;
*) echo 'timelineSemaphore = false';;
esac''')
result = self.preflight()
self.assertEqual(result.returncode, 1)
self.assertIn('FAIL vulkan-timeline', result.stdout)
def test_unreachable_display_only_warns(self):
self.stub('xdpyinfo', 'exit 1')
result = self.preflight(display=':99')
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('warn x11', result.stdout)
self.assertIn('XAUTHORITY', result.stdout)
def test_headless_host_only_warns(self):
result = self.preflight(display=None)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn('DISPLAY unset', result.stdout)
def test_json_report_names_every_check(self):
result = self.preflight('--json')
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
report = json.loads(result.stdout)
self.assertTrue(report['ok'])
names = {check['check'] for check in report['checks']}
self.assertTrue({'python', 'cmake', 'git-lfs', 'lfs-pointer', 'kernel', 'llvm',
'llvm-tools', 'vulkan', 'x11'} <= names)
self.assertTrue(all('required' in check and 'detail' in check for check in report['checks']))
class LfsPointerFixture(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory(prefix='lfs pointer ', dir=os.environ.get('TMPDIR'))
self.addCleanup(self.tmp.cleanup)
self.source = Path(self.tmp.name) / 'darling-src'
self.source.mkdir()
self.preflight = load('preflight_lfs', PREFLIGHT)
self.preflight.SOURCE = self.source
def check(self):
reports = []
self.preflight.check_lfs_pointers(lambda *arguments, **keywords: reports.append(
(arguments, keywords)))
self.assertEqual(len(reports), 1)
arguments, keywords = reports[0]
name, ok, detail = arguments[:3]
hint = arguments[3] if len(arguments) > 3 else keywords.get('hint')
return name, ok, detail, hint
def write(self, name, content):
path = self.source / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(content)
return path
def test_pointer_text_in_a_library_is_reported(self):
self.write('lib/libswiftCore.dylib',
b'version https://git-lfs.github.com/spec/v1\noid sha256:abc\nsize 123\n')
self.write('lib/libreal.dylib', b'\xcf\xfa\xed\xfe' + b'\0' * 64)
name, ok, detail, hint = self.check()
self.assertEqual(name, 'lfs-pointer')
self.assertFalse(ok)
self.assertIn('lib/libswiftCore.dylib', detail)
self.assertNotIn('libreal', detail)
self.assertIn('git lfs pull', hint)
def test_pointer_text_outside_a_library_is_ignored(self):
self.write('notes.dylib.txt', b'version https://git-lfs.github.com/spec/v1\n')
self.write('lib/libreal.dylib', b'not a pointer')
_, ok, detail, _ = self.check()
self.assertTrue(ok, detail)
def test_missing_checkout_is_not_a_failure(self):
self.preflight.SOURCE = self.source / 'not fetched'
_, ok, detail, _ = self.check()
self.assertTrue(ok)
self.assertIn('no prepared checkout', detail)
class LlvmToolsFixture(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory(prefix='llvm tools ', dir=os.environ.get('TMPDIR'))
self.addCleanup(self.tmp.cleanup)
self.bin = Path(self.tmp.name) / 'bin'
self.bin.mkdir()
self.saved = os.environ.get('PATH')
os.environ['PATH'] = str(self.bin)
self.addCleanup(self.restore)
self.tools = load('llvm_tools', ROOT / 'tools/llvm_tools.py')
def restore(self):
os.environ['PATH'] = self.saved or ''
def stub(self, name, body='exit 0', folder=None):
path = (folder or self.bin) / name
path.write_text('#!/bin/sh\n' + body + '\n')
path.chmod(0o755)
return path
def test_prefers_the_unversioned_name(self):
unversioned = self.stub('llvm-objdump')
self.stub('llvm-objdump-23')
self.assertEqual(self.tools.find('llvm-objdump'), str(unversioned))
def test_prefers_the_highest_versioned_name(self):
self.stub('llvm-objdump-15')
newest = self.stub('llvm-objdump-23')
self.stub('llvm-objdump-9')
self.assertEqual(self.tools.find('llvm-objdump'), str(newest))
def test_ignores_a_name_that_is_not_a_version(self):
self.stub('llvm-objdump-tools')
self.assertIsNone(self.tools.find('llvm-objdump'))
def test_falls_back_to_llvm_config_bindir(self):
libexec = Path(self.tmp.name) / 'libexec'
libexec.mkdir()
expected = self.stub('llvm-nm-22', folder=libexec)
self.stub('llvm-config', 'echo "%s"' % libexec)
self.assertEqual(self.tools.find('llvm-nm'), str(expected))
def test_require_explains_what_to_install(self):
with self.assertRaises(SystemExit) as raised:
self.tools.require('llvm-objdump')
self.assertIn('llvm-objdump not found', str(raised.exception))
self.assertIn('llvm-config --bindir', str(raised.exception))
class FileDigestFixture(unittest.TestCase):
def test_sha256_matches_hashlib(self):
digest = load('file_digest', ROOT / 'tools/file_digest.py')
with tempfile.TemporaryDirectory(prefix='digest ', dir=os.environ.get('TMPDIR')) as folder:
path = Path(folder) / 'payload'
payload = b'darling' * 400_000 # More than one chunk.
path.write_bytes(payload)
self.assertEqual(digest.sha256(path), hashlib.sha256(payload).hexdigest())
+6
View File
@@ -13,6 +13,12 @@ SCRIPT = Path(__file__).resolve().parents[1] / 'tools/ensure-darling-runtime.py'
class RuntimeCleanupScopeTests(unittest.TestCase):
def enterContext(self, manager):
"""unittest.TestCase.enterContext() is Python 3.11+; Ubuntu 22.04 ships 3.10."""
value = manager.__enter__()
self.addCleanup(manager.__exit__, None, None, None)
return value
def setUp(self):
temporary = tempfile.TemporaryDirectory(prefix='cleanup-scope-')
self.addCleanup(temporary.cleanup)
+4
View File
@@ -65,7 +65,11 @@ class StopRuntimeTests(unittest.TestCase):
def __init__(self, *parts, **kwargs):
if parts and str(parts[0]) == '/proc':
parts = (proc, *parts[1:])
try:
super().__init__(*parts, **kwargs)
except TypeError:
# Python 3.10's PurePath.__init__ takes no arguments; 3.12 stores them.
super().__init__()
def read_text(self, *args, **kwargs):
if self.name == 'status' and owner.status_error is not None:
+6 -3
View File
@@ -4,12 +4,15 @@ from pathlib import Path
import shlex
import subprocess
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
root = Path(__file__).resolve().parent.parent
if '--native' in sys.argv:
indium = root / 'build/darling-src/src/external/metal/deps/indium'
llvm_include = subprocess.check_output(['llvm-config', '--includedir'], text=True).strip()
llvm_libdir = Path(subprocess.check_output(['llvm-config', '--libdir'], text=True).strip())
llvm_names = shlex.split(subprocess.check_output(['llvm-config', '--link-shared', '--libnames'], text=True))
llvm_include = llvm_tools.config('--includedir')
llvm_libdir = Path(llvm_tools.config('--libdir'))
llvm_names = shlex.split(llvm_tools.config('--link-shared', '--libnames'))
if len(llvm_names) != 1:
raise SystemExit('Iridium requires one shared LLVM library, got: ' + repr(llvm_names))
llvm_library = llvm_libdir / llvm_names[0]
+6 -3
View File
@@ -5,6 +5,9 @@ import os
import shlex
import subprocess
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
root = Path(__file__).resolve().parent.parent
folder = root / 'build/darling-src/build/src/external/metal/deps/indium'
# Keep the validated LLVM major locally when a system update replaces it.
@@ -24,9 +27,9 @@ if (pinned_llvm / 'lib/libLLVM-22.so').exists() and os.environ.get('RIOT_IRIDIUM
llvm_library = pinned_llvm / 'lib/libLLVM-22.so'
else:
# Do not combine cached CMake include paths with a newer llvm-config library.
llvm_include = Path(subprocess.check_output(['llvm-config', '--includedir'], text=True).strip())
llvm_libdir = Path(subprocess.check_output(['llvm-config', '--libdir'], text=True).strip())
llvm_names = shlex.split(subprocess.check_output(['llvm-config', '--link-shared', '--libnames'], text=True))
llvm_include = Path(llvm_tools.config('--includedir'))
llvm_libdir = Path(llvm_tools.config('--libdir'))
llvm_names = shlex.split(llvm_tools.config('--link-shared', '--libnames'))
if len(llvm_names) != 1:
raise SystemExit('Iridium requires one shared LLVM library, got: ' + repr(llvm_names))
llvm_library = llvm_libdir / llvm_names[0]
+6 -2
View File
@@ -7,13 +7,17 @@ through it, leaving %rdi unrelated: the thunk then returns NULL and the first wr
Usage: check-darwin-tlv-calls.py BINARY... ; exits 1 and lists every bad access it finds.
"""
import re
from pathlib import Path
import subprocess
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
def thread_vars_ranges(path):
"""Address ranges of the __thread_vars sections, which hold the TLV descriptors."""
output = subprocess.run(['llvm-objdump', '--macho', '--section-headers', path],
output = subprocess.run([llvm_tools.require('llvm-objdump'), '--macho', '--section-headers', path],
capture_output=True, text=True, check=True).stdout
ranges = []
for line in output.splitlines():
@@ -31,7 +35,7 @@ def bad_accesses(path):
ranges = thread_vars_ranges(path)
if not ranges:
return []
disassembly = subprocess.run(['llvm-objdump', '-d', '--no-show-raw-insn', path],
disassembly = subprocess.run([llvm_tools.require('llvm-objdump'), '-d', '--no-show-raw-insn', path],
capture_output=True, text=True, check=True).stdout
load = re.compile(r'(?:leaq|movq)\s+-?0x[0-9a-f]+\(%rip\), %(\w+)\s+#+ 0x([0-9a-f]+)')
found, function, pending = [], '?', {}
+15
View File
@@ -0,0 +1,15 @@
"""SHA-256 of a file without hashlib.file_digest().
hashlib.file_digest() is Python 3.11+; Ubuntu 22.04 ships Python 3.10, where the
installers failed only after the build had finished (issue #137). This reads in
chunks, so the memory use does not depend on the file size.
"""
import hashlib
def sha256(path):
checksum = hashlib.sha256()
with open(path, 'rb') as source:
for chunk in iter(lambda: source.read(1 << 20), b''):
checksum.update(chunk)
return checksum.hexdigest()
+3 -3
View File
@@ -1,7 +1,6 @@
#!/usr/bin/env python3
"""Atomically install a built Darling server; never stop or start any process."""
import argparse
import hashlib
import json
import os
from pathlib import Path
@@ -9,12 +8,13 @@ import shutil
import stat
import tempfile
from file_digest import sha256
ROOT = Path(__file__).resolve().parent.parent
def digest(path):
with path.open("rb") as input:
return hashlib.file_digest(input, "sha256").hexdigest()
return sha256(path)
def live_binaries(destination):
+2 -3
View File
@@ -9,7 +9,6 @@ tools/install-riot-memory-runtime.py (its default source is the rebuilt
build/riot-libsystem_kernel.dylib); this installer does not touch it.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
@@ -18,14 +17,14 @@ import stat
import tempfile
from runtime_paths import runtime_paths
from file_digest import sha256
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_ARTIFACT = ROOT / "build/riot-Foundation"
def digest(path):
with path.open("rb") as input:
return hashlib.file_digest(input, "sha256").hexdigest()
return sha256(path)
def copy_framework(sdk_framework, framework):
+2 -3
View File
@@ -15,7 +15,6 @@ and is never run against the real prefix by the build. Nothing is activated
until the guest processes are restarted.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
@@ -24,6 +23,7 @@ import stat
import tempfile
from runtime_paths import validate_runtime_paths
from file_digest import sha256
ROOT = Path(__file__).resolve().parents[1]
@@ -40,8 +40,7 @@ LIBRARIES = [
def digest(path):
with path.open("rb") as input:
return hashlib.file_digest(input, "sha256").hexdigest()
return sha256(path)
def destinations(prefix, guest_home):
+2 -3
View File
@@ -1,7 +1,6 @@
#!/usr/bin/env python3
"""Install the tested reclamation library atomically, without restarting clients."""
import argparse
import hashlib
import json
import os
from pathlib import Path
@@ -11,6 +10,7 @@ import stat
import tempfile
from runtime_paths import runtime_paths
from file_digest import sha256
ROOT = Path(__file__).resolve().parents[1]
# The optimized relink from tools/build-riot-kqueue-runtime.py is the library the
@@ -19,8 +19,7 @@ DEFAULT_ARTIFACT = ROOT / "build/riot-libsystem_kernel.dylib"
def digest(path):
with path.open("rb") as input:
return hashlib.file_digest(input, "sha256").hexdigest()
return sha256(path)
def install(binary, destinations, backup_folder):
+93
View File
@@ -0,0 +1,93 @@
"""Find LLVM command line tools, including versioned Debian/Ubuntu names.
Ubuntu's llvm-<N>-dev installs llvm-objdump and llvm-nm in /usr/lib/llvm-<N>/bin
without an unversioned name in PATH, so a bare ['llvm-objdump'] run fails on an
otherwise complete host (issue #138). Resolution order: an unversioned tool on
PATH, then the highest versioned name on PATH, then the directory llvm-config
reports. Nothing here runs a guest process.
"""
import os
import subprocess
def _collect(directory, name, found):
"""Add every `name` and `name-<digits>` executable in the directory to found."""
try:
entries = os.listdir(directory)
except OSError:
return
for entry in entries:
suffix = entry[len(name) + 1:]
if entry == name:
version = 0 # An unversioned name always wins.
elif entry.startswith(name + '-') and suffix.isdigit():
version = int(suffix)
else:
continue
path = os.path.join(directory or '.', entry)
if os.path.isfile(path) and os.access(path, os.X_OK):
found.setdefault(version, path)
def _executables(name):
"""{version: path} for `name` and for every `name-<digits>` on PATH."""
found = {}
for folder in os.environ.get('PATH', '').split(os.pathsep):
_collect(folder or '.', name, found)
return found
def _preferred(candidates):
return candidates[0 if 0 in candidates else max(candidates)] if candidates else None
def _bindir():
"""`llvm-config --bindir` from the preferred llvm-config, or None."""
configuration = _preferred(_executables('llvm-config'))
if not configuration:
return None
result = subprocess.run([configuration, '--bindir'],
capture_output=True, text=True, errors='replace')
directory = result.stdout.strip()
if result.returncode or not os.path.isdir(directory):
return None
return directory
def find(name):
"""Path of an LLVM tool, or None when this host does not have it."""
candidates = _executables(name)
if candidates:
return _preferred(candidates)
directory = _bindir()
if directory:
in_bindir = {}
_collect(directory, name, in_bindir)
if in_bindir:
return _preferred(in_bindir)
return None
def require(name):
"""Path of an LLVM tool, or exit with what was tried and what to install."""
path = find(name)
if path:
return path
raise SystemExit(f'{name} not found: looked for an unversioned {name}, a versioned '
f'{name}-<N> in PATH and `llvm-config --bindir`. Install the LLVM tools '
f'package (for example llvm-dev, llvm15-tools or llvm23-tools).')
def config(*arguments):
"""Run llvm-config with the arguments, or exit with an actionable message."""
path = require('llvm-config')
result = subprocess.run([path, *arguments], capture_output=True, text=True, errors='replace')
if result.returncode:
raise SystemExit(f'{path} {" ".join(arguments)} failed: {result.stderr.strip()}')
return result.stdout.strip()
def major():
"""LLVM major version reported by llvm-config, or None when unparsable."""
head = config('--version').strip().split('.', 1)[0]
return int(head) if head.isdigit() else None
+5 -1
View File
@@ -3,11 +3,15 @@
from pathlib import Path
import re
import subprocess
import sys
import tempfile
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
root = Path(__file__).resolve().parent.parent
indium = root / 'build/darling-src/src/external/metal/deps/indium'
include = Path(subprocess.check_output(['llvm-config', '--includedir'], text=True).strip())
include = Path(llvm_tools.config('--includedir'))
header = (include / 'llvm/Config/llvm-config.h').read_text()
major = int(re.search(r'^#define LLVM_VERSION_MAJOR (\d+)', header, re.MULTILINE)[1])
with tempfile.TemporaryDirectory(prefix='iridium-llvm-version-') as directory:
+6 -1
View File
@@ -23,6 +23,9 @@ import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
ROOT = Path(__file__).resolve().parent.parent
GAME = (Path.home() / ".darling/Applications/League of Legends.app/Contents/LoL/Game"
/ "LeagueofLegends.app/Contents/MacOS/LeagueofLegends")
@@ -44,7 +47,9 @@ MIN_FUNCTION_INSTRUCTIONS = 20
def tool(*arguments):
return subprocess.run(arguments, capture_output=True, text=True, errors="replace").stdout
"""Run an LLVM tool, resolving versioned names from PATH or llvm-config."""
return subprocess.run((llvm_tools.require(arguments[0]), *arguments[1:]),
capture_output=True, text=True, errors="replace").stdout
def load_commands(path, requested):
+13 -8
View File
@@ -16,6 +16,9 @@ import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import llvm_tools
ROOT = Path(__file__).resolve().parent.parent
@@ -27,11 +30,16 @@ def load_audit():
return module
def headers(path):
return subprocess.run(["llvm-objdump", "--macho", "--private-headers", str(path)],
def run(*arguments):
"""Run an LLVM tool, resolving versioned names from PATH or llvm-config."""
return subprocess.run((llvm_tools.require(arguments[0]), *arguments[1:]),
capture_output=True, text=True, errors="replace").stdout
def headers(path):
return run("llvm-objdump", "--macho", "--private-headers", str(path))
def identity(path):
"""Install name and the two versions from LC_ID_DYLIB."""
text = headers(path)
@@ -43,8 +51,7 @@ def identity(path):
def dependencies(path):
text = subprocess.run(["llvm-objdump", "--macho", "--dylibs-used", str(path)],
capture_output=True, text=True, errors="replace").stdout
text = run("llvm-objdump", "--macho", "--dylibs-used", str(path))
return sorted(re.findall(r"^\t(\S+) \(compatibility version", text, re.M))
@@ -70,8 +77,7 @@ def weak_definitions(path):
"""Symbols the library defines only weakly: the -O0 build emits every
inline function out of line, an optimized build may drop the unused ones,
and the ABI makes a weak definition optional."""
text = subprocess.run(["llvm-nm", "-m", str(path)], capture_output=True, text=True,
errors="replace").stdout
text = run("llvm-nm", "-m", str(path))
weak = set()
for line in text.splitlines():
if "weak external" in line:
@@ -80,8 +86,7 @@ def weak_definitions(path):
def symbols(path, flag):
text = subprocess.run(["llvm-nm", flag, str(path)], capture_output=True, text=True,
errors="replace").stdout
text = run("llvm-nm", flag, str(path))
found = set()
for line in text.splitlines():
parts = line.split()