mirror of
https://github.com/lucannez64/darling-lol.git
synced 2026-10-26 19:47:42 +00:00
Merge commit 'refs/review/mint-150' into fix/mint-review-integration
Signed-off-by: lucannez64 <[email protected]> # Conflicts: # .github/workflows/source-check.yml
This commit is contained in:
commit
aa5beb39d3
22 files changed
+897
-32
No files matched your search
@@ -44,3 +44,4 @@ jobs:
|
||||
- run: python -m unittest discover -s tests -p 'test_hide_league_client_in_game.py'
|
||||
- run: python -m unittest discover -s tests -p 'test_client_in_game_tricks.py'
|
||||
- run: python -m unittest discover -s tests -p 'test_setup_portability.py'
|
||||
- run: python -m unittest discover -s tests -p 'test_preflight.py'
|
||||
@@ -41,7 +41,7 @@ Game files, Riot installers, account databases, tokens, local prefixes, captures
|
||||
## Requirements
|
||||
|
||||
- x86-64 Linux **5.14 or newer**, an X11 session or Xwayland, and a working hardware Vulkan/OpenGL driver. AMD RADV and the NVIDIA driver (proprietary and open kernel modules) have been exercised, see [NVIDIA](docs/TROUBLESHOOTING.md#nvidia); other drivers are not validated. Hybrid laptops usually render on the integrated GPU unless started with PRIME offload. See [hybrid graphics](docs/TROUBLESHOOTING.md#hybrid-graphics-laptops-amdintel-igpu--nvidia).
|
||||
- Git + Git LFS, Bash, curl, unzip, CMake, Make, Clang/LLD, LLVM development files, a C/C++ compiler, Python 3.12+ and the Darling build dependencies. The patch includes LLVM 22/23 branch-opcode compatibility; the new PC used LLVM/Clang 23. Node.js is needed only for the offline JavaScript checks.
|
||||
- Git + Git LFS (2.13 or newer), Bash, curl, unzip, CMake 3.20+, Make, Clang/LLD, LLVM development files (16 or newer), a C/C++ compiler, Python 3.10+ and the Darling build dependencies. `scripts/build.sh` and `scripts/install-runtime.sh` run `scripts/preflight.py` first and stop on a version that cannot build or run the result; `RCLOL_SKIP_PREFLIGHT=1` bypasses that gate. The patch includes LLVM 22/23 branch-opcode compatibility, and LLVM 22 or 23 is what the Metal overlays are tested with. Node.js is needed only for the offline JavaScript checks.
|
||||
- X11/XCB/SHM development files for the current runtime helpers.
|
||||
- Enough free disk for Darling sources/build, Git LFS assets and the game: budget at least 50 GB. Building is substantially larger than this source-only repository.
|
||||
|
||||
|
||||
@@ -55,6 +55,8 @@ Linux 5.14 or newer is required for `MADV_POPULATE_WRITE`, used when Darwin's al
|
||||
|
||||
The runtime installer requires native `pkg-config` entries for `x11`, `x11-xcb`, `xcb-shm` and `xext`, plus the base Darling SDK.
|
||||
|
||||
`scripts/build.sh` and `scripts/install-runtime.sh` start with `scripts/preflight.py`, which checks the host versions the build and the runtime need: Python, CMake, Git LFS, the kernel, LLVM and its command line tools, the Vulkan API level, and the X display. A required check that fails stops immediately with one actionable line; the display check only warns, because a build does not need one. Run it directly with `python3 scripts/preflight.py` (`--json` for a machine-readable report), and set `RCLOL_SKIP_PREFLIGHT=1` to bypass the gate for one run.
|
||||
|
||||
Run configuration, builds and the per-user runtime installation as your ordinary account. Use sudo only for the base CMake installation and the incremental server installer. Running the whole build or per-user install under sudo can create root-owned files and configure the wrong guest home. `scripts/install-runtime.sh` initializes the default user prefix and creates `~/.darling/private/tmp` with mode `1777`, so the installer can write guest temporary files. It then shuts down Darling before replacing the paired runtime: first exit the game and both clients manually.
|
||||
|
||||
The normal launcher starts neither an Electron inspector nor client API/UI helpers. It clears inherited `DYLD_INSERT_LIBRARIES`, retains Darling runtime library/framework overrides, supplies an absolute log directory to the launcher, and selects the League product. The log-directory/Chromium argument shim, empty-autofill hook and optional native VPX bridge are no longer injected or built by runtime installation. Existing copies on disk are inactive and may remain. Child-process log-directory rewriting and Chromium feature overrides previously supplied by the shim are removed; client startup without those workarounds needs manual validation. The first-time installer uses `--skip-to-install` and omits the installed client's `--app-root`, avoiding an immediate exit or inaccessible intro screen. The project ships no inspector hooks, client API helpers, input automation or client binary patches. Keep account data, logs and captures private.
|
||||
|
||||
@@ -48,6 +48,29 @@ never delete a populated Riot/game prefix just to repair the launcher.
|
||||
|
||||
Use the [setup instructions](SETUP.md) from the repository root as your ordinary user. Exit the game and both clients manually before a full installation or Darling restart. Incremental server/memory installers can replace files without interrupting processes, but their changes need a later full restart to activate.
|
||||
|
||||
## Setup stops at preflight
|
||||
|
||||
`scripts/build.sh` and `scripts/install-runtime.sh` now run `scripts/preflight.py` before the source fetch, the build and the per-user install. Host versions that cannot build or run the result fail in seconds with one actionable line instead of after the long steps ([#137](https://github.com/lucannez64/darling-lol/issues/137), [#138](https://github.com/lucannez64/darling-lol/issues/138), [#143](https://github.com/lucannez64/darling-lol/issues/143)):
|
||||
|
||||
- **python** 3.10 or newer. The per-user installers hash files with a streaming helper instead of `hashlib.file_digest`, which only exists from Python 3.11, so the Ubuntu 22.04 default no longer fails at the end of the install.
|
||||
- **cmake** 3.20 or newer. The bundled indium CMakeLists requires it.
|
||||
- **git-lfs** 2.13 or newer. The pinned submodules ship libraries as LFS objects.
|
||||
- **lfs-pointer**: when `build/darling-src` is already prepared, small `.dylib`, `.a`, `.so` and `.o` files are checked for Git LFS pointer text. A pointer file links but crashes the guest, so this is a required check rather than a warning. It passes before the first fetch, when there is nothing to inspect yet.
|
||||
- **kernel** 5.14 or newer for `MADV_POPULATE_WRITE`.
|
||||
- **llvm** 16 or newer; 22 or 23 is what the patches are tested with. Iridium calls `LLVMGetVersion`, which LLVM 15 does not export, so an LLVM 15 runtime aborts as soon as Riot creates a Metal device ([#139](https://github.com/lucannez64/darling-lol/issues/139)). LLVM 16-21 builds with a warning.
|
||||
- **llvm-tools**: `llvm-objdump` and `llvm-nm`. They are now resolved as an unversioned name, as `llvm-<tool>-<N>`, or from `llvm-config --bindir`, so Ubuntu's `llvm-15-dev` no longer needs symlinks in `/usr/local/bin`.
|
||||
- **vulkan** API 1.3 and timeline semaphores. Indium rejects older devices, so this stops the setup unless one device reports both the version and the semaphores, as `vulkaninfo` shows per GPU. An installed `vulkaninfo` that fails or times out is also required to be fixed, because the loader or the ICD files are then broken; without `vulkaninfo` at all it only warns, because the tool may be missing even when the driver is fine.
|
||||
- **x11** only warns: `DISPLAY` is probed with `xdpyinfo`, and the GL/EGL vendor is reported when a probe is installed. An inaccessible display is what made `iokitd` restart in a loop before [#140](https://github.com/lucannez64/darling-lol/issues/140); export `XAUTHORITY` from servers and SSH sessions. A build without a display is still valid.
|
||||
|
||||
Run the same checks by hand, from the checkout:
|
||||
|
||||
```sh
|
||||
python3 scripts/preflight.py
|
||||
python3 scripts/preflight.py --json
|
||||
```
|
||||
|
||||
If a required check is stricter than your setup, `RCLOL_SKIP_PREFLIGHT=1 ./scripts/build.sh` (and the same for `scripts/install-runtime.sh`) skips the gate for that run. Include the preflight output when reporting a setup failure, and keep `--json` output when a tool needs to parse it.
|
||||
|
||||
## Memory installer: permission denied
|
||||
|
||||
An earlier sudo server installation can create root-owned `build/backups`. The old memory installer tried to create `build/backups/memory-runtime` there and failed at `backup_folder.mkdir(...)`. The corrected default is a separate user-owned `build/memory-runtime-backups` directory:
|
||||
|
||||
@@ -2,6 +2,9 @@
|
||||
set -eu
|
||||
repo=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
||||
cd "$repo"
|
||||
# Refuse before the long fetch and build when the host cannot build or run the result.
|
||||
# Runs before configure.py so an old interpreter gets this message, not a traceback.
|
||||
[ "${RCLOL_SKIP_PREFLIGHT:-0}" = 1 ] || python3 scripts/preflight.py
|
||||
python3 scripts/configure.py
|
||||
if [ -d build/venv/bin ]; then PATH="$repo/build/venv/bin:$PATH"; export PATH; fi
|
||||
python3 scripts/prepare.py
|
||||
|
||||
@@ -13,6 +13,8 @@ fi
|
||||
python3 scripts/configure.py
|
||||
. build/runtime/tools/runtime-paths.sh
|
||||
if [ -d build/venv/bin ]; then PATH="$repo/build/venv/bin:$PATH"; export PATH; fi
|
||||
# Check versions before any runtime helper touches the prefix or the running session.
|
||||
[ "${RCLOL_SKIP_PREFLIGHT:-0}" = 1 ] || python3 scripts/preflight.py
|
||||
# A server killed by the base install leaves a stale .init.pid that makes darling shell fail.
|
||||
python3 build/runtime/tools/ensure-darling-runtime.py
|
||||
# Initialize the normal user-owned prefix before installing overrides.
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check host versions the build and the installed runtime need, before the expensive steps.
|
||||
|
||||
Required checks exit 1 with one actionable line each; optional checks only warn.
|
||||
Set RCLOL_SKIP_PREFLIGHT=1 in the environment to bypass the gate. Nothing here
|
||||
touches the prefix, the guest or the network.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT / 'tools'))
|
||||
import llvm_tools
|
||||
|
||||
MIN_PYTHON = (3, 10)
|
||||
MIN_LLVM = 16 # LLVMGetVersion, checked by Iridium at startup, appeared in LLVM 16.
|
||||
TESTED_LLVM = 22 # The patch tracks LLVM 22/23.
|
||||
MIN_CMAKE = (3, 20) # The bundled indium CMakeLists requires 3.20.
|
||||
MIN_LFS = (2, 13) # Debian 11 ships 2.13, the oldest release the setup is tested with.
|
||||
MIN_KERNEL = (5, 14) # MADV_POPULATE_WRITE.
|
||||
MIN_VULKAN = (1, 3) # Indium rejects older devices and needs timeline semaphores.
|
||||
|
||||
SOURCE = Path(os.environ.get('RCLOL_PREFLIGHT_SOURCE') or ROOT / 'build/darling-src')
|
||||
POINTER = b'version https://git-lfs.github.com/spec/v1'
|
||||
LIBRARY_SUFFIXES = ('.dylib', '.a', '.so', '.o')
|
||||
|
||||
|
||||
def version(text):
|
||||
"""(major, minor) from the first x.y in text, or None."""
|
||||
match = re.search(r'(\d+)\.(\d+)', text or '')
|
||||
return (int(match.group(1)), int(match.group(2))) if match else None
|
||||
|
||||
|
||||
def dotted(value):
|
||||
return '.'.join(str(part) for part in value) if value else 'unknown'
|
||||
|
||||
|
||||
def output(command, timeout=30):
|
||||
"""stdout of a command, or None when it is missing, fails or times out."""
|
||||
try:
|
||||
result = subprocess.run(command, capture_output=True, text=True, errors='replace',
|
||||
timeout=timeout)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
return result.stdout if result.returncode == 0 else None
|
||||
|
||||
|
||||
def check_python(report):
|
||||
found = platform.python_version()
|
||||
report('python', sys.version_info[:2] >= MIN_PYTHON, found,
|
||||
f'Python {dotted(MIN_PYTHON)} or newer is required; older releases lack standard '
|
||||
'library calls the helpers use. Ubuntu 22.04 ships 3.10, which is enough.')
|
||||
|
||||
|
||||
def check_cmake(report):
|
||||
text = output(['cmake', '--version'])
|
||||
found = version(text.splitlines()[0]) if text else None
|
||||
report('cmake', found is not None and found >= MIN_CMAKE, dotted(found),
|
||||
f'CMake {dotted(MIN_CMAKE)} or newer is required by the configured build.')
|
||||
|
||||
|
||||
def check_git_lfs(report):
|
||||
text = output(['git', 'lfs', 'version'])
|
||||
found = version(text)
|
||||
report('git-lfs', found is not None and found >= MIN_LFS, dotted(found),
|
||||
f'Git LFS {dotted(MIN_LFS)} or newer is required: the pinned submodules ship '
|
||||
'libraries as LFS objects and pointer text crashes the guest.')
|
||||
|
||||
|
||||
def lfs_pointers(source):
|
||||
"""Small library-like files under source that are still Git LFS pointer text."""
|
||||
found = []
|
||||
for folder, directories, files in os.walk(source):
|
||||
directories[:] = [name for name in directories if name not in ('.git', '__pycache__')]
|
||||
if Path(folder) == source:
|
||||
directories[:] = [name for name in directories if name != 'build']
|
||||
for name in files:
|
||||
if not name.endswith(LIBRARY_SUFFIXES):
|
||||
continue
|
||||
path = Path(folder) / name
|
||||
# A pointer is ~130 bytes. Skip everything else before opening it, so a
|
||||
# prepared checkout adds about a second instead of opening every file.
|
||||
try:
|
||||
if path.is_symlink() or path.stat().st_size > 1024:
|
||||
continue
|
||||
with path.open('rb') as handle:
|
||||
if handle.read(len(POINTER)) == POINTER:
|
||||
found.append(path)
|
||||
except OSError:
|
||||
continue
|
||||
return found
|
||||
|
||||
|
||||
def check_lfs_pointers(report):
|
||||
if not SOURCE.is_dir():
|
||||
# The first build runs this before prepare.py fetches the pinned sources.
|
||||
report('lfs-pointer', True, 'no prepared checkout yet')
|
||||
return
|
||||
pointers = lfs_pointers(SOURCE)
|
||||
shown = ', '.join(str(path.relative_to(SOURCE)) for path in pointers[:3])
|
||||
if len(pointers) > 3:
|
||||
shown += f' (+{len(pointers) - 3} more)'
|
||||
report('lfs-pointer', not pointers,
|
||||
f'{len(pointers)} library file(s) still contain pointer text: {shown}'
|
||||
if pointers else 'no pointer text in the prepared libraries',
|
||||
'Git LFS did not fetch these objects. Install git-lfs and re-run '
|
||||
'./scripts/build.sh (or git lfs pull in that checkout); a pointer file links but '
|
||||
'crashes the guest at run time.')
|
||||
|
||||
|
||||
def check_kernel(report):
|
||||
release = os.environ.get('RCLOL_PREFLIGHT_KERNEL_RELEASE') or platform.release()
|
||||
found = version(release)
|
||||
report('kernel', found is not None and found >= MIN_KERNEL, release,
|
||||
f'Linux {dotted(MIN_KERNEL)} or newer is required for MADV_POPULATE_WRITE, which '
|
||||
"Darwin's allocator uses when it reuses pages.")
|
||||
|
||||
|
||||
def check_llvm(report):
|
||||
configuration = llvm_tools.find('llvm-config')
|
||||
text = output([configuration, '--version']) if configuration else None
|
||||
found = version(text.splitlines()[0]) if text else None
|
||||
major = found[0] if found else None
|
||||
report('llvm', major is not None and major >= MIN_LLVM,
|
||||
f'{dotted(found)} ({configuration or "llvm-config not found"})',
|
||||
f'LLVM {MIN_LLVM} or newer is required: Iridium calls LLVMGetVersion at startup and '
|
||||
'LLVM 15 does not export it. LLVM 22 or 23 is what the patches are tested with. '
|
||||
'Install the development package (for example llvm-dev or llvm23-dev).')
|
||||
if major is not None and MIN_LLVM <= major < TESTED_LLVM:
|
||||
report('llvm-tested', False, dotted(found),
|
||||
f'LLVM {major} builds, but the patch tracks LLVM {TESTED_LLVM}/23 and older '
|
||||
'majors are untested; use 22 or 23 to be safe.', required=False)
|
||||
|
||||
|
||||
def check_llvm_tools(report):
|
||||
tools = {name: llvm_tools.find(name) for name in ('llvm-objdump', 'llvm-nm')}
|
||||
missing = [name for name, path in tools.items() if not path]
|
||||
present = ', '.join(f'{name}={path}' for name, path in tools.items() if path)
|
||||
report('llvm-tools', not missing, present or 'none found',
|
||||
f'Install the LLVM tools package (for example llvm-dev or llvm23-tools): {", ".join(missing)} '
|
||||
'not found as an unversioned name, as llvm-<tool>-<N> or in `llvm-config --bindir`. '
|
||||
'The runtime install validates Darwin binaries with them.')
|
||||
|
||||
|
||||
def gpu_blocks(text):
|
||||
"""{index: [lines]} for each `GPUn:` block in vulkaninfo output."""
|
||||
blocks = {}
|
||||
current = None
|
||||
for line in (text or '').splitlines():
|
||||
header = re.match(r'GPU(\d+):\s*$', line)
|
||||
if header:
|
||||
current = blocks.setdefault(int(header.group(1)), [])
|
||||
continue
|
||||
if current is not None:
|
||||
current.append(line)
|
||||
return blocks
|
||||
|
||||
|
||||
def first_match(pattern, lines):
|
||||
for line in lines:
|
||||
found = re.search(pattern, line)
|
||||
if found:
|
||||
return found.group(1).strip()
|
||||
return None
|
||||
|
||||
|
||||
def vulkan_devices():
|
||||
"""One record per Vulkan device, or None when vulkaninfo does not run.
|
||||
|
||||
A device is kept as a whole record so a host with an old fast GPU and a new
|
||||
GPU without timeline semaphores cannot pass by combining the two. The full
|
||||
dump carries the features; `--summary` carries the versions that every
|
||||
version of the tool prints.
|
||||
"""
|
||||
summary = output(['vulkaninfo', '--summary'])
|
||||
if summary is None:
|
||||
return None
|
||||
features = gpu_blocks(output(['vulkaninfo'], timeout=60))
|
||||
devices = []
|
||||
for index, lines in sorted(gpu_blocks(summary).items()):
|
||||
found = features.get(index, [])
|
||||
values = [value == 'true'
|
||||
for value in re.findall(r'timelineSemaphore\s*=\s*(true|false)',
|
||||
'\n'.join(found))]
|
||||
devices.append({'name': first_match(r'deviceName\s*=\s*(.+)', lines),
|
||||
'api': version(first_match(r'apiVersion\s*=\s*(\d+\.\d+)', lines)),
|
||||
'timeline': any(values) if values else None})
|
||||
return devices
|
||||
|
||||
|
||||
def describe_device(device):
|
||||
text = f'{device["name"] or "unnamed device"} {dotted(device["api"])}'
|
||||
if device['timeline'] is not None:
|
||||
text += ', timelineSemaphore ' + ('true' if device['timeline'] else 'false')
|
||||
return text
|
||||
|
||||
|
||||
def check_vulkan(report):
|
||||
devices = vulkan_devices()
|
||||
if devices is None and shutil.which('vulkaninfo'):
|
||||
report('vulkan', False, 'vulkaninfo did not finish',
|
||||
'vulkaninfo is installed but exited with an error or timed out. Run '
|
||||
'`vulkaninfo --summary` for the loader or ICD error, then fix or reinstall the '
|
||||
'driver: the runtime needs Vulkan 1.3 and timeline semaphores.')
|
||||
return
|
||||
if devices is None:
|
||||
report('vulkan', False, 'vulkaninfo not available',
|
||||
'Install vulkan-tools to check the device. The runtime needs Vulkan 1.3 and '
|
||||
'timeline semaphores; without a suitable device the game cannot render.',
|
||||
required=False)
|
||||
return
|
||||
if not devices:
|
||||
report('vulkan', False, 'no devices reported',
|
||||
'`vulkaninfo --summary` listed no Vulkan device. Check the loader and the ICD '
|
||||
'files, or reinstall the driver: the game cannot render without a device.')
|
||||
return
|
||||
# A dump that carries no feature section still gets the version check, as before.
|
||||
checked = any(device['timeline'] is not None for device in devices)
|
||||
usable = [device for device in devices
|
||||
if device['api'] is not None and device['api'] >= MIN_VULKAN
|
||||
and (device['timeline'] or not checked)]
|
||||
report('vulkan', bool(usable), '; '.join(describe_device(device) for device in devices),
|
||||
f'Vulkan {dotted(MIN_VULKAN)} or newer and timeline semaphores on the same device are '
|
||||
'required: Indium rejects older devices and uses the semaphores. Update Mesa or the '
|
||||
'vendor driver, or check `vulkaninfo`.')
|
||||
|
||||
|
||||
def check_display(report):
|
||||
display = os.environ.get('DISPLAY')
|
||||
if not display:
|
||||
report('x11', False, 'DISPLAY unset',
|
||||
'Without X11/Xwayland the runtime boots but the game cannot present; expected on '
|
||||
'a headless host that is only used to build.', required=False)
|
||||
return
|
||||
if not shutil.which('xdpyinfo'):
|
||||
report('x11', False, f'DISPLAY={display}, xdpyinfo not installed',
|
||||
'Install x11-utils to check the display before a launch.', required=False)
|
||||
return
|
||||
if output(['xdpyinfo'], timeout=15) is None:
|
||||
report('x11', False, f'DISPLAY={display}',
|
||||
f'Cannot open {display}. On servers, SSH and sudo sessions export XAUTHORITY '
|
||||
'with a cookie the user running Darling can read; otherwise iokitd runs without '
|
||||
'X11/EGL and the game cannot render. Builds still work without it.',
|
||||
required=False)
|
||||
return
|
||||
vendor = ''
|
||||
for probe in (['glxinfo', '-B'], ['eglinfo']):
|
||||
text = output(probe, timeout=20)
|
||||
if text:
|
||||
found = re.search(r'(?:OpenGL vendor string|EGL_VENDOR):\s*(.+)', text)
|
||||
if found:
|
||||
vendor = f', {probe[0]} vendor {found.group(1).strip()}'
|
||||
break
|
||||
report('x11', True, f'DISPLAY={display} (xdpyinfo){vendor}')
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--json', action='store_true', help='machine-readable output')
|
||||
args = parser.parse_args()
|
||||
|
||||
checks = []
|
||||
|
||||
def report(name, ok, detail, hint=None, required=True):
|
||||
checks.append({'check': name, 'ok': bool(ok), 'required': required,
|
||||
'detail': detail, 'hint': None if ok else hint})
|
||||
|
||||
for check in (check_python, check_cmake, check_git_lfs, check_lfs_pointers, check_kernel,
|
||||
check_llvm, check_llvm_tools, check_vulkan, check_display):
|
||||
check(report)
|
||||
|
||||
failed = [check for check in checks if check['required'] and not check['ok']]
|
||||
warned = [check for check in checks if not check['required'] and not check['ok']]
|
||||
passed = [check for check in checks if check['ok']]
|
||||
if args.json:
|
||||
print(json.dumps({'ok': not failed, 'checks': checks}, indent=2))
|
||||
else:
|
||||
print('preflight: host checks the build and runtime need')
|
||||
for check in checks:
|
||||
state = 'ok ' if check['ok'] else ('FAIL' if check['required'] else 'warn')
|
||||
print(f' {state} {check["check"]:<13} {check["detail"]}')
|
||||
if check['hint'] and not check['ok']:
|
||||
print(f' {check["hint"]}')
|
||||
summary = f'preflight: {len(passed)} checks passed'
|
||||
if warned:
|
||||
summary += f', {len(warned)} warning' + ('s' if len(warned) != 1 else '')
|
||||
if failed:
|
||||
summary += (f', {len(failed)} required check'
|
||||
+ ('s' if len(failed) != 1 else '') + ' failed')
|
||||
print(summary)
|
||||
if failed:
|
||||
if not args.json:
|
||||
print('preflight: fix the required checks above, or set RCLOL_SKIP_PREFLIGHT=1 to bypass',
|
||||
file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,377 @@
|
||||
"""Preflight and LLVM tool discovery, with fake host tools and no Darling runtime."""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
PREFLIGHT = ROOT / 'scripts/preflight.py'
|
||||
KERNEL = '6.1.0-test'
|
||||
|
||||
|
||||
def load(name, path):
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class PreflightFixture(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory(prefix='preflight fixture ', dir=os.environ.get('TMPDIR'))
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.bin = Path(self.tmp.name) / 'bin'
|
||||
self.bin.mkdir()
|
||||
self.llvm_bindir = Path(self.tmp.name) / 'llvm bin'
|
||||
self.llvm_bindir.mkdir()
|
||||
self.good()
|
||||
|
||||
def good(self):
|
||||
self.stub('llvm-config', '''case "$1" in
|
||||
--version) echo 23.1.5;;
|
||||
--bindir) printf '%s\\n' "LLVM_BINDIR";;
|
||||
esac'''.replace('LLVM_BINDIR', str(self.llvm_bindir)))
|
||||
self.stub('llvm-objdump', 'exit 0')
|
||||
self.stub('llvm-nm', 'exit 0')
|
||||
self.stub('cmake', 'echo "cmake version 3.31.6"')
|
||||
self.stub('git', 'case "$*" in "lfs version") echo "git-lfs/3.7.0 (GitHub; linux amd64)";; esac')
|
||||
self.vulkan([('1.3.289', 'Test GPU', True)])
|
||||
self.stub('xdpyinfo', 'exit 0')
|
||||
|
||||
def vulkan(self, devices):
|
||||
"""Stub vulkaninfo with [(apiVersion, name, timeline)] per device."""
|
||||
summary = ['Devices:', '========']
|
||||
features = ['Device Properties and Extensions:', '=================================']
|
||||
for index, (api, name, timeline) in enumerate(devices):
|
||||
summary += [f'GPU{index}:', f'\tapiVersion = {api}', f'\tdeviceName = {name}']
|
||||
features += [f'GPU{index}:', 'VkPhysicalDeviceProperties:',
|
||||
f'\tapiVersion = {api}', f'\tdeviceName = {name}']
|
||||
if timeline is not None:
|
||||
features += ['VkPhysicalDeviceVulkan12Features:',
|
||||
f'\ttimelineSemaphore = {"true" if timeline else "false"}']
|
||||
self.stub('vulkaninfo', 'case "$*" in\n'
|
||||
" --summary) printf '%s\\n' " + ' '.join(f"'{line}'" for line in summary) + ';;\n'
|
||||
" *) printf '%s\\n' " + ' '.join(f"'{line}'" for line in features) + ';;\nesac')
|
||||
|
||||
def stub(self, name, body, folder=None):
|
||||
path = (folder or self.bin) / name
|
||||
path.write_text('#!/bin/sh\nset -eu\n' + body + '\n')
|
||||
path.chmod(0o755)
|
||||
return path
|
||||
|
||||
def preflight(self, *arguments, kernel=KERNEL, display=':0'):
|
||||
env = dict(os.environ, PATH=str(self.bin),
|
||||
RCLOL_PREFLIGHT_KERNEL_RELEASE=kernel,
|
||||
RCLOL_PREFLIGHT_SOURCE=str(Path(self.tmp.name) / 'not fetched'))
|
||||
if display is None:
|
||||
env.pop('DISPLAY', None)
|
||||
else:
|
||||
env['DISPLAY'] = display
|
||||
return subprocess.run([sys.executable, str(PREFLIGHT), *arguments], env=env,
|
||||
text=True, capture_output=True, timeout=120)
|
||||
|
||||
def test_all_required_checks_pass(self):
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('preflight: ', result.stdout)
|
||||
self.assertNotIn('FAIL', result.stdout)
|
||||
self.assertIn('ok llvm-tools', result.stdout)
|
||||
|
||||
|
||||
def test_llvm_15_stops_with_the_reason(self):
|
||||
self.stub('llvm-config', 'echo 15.0.7')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL llvm', result.stdout)
|
||||
self.assertIn('LLVM 16 or newer is required', result.stdout)
|
||||
self.assertIn('LLVMGetVersion', result.stdout)
|
||||
self.assertIn('RCLOL_SKIP_PREFLIGHT=1', result.stderr)
|
||||
|
||||
|
||||
def test_untested_llvm_major_warns_but_builds(self):
|
||||
self.stub('llvm-config', 'case "$1" in --version) echo 17.0.6;; esac')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('warn llvm-tested', result.stdout)
|
||||
self.assertIn('1 warning', result.stdout)
|
||||
|
||||
|
||||
def test_versioned_llvm_tools_are_found(self):
|
||||
(self.bin / 'llvm-objdump').unlink()
|
||||
(self.bin / 'llvm-nm').unlink()
|
||||
self.stub('llvm-objdump-15', 'exit 0')
|
||||
self.stub('llvm-nm-15', 'exit 0')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('llvm-objdump-15', result.stdout)
|
||||
self.assertIn('llvm-nm-15', result.stdout)
|
||||
|
||||
|
||||
def test_llvm_tools_come_from_llvm_config_bindir(self):
|
||||
(self.bin / 'llvm-objdump').unlink()
|
||||
(self.bin / 'llvm-nm').unlink()
|
||||
self.stub('llvm-objdump-23', 'exit 0', folder=self.llvm_bindir)
|
||||
self.stub('llvm-nm-23', 'exit 0', folder=self.llvm_bindir)
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('llvm bin', result.stdout)
|
||||
|
||||
|
||||
def test_missing_llvm_tool_names_what_to_install(self):
|
||||
(self.bin / 'llvm-nm').unlink()
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL llvm-tools', result.stdout)
|
||||
self.assertIn('llvm-nm not found', result.stdout)
|
||||
self.assertIn('llvm23-tools', result.stdout)
|
||||
|
||||
|
||||
def test_missing_llvm_config_names_the_package(self):
|
||||
(self.bin / 'llvm-config').unlink()
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL llvm', result.stdout)
|
||||
self.assertIn('llvm-config not found', result.stdout)
|
||||
self.assertIn('llvm-dev', result.stdout)
|
||||
|
||||
|
||||
def test_old_cmake_stops_the_setup(self):
|
||||
self.stub('cmake', 'echo "cmake version 3.16.3"')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL cmake', result.stdout)
|
||||
self.assertIn('CMake 3.20', result.stdout)
|
||||
|
||||
|
||||
def test_old_git_lfs_stops_the_setup(self):
|
||||
self.stub('git', 'case "$*" in "lfs version") echo "git-lfs/2.9.2 (GitHub; linux amd64)";; esac')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL git-lfs', result.stdout)
|
||||
self.assertIn('Git LFS 2.13', result.stdout)
|
||||
|
||||
|
||||
def test_old_kernel_is_required(self):
|
||||
result = self.preflight(kernel='5.10.0-28-amd64')
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL kernel', result.stdout)
|
||||
self.assertIn('Linux 5.14', result.stdout)
|
||||
|
||||
|
||||
def test_old_vulkan_stops_the_setup(self):
|
||||
self.vulkan([('1.2.170', 'Old GPU', True)])
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL vulkan', result.stdout)
|
||||
self.assertIn('Vulkan 1.3 or newer', result.stdout)
|
||||
self.assertIn('Old GPU 1.2', result.stdout)
|
||||
|
||||
|
||||
def test_missing_vulkaninfo_only_warns(self):
|
||||
(self.bin / 'vulkaninfo').unlink()
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('warn vulkan', result.stdout)
|
||||
self.assertIn('vulkaninfo not available', result.stdout)
|
||||
self.assertIn('Install vulkan-tools', result.stdout)
|
||||
|
||||
|
||||
def test_broken_vulkaninfo_is_required(self):
|
||||
self.stub('vulkaninfo', 'exit 1')
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL vulkan', result.stdout)
|
||||
self.assertIn('vulkaninfo did not finish', result.stdout)
|
||||
self.assertIn('timed out', result.stdout)
|
||||
|
||||
|
||||
def test_missing_timeline_semaphores_is_reported(self):
|
||||
self.vulkan([('1.3.289', 'Test GPU', False)])
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL vulkan', result.stdout)
|
||||
self.assertIn('timelineSemaphore false', result.stdout)
|
||||
self.assertIn('on the same device', result.stdout)
|
||||
|
||||
|
||||
def test_a_second_device_can_satisfy_both_requirements(self):
|
||||
self.vulkan([('1.4.309', 'New GPU without semaphores', False),
|
||||
('1.4.309', 'Second GPU', True)])
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('ok vulkan', result.stdout)
|
||||
|
||||
|
||||
def test_requirements_are_not_combined_across_devices(self):
|
||||
self.vulkan([('1.2.170', 'Old GPU with semaphores', True),
|
||||
('1.4.309', 'New GPU without semaphores', False)])
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL vulkan', result.stdout)
|
||||
self.assertIn('Old GPU with semaphores 1.2', result.stdout)
|
||||
self.assertIn('New GPU without semaphores 1.4', result.stdout)
|
||||
|
||||
|
||||
def test_no_reported_device_is_required(self):
|
||||
self.vulkan([])
|
||||
result = self.preflight()
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn('FAIL vulkan', result.stdout)
|
||||
self.assertIn('no devices reported', result.stdout)
|
||||
|
||||
|
||||
def test_unreachable_display_only_warns(self):
|
||||
self.stub('xdpyinfo', 'exit 1')
|
||||
result = self.preflight(display=':99')
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('warn x11', result.stdout)
|
||||
self.assertIn('XAUTHORITY', result.stdout)
|
||||
|
||||
|
||||
def test_headless_host_only_warns(self):
|
||||
result = self.preflight(display=None)
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn('DISPLAY unset', result.stdout)
|
||||
|
||||
|
||||
def test_json_report_names_every_check(self):
|
||||
result = self.preflight('--json')
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
report = json.loads(result.stdout)
|
||||
self.assertTrue(report['ok'])
|
||||
names = {check['check'] for check in report['checks']}
|
||||
self.assertTrue({'python', 'cmake', 'git-lfs', 'lfs-pointer', 'kernel', 'llvm',
|
||||
'llvm-tools', 'vulkan', 'x11'} <= names)
|
||||
self.assertTrue(all('required' in check and 'detail' in check for check in report['checks']))
|
||||
|
||||
|
||||
class LfsPointerFixture(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory(prefix='lfs pointer ', dir=os.environ.get('TMPDIR'))
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.source = Path(self.tmp.name) / 'darling-src'
|
||||
self.source.mkdir()
|
||||
self.preflight = load('preflight_lfs', PREFLIGHT)
|
||||
self.preflight.SOURCE = self.source
|
||||
|
||||
def check(self):
|
||||
reports = []
|
||||
self.preflight.check_lfs_pointers(lambda *arguments, **keywords: reports.append(
|
||||
(arguments, keywords)))
|
||||
self.assertEqual(len(reports), 1)
|
||||
arguments, keywords = reports[0]
|
||||
name, ok, detail = arguments[:3]
|
||||
hint = arguments[3] if len(arguments) > 3 else keywords.get('hint')
|
||||
return name, ok, detail, hint
|
||||
|
||||
def write(self, name, content):
|
||||
path = self.source / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(content)
|
||||
return path
|
||||
|
||||
def test_pointer_text_in_a_library_is_reported(self):
|
||||
self.write('lib/libswiftCore.dylib',
|
||||
b'version https://git-lfs.github.com/spec/v1\noid sha256:abc\nsize 123\n')
|
||||
self.write('lib/libreal.dylib', b'\xcf\xfa\xed\xfe' + b'\0' * 64)
|
||||
name, ok, detail, hint = self.check()
|
||||
self.assertEqual(name, 'lfs-pointer')
|
||||
self.assertFalse(ok)
|
||||
self.assertIn('lib/libswiftCore.dylib', detail)
|
||||
self.assertNotIn('libreal', detail)
|
||||
self.assertIn('git lfs pull', hint)
|
||||
|
||||
def test_pointer_text_outside_a_library_is_ignored(self):
|
||||
self.write('notes.dylib.txt', b'version https://git-lfs.github.com/spec/v1\n')
|
||||
self.write('lib/libreal.dylib', b'not a pointer')
|
||||
_, ok, detail, _ = self.check()
|
||||
self.assertTrue(ok, detail)
|
||||
|
||||
def test_missing_checkout_is_not_a_failure(self):
|
||||
self.preflight.SOURCE = self.source / 'not fetched'
|
||||
_, ok, detail, _ = self.check()
|
||||
self.assertTrue(ok)
|
||||
self.assertIn('no prepared checkout', detail)
|
||||
|
||||
|
||||
class LlvmToolsFixture(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory(prefix='llvm tools ', dir=os.environ.get('TMPDIR'))
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.bin = Path(self.tmp.name) / 'bin'
|
||||
self.bin.mkdir()
|
||||
self.saved = os.environ.get('PATH')
|
||||
os.environ['PATH'] = str(self.bin)
|
||||
self.addCleanup(self.restore)
|
||||
self.tools = load('llvm_tools', ROOT / 'tools/llvm_tools.py')
|
||||
|
||||
def restore(self):
|
||||
os.environ['PATH'] = self.saved or ''
|
||||
|
||||
def stub(self, name, body='exit 0', folder=None):
|
||||
path = (folder or self.bin) / name
|
||||
path.write_text('#!/bin/sh\n' + body + '\n')
|
||||
path.chmod(0o755)
|
||||
return path
|
||||
|
||||
def test_prefers_the_unversioned_name(self):
|
||||
unversioned = self.stub('llvm-objdump')
|
||||
self.stub('llvm-objdump-23')
|
||||
self.assertEqual(self.tools.find('llvm-objdump'), str(unversioned))
|
||||
|
||||
|
||||
def test_prefers_the_highest_versioned_name(self):
|
||||
self.stub('llvm-objdump-15')
|
||||
newest = self.stub('llvm-objdump-23')
|
||||
self.stub('llvm-objdump-9')
|
||||
self.assertEqual(self.tools.find('llvm-objdump'), str(newest))
|
||||
|
||||
|
||||
def test_ignores_a_name_that_is_not_a_version(self):
|
||||
self.stub('llvm-objdump-tools')
|
||||
self.assertIsNone(self.tools.find('llvm-objdump'))
|
||||
|
||||
|
||||
def test_falls_back_to_llvm_config_bindir(self):
|
||||
libexec = Path(self.tmp.name) / 'libexec'
|
||||
libexec.mkdir()
|
||||
expected = self.stub('llvm-nm-22', folder=libexec)
|
||||
self.stub('llvm-config', 'echo "%s"' % libexec)
|
||||
self.assertEqual(self.tools.find('llvm-nm'), str(expected))
|
||||
|
||||
|
||||
def test_llvm_config_timeout_is_reported(self):
|
||||
self.stub('llvm-config', 'exec /bin/sleep 5')
|
||||
self.tools.TIMEOUT = 1
|
||||
with self.assertRaises(SystemExit) as raised:
|
||||
self.tools.config('--version')
|
||||
self.assertIn('did not finish within 1 seconds', str(raised.exception))
|
||||
self.assertIn('reinstall the LLVM development package', str(raised.exception))
|
||||
|
||||
|
||||
def test_llvm_config_timeout_does_not_hang_the_bindir_probe(self):
|
||||
self.stub('llvm-config', 'exec /bin/sleep 5')
|
||||
self.tools.TIMEOUT = 1
|
||||
self.assertIsNone(self.tools.find('llvm-objdump'))
|
||||
|
||||
|
||||
def test_require_explains_what_to_install(self):
|
||||
with self.assertRaises(SystemExit) as raised:
|
||||
self.tools.require('llvm-objdump')
|
||||
self.assertIn('llvm-objdump not found', str(raised.exception))
|
||||
self.assertIn('llvm-config --bindir', str(raised.exception))
|
||||
|
||||
|
||||
class FileDigestFixture(unittest.TestCase):
|
||||
def test_sha256_matches_hashlib(self):
|
||||
digest = load('file_digest', ROOT / 'tools/file_digest.py')
|
||||
with tempfile.TemporaryDirectory(prefix='digest ', dir=os.environ.get('TMPDIR')) as folder:
|
||||
path = Path(folder) / 'payload'
|
||||
payload = b'darling' * 400_000 # More than one chunk.
|
||||
path.write_bytes(payload)
|
||||
self.assertEqual(digest.sha256(path), hashlib.sha256(payload).hexdigest())
|
||||
@@ -13,6 +13,12 @@ SCRIPT = Path(__file__).resolve().parents[1] / 'tools/ensure-darling-runtime.py'
|
||||
|
||||
|
||||
class RuntimeCleanupScopeTests(unittest.TestCase):
|
||||
def enterContext(self, manager):
|
||||
"""unittest.TestCase.enterContext() is Python 3.11+; Ubuntu 22.04 ships 3.10."""
|
||||
value = manager.__enter__()
|
||||
self.addCleanup(manager.__exit__, None, None, None)
|
||||
return value
|
||||
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory(prefix='cleanup-scope-')
|
||||
self.addCleanup(temporary.cleanup)
|
||||
|
||||
@@ -65,7 +65,11 @@ class StopRuntimeTests(unittest.TestCase):
|
||||
def __init__(self, *parts, **kwargs):
|
||||
if parts and str(parts[0]) == '/proc':
|
||||
parts = (proc, *parts[1:])
|
||||
super().__init__(*parts, **kwargs)
|
||||
try:
|
||||
super().__init__(*parts, **kwargs)
|
||||
except TypeError:
|
||||
# Python 3.10's PurePath.__init__ takes no arguments; 3.12 stores them.
|
||||
super().__init__()
|
||||
|
||||
def read_text(self, *args, **kwargs):
|
||||
if self.name == 'status' and owner.status_error is not None:
|
||||
|
||||
@@ -4,12 +4,15 @@ from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
if '--native' in sys.argv:
|
||||
indium = root / 'build/darling-src/src/external/metal/deps/indium'
|
||||
llvm_include = subprocess.check_output(['llvm-config', '--includedir'], text=True).strip()
|
||||
llvm_libdir = Path(subprocess.check_output(['llvm-config', '--libdir'], text=True).strip())
|
||||
llvm_names = shlex.split(subprocess.check_output(['llvm-config', '--link-shared', '--libnames'], text=True))
|
||||
llvm_include = llvm_tools.config('--includedir')
|
||||
llvm_libdir = Path(llvm_tools.config('--libdir'))
|
||||
llvm_names = shlex.split(llvm_tools.config('--link-shared', '--libnames'))
|
||||
if len(llvm_names) != 1:
|
||||
raise SystemExit('Iridium requires one shared LLVM library, got: ' + repr(llvm_names))
|
||||
llvm_library = llvm_libdir / llvm_names[0]
|
||||
|
||||
@@ -5,6 +5,9 @@ import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
folder = root / 'build/darling-src/build/src/external/metal/deps/indium'
|
||||
# Keep the validated LLVM major locally when a system update replaces it.
|
||||
@@ -24,9 +27,9 @@ if (pinned_llvm / 'lib/libLLVM-22.so').exists() and os.environ.get('RIOT_IRIDIUM
|
||||
llvm_library = pinned_llvm / 'lib/libLLVM-22.so'
|
||||
else:
|
||||
# Do not combine cached CMake include paths with a newer llvm-config library.
|
||||
llvm_include = Path(subprocess.check_output(['llvm-config', '--includedir'], text=True).strip())
|
||||
llvm_libdir = Path(subprocess.check_output(['llvm-config', '--libdir'], text=True).strip())
|
||||
llvm_names = shlex.split(subprocess.check_output(['llvm-config', '--link-shared', '--libnames'], text=True))
|
||||
llvm_include = Path(llvm_tools.config('--includedir'))
|
||||
llvm_libdir = Path(llvm_tools.config('--libdir'))
|
||||
llvm_names = shlex.split(llvm_tools.config('--link-shared', '--libnames'))
|
||||
if len(llvm_names) != 1:
|
||||
raise SystemExit('Iridium requires one shared LLVM library, got: ' + repr(llvm_names))
|
||||
llvm_library = llvm_libdir / llvm_names[0]
|
||||
|
||||
@@ -7,13 +7,17 @@ through it, leaving %rdi unrelated: the thunk then returns NULL and the first wr
|
||||
Usage: check-darwin-tlv-calls.py BINARY... ; exits 1 and lists every bad access it finds.
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
|
||||
|
||||
def thread_vars_ranges(path):
|
||||
"""Address ranges of the __thread_vars sections, which hold the TLV descriptors."""
|
||||
output = subprocess.run(['llvm-objdump', '--macho', '--section-headers', path],
|
||||
output = subprocess.run([llvm_tools.require('llvm-objdump'), '--macho', '--section-headers', path],
|
||||
capture_output=True, text=True, check=True).stdout
|
||||
ranges = []
|
||||
for line in output.splitlines():
|
||||
@@ -31,7 +35,7 @@ def bad_accesses(path):
|
||||
ranges = thread_vars_ranges(path)
|
||||
if not ranges:
|
||||
return []
|
||||
disassembly = subprocess.run(['llvm-objdump', '-d', '--no-show-raw-insn', path],
|
||||
disassembly = subprocess.run([llvm_tools.require('llvm-objdump'), '-d', '--no-show-raw-insn', path],
|
||||
capture_output=True, text=True, check=True).stdout
|
||||
load = re.compile(r'(?:leaq|movq)\s+-?0x[0-9a-f]+\(%rip\), %(\w+)\s+#+ 0x([0-9a-f]+)')
|
||||
found, function, pending = [], '?', {}
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
"""SHA-256 of a file without hashlib.file_digest().
|
||||
|
||||
hashlib.file_digest() is Python 3.11+; Ubuntu 22.04 ships Python 3.10, where the
|
||||
installers failed only after the build had finished (issue #137). This reads in
|
||||
chunks, so the memory use does not depend on the file size.
|
||||
"""
|
||||
import hashlib
|
||||
|
||||
|
||||
def sha256(path):
|
||||
checksum = hashlib.sha256()
|
||||
with open(path, 'rb') as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b''):
|
||||
checksum.update(chunk)
|
||||
return checksum.hexdigest()
|
||||
@@ -1,7 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Atomically install a built Darling server; never stop or start any process."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -9,12 +8,13 @@ import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
|
||||
from file_digest import sha256
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def digest(path):
|
||||
with path.open("rb") as input:
|
||||
return hashlib.file_digest(input, "sha256").hexdigest()
|
||||
return sha256(path)
|
||||
|
||||
|
||||
def live_binaries(destination):
|
||||
|
||||
@@ -9,7 +9,6 @@ tools/install-riot-memory-runtime.py (its default source is the rebuilt
|
||||
build/riot-libsystem_kernel.dylib); this installer does not touch it.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -18,14 +17,14 @@ import stat
|
||||
import tempfile
|
||||
|
||||
from runtime_paths import runtime_paths
|
||||
from file_digest import sha256
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
DEFAULT_ARTIFACT = ROOT / "build/riot-Foundation"
|
||||
|
||||
|
||||
def digest(path):
|
||||
with path.open("rb") as input:
|
||||
return hashlib.file_digest(input, "sha256").hexdigest()
|
||||
return sha256(path)
|
||||
|
||||
|
||||
def copy_framework(sdk_framework, framework):
|
||||
|
||||
@@ -15,7 +15,6 @@ and is never run against the real prefix by the build. Nothing is activated
|
||||
until the guest processes are restarted.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -24,6 +23,7 @@ import stat
|
||||
import tempfile
|
||||
|
||||
from runtime_paths import validate_runtime_paths
|
||||
from file_digest import sha256
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
@@ -40,8 +40,7 @@ LIBRARIES = [
|
||||
|
||||
|
||||
def digest(path):
|
||||
with path.open("rb") as input:
|
||||
return hashlib.file_digest(input, "sha256").hexdigest()
|
||||
return sha256(path)
|
||||
|
||||
|
||||
def destinations(prefix, guest_home):
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Install the tested reclamation library atomically, without restarting clients."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -11,6 +10,7 @@ import stat
|
||||
import tempfile
|
||||
|
||||
from runtime_paths import runtime_paths
|
||||
from file_digest import sha256
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
# The optimized relink from tools/build-riot-kqueue-runtime.py is the library the
|
||||
@@ -19,8 +19,7 @@ DEFAULT_ARTIFACT = ROOT / "build/riot-libsystem_kernel.dylib"
|
||||
|
||||
|
||||
def digest(path):
|
||||
with path.open("rb") as input:
|
||||
return hashlib.file_digest(input, "sha256").hexdigest()
|
||||
return sha256(path)
|
||||
|
||||
|
||||
def install(binary, destinations, backup_folder):
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Find LLVM command line tools, including versioned Debian/Ubuntu names.
|
||||
|
||||
Ubuntu's llvm-<N>-dev installs llvm-objdump and llvm-nm in /usr/lib/llvm-<N>/bin
|
||||
without an unversioned name in PATH, so a bare ['llvm-objdump'] run fails on an
|
||||
otherwise complete host (issue #138). Resolution order: an unversioned tool on
|
||||
PATH, then the highest versioned name on PATH, then the directory llvm-config
|
||||
reports. Nothing here runs a guest process.
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
TIMEOUT = 30 # A probe that hangs must not stall the preflight or the build.
|
||||
|
||||
|
||||
def _collect(directory, name, found):
|
||||
"""Add every `name` and `name-<digits>` executable in the directory to found."""
|
||||
try:
|
||||
entries = os.listdir(directory)
|
||||
except OSError:
|
||||
return
|
||||
for entry in entries:
|
||||
suffix = entry[len(name) + 1:]
|
||||
if entry == name:
|
||||
version = 0 # An unversioned name always wins.
|
||||
elif entry.startswith(name + '-') and suffix.isdigit():
|
||||
version = int(suffix)
|
||||
else:
|
||||
continue
|
||||
path = os.path.join(directory or '.', entry)
|
||||
if os.path.isfile(path) and os.access(path, os.X_OK):
|
||||
found.setdefault(version, path)
|
||||
|
||||
|
||||
def _executables(name):
|
||||
"""{version: path} for `name` and for every `name-<digits>` on PATH."""
|
||||
found = {}
|
||||
for folder in os.environ.get('PATH', '').split(os.pathsep):
|
||||
_collect(folder or '.', name, found)
|
||||
return found
|
||||
|
||||
|
||||
def _preferred(candidates):
|
||||
return candidates[0 if 0 in candidates else max(candidates)] if candidates else None
|
||||
|
||||
|
||||
def _run(command):
|
||||
"""Run a short probe; return (completed process, problem), one of them None."""
|
||||
try:
|
||||
result = subprocess.run(command, capture_output=True, text=True, errors='replace',
|
||||
timeout=TIMEOUT)
|
||||
except subprocess.TimeoutExpired:
|
||||
return None, f'did not finish within {TIMEOUT} seconds'
|
||||
except OSError as error:
|
||||
return None, f'could not be started: {error}'
|
||||
return result, None
|
||||
|
||||
|
||||
def _bindir():
|
||||
"""`llvm-config --bindir` from the preferred llvm-config, or None."""
|
||||
configuration = _preferred(_executables('llvm-config'))
|
||||
if not configuration:
|
||||
return None
|
||||
result, _ = _run([configuration, '--bindir'])
|
||||
if result is None:
|
||||
return None
|
||||
directory = result.stdout.strip()
|
||||
if result.returncode or not os.path.isdir(directory):
|
||||
return None
|
||||
return directory
|
||||
|
||||
|
||||
def find(name):
|
||||
"""Path of an LLVM tool, or None when this host does not have it."""
|
||||
candidates = _executables(name)
|
||||
if candidates:
|
||||
return _preferred(candidates)
|
||||
directory = _bindir()
|
||||
if directory:
|
||||
in_bindir = {}
|
||||
_collect(directory, name, in_bindir)
|
||||
if in_bindir:
|
||||
return _preferred(in_bindir)
|
||||
return None
|
||||
|
||||
|
||||
def require(name):
|
||||
"""Path of an LLVM tool, or exit with what was tried and what to install."""
|
||||
path = find(name)
|
||||
if path:
|
||||
return path
|
||||
raise SystemExit(f'{name} not found: looked for an unversioned {name}, a versioned '
|
||||
f'{name}-<N> in PATH and `llvm-config --bindir`. Install the LLVM tools '
|
||||
f'package (for example llvm-dev, llvm15-tools or llvm23-tools).')
|
||||
|
||||
|
||||
def config(*arguments):
|
||||
"""Run llvm-config with the arguments, or exit with an actionable message."""
|
||||
path = require('llvm-config')
|
||||
result, problem = _run([path, *arguments])
|
||||
called = f'{path} {" ".join(arguments)}'
|
||||
if problem:
|
||||
raise SystemExit(f'{called} {problem}; reinstall the LLVM development package.')
|
||||
if result.returncode:
|
||||
raise SystemExit(f'{called} failed: {result.stderr.strip()}')
|
||||
return result.stdout.strip()
|
||||
@@ -3,11 +3,15 @@
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
indium = root / 'build/darling-src/src/external/metal/deps/indium'
|
||||
include = Path(subprocess.check_output(['llvm-config', '--includedir'], text=True).strip())
|
||||
include = Path(llvm_tools.config('--includedir'))
|
||||
header = (include / 'llvm/Config/llvm-config.h').read_text()
|
||||
major = int(re.search(r'^#define LLVM_VERSION_MAJOR (\d+)', header, re.MULTILINE)[1])
|
||||
with tempfile.TemporaryDirectory(prefix='iridium-llvm-version-') as directory:
|
||||
|
||||
@@ -23,6 +23,9 @@ import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
GAME = (Path.home() / ".darling/Applications/League of Legends.app/Contents/LoL/Game"
|
||||
/ "LeagueofLegends.app/Contents/MacOS/LeagueofLegends")
|
||||
@@ -44,7 +47,9 @@ MIN_FUNCTION_INSTRUCTIONS = 20
|
||||
|
||||
|
||||
def tool(*arguments):
|
||||
return subprocess.run(arguments, capture_output=True, text=True, errors="replace").stdout
|
||||
"""Run an LLVM tool, resolving versioned names from PATH or llvm-config."""
|
||||
return subprocess.run((llvm_tools.require(arguments[0]), *arguments[1:]),
|
||||
capture_output=True, text=True, errors="replace").stdout
|
||||
|
||||
|
||||
def load_commands(path, requested):
|
||||
|
||||
@@ -16,6 +16,9 @@ import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import llvm_tools
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@@ -27,11 +30,16 @@ def load_audit():
|
||||
return module
|
||||
|
||||
|
||||
def headers(path):
|
||||
return subprocess.run(["llvm-objdump", "--macho", "--private-headers", str(path)],
|
||||
def run(*arguments):
|
||||
"""Run an LLVM tool, resolving versioned names from PATH or llvm-config."""
|
||||
return subprocess.run((llvm_tools.require(arguments[0]), *arguments[1:]),
|
||||
capture_output=True, text=True, errors="replace").stdout
|
||||
|
||||
|
||||
def headers(path):
|
||||
return run("llvm-objdump", "--macho", "--private-headers", str(path))
|
||||
|
||||
|
||||
def identity(path):
|
||||
"""Install name and the two versions from LC_ID_DYLIB."""
|
||||
text = headers(path)
|
||||
@@ -43,8 +51,7 @@ def identity(path):
|
||||
|
||||
|
||||
def dependencies(path):
|
||||
text = subprocess.run(["llvm-objdump", "--macho", "--dylibs-used", str(path)],
|
||||
capture_output=True, text=True, errors="replace").stdout
|
||||
text = run("llvm-objdump", "--macho", "--dylibs-used", str(path))
|
||||
return sorted(re.findall(r"^\t(\S+) \(compatibility version", text, re.M))
|
||||
|
||||
|
||||
@@ -70,8 +77,7 @@ def weak_definitions(path):
|
||||
"""Symbols the library defines only weakly: the -O0 build emits every
|
||||
inline function out of line, an optimized build may drop the unused ones,
|
||||
and the ABI makes a weak definition optional."""
|
||||
text = subprocess.run(["llvm-nm", "-m", str(path)], capture_output=True, text=True,
|
||||
errors="replace").stdout
|
||||
text = run("llvm-nm", "-m", str(path))
|
||||
weak = set()
|
||||
for line in text.splitlines():
|
||||
if "weak external" in line:
|
||||
@@ -80,8 +86,7 @@ def weak_definitions(path):
|
||||
|
||||
|
||||
def symbols(path, flag):
|
||||
text = subprocess.run(["llvm-nm", flag, str(path)], capture_output=True, text=True,
|
||||
errors="replace").stdout
|
||||
text = run("llvm-nm", flag, str(path))
|
||||
found = set()
|
||||
for line in text.splitlines():
|
||||
parts = line.split()
|
||||
|
||||
Reference in new issue
Block a user