Merge pull request #12621 from Ragbie/fix/windows-defer-needed-runtime-modules

fix(relinker): defer the runtime modules another deferred module needs on Windows

core/relinker/relinker/src/guest/GuestModuleBuilder.cpp

@@ -317,7 +317,7 @@ std::vector<GuestArtifact> GuestModuleBuilder::Build(const std::filesystem::path
     for (const auto provider : executableProviders) require(provider);
     for (std::size_t index = 0; index < images.size(); ++index) {
         if (!isRuntime(index)) require(index);
-        if (windows) for (const auto dependency : dependencies[index]) require(dependency);
+        if (windows) for (const auto dependency : dependencies[index]) if (standardDirectory(dependency)) require(dependency);
     }
     const auto deferred = [&](const std::size_t index) {
         return !required[index] && isRuntime(index);

core/relinker/relinker/tests/ConversionTests.cpp

@@ -372,6 +372,21 @@ void runtimeDirectories() {
         require(output.Modules.size() == 1, "Module was not converted");
         require(exports(output.Modules.front().Bytes, Relinker::GuestInitializeExport) == (std::string_view(directory) == "Media/Modules"), "Wrong start of a runtime module the executable needs");
     }
+    for (const auto* directory : {"sce_module", "sce_modules", "prx", "Media/Modules"}) {
+        auto source = input();
+        const auto parent = source.ModuleRoot / directory;
+        const auto plugins = source.ModuleRoot / "Media/Plugins";
+        source.Modules.push_back({plugins / "plugin.prx", fixture("needed.prx")});
+        source.Modules.push_back({parent / "needed.prx", fixture()});
+        source.RuntimeDirectories.push_back(plugins);
+        source.RuntimeDirectories.push_back(parent);
+        const auto output = converter.Convert(std::move(source), options);
+        require(output.Modules.size() == 2, "Modules were not converted");
+        for (const auto& module : output.Modules) {
+            const bool plugin = module.SourcePath.filename() == "plugin.prx";
+            require(exports(module.Bytes, Relinker::GuestInitializeExport) == (plugin || std::string_view(directory) == "Media/Modules"), "Wrong start of a runtime module another runtime module needs");
+        }
+    }
 }
 
 void relocationValidation() {

core/relinker/relinker/tests/test_guest_deferred_dependencies.py

@@ -8,6 +8,7 @@ import tempfile
 
 from test_guest_intel_trampolines import main_fixture
 from test_guest_module_args import args_module, load_and_record
+from test_guest_module_dir import pe_export_names
 from test_guest_module_directories import module_with_symbol, needed_libraries
 from test_guest_needed_modules import executable_with_needed, module_with_needed
 from test_guest_symbol_names import DYNAMIC, GOT, RELOCATIONS, importing_executable
@@ -113,6 +114,15 @@ def main():
                     dependency = f"@executable_path/app0/Media/Plugins/{name}.prx.guest.prx"
                     expected = root == "imported" and name == "provider"
                     assert (dependency in needed) == expected, (root, name, needed)
+            windows = case / "windows"
+            windows.mkdir()
+            target = windows / "output.exe"
+            result = subprocess.run([str(relinker), "--windows", "--skip-syscall-check", "--module-dir", "Media/Plugins", str(source), str(target)],
+                                    capture_output=True, text=True, timeout=30)
+            assert result.returncode == 0, (root, result.stdout, result.stderr)
+            for name in ("plugin", "consumer", "provider"):
+                exports = pe_export_names((windows / "app0" / "Media" / "Plugins" / f"{name}.prx.guest.prx").read_bytes())
+                assert ("__aps5_guest_initialize" in exports) == (root != "bundled"), (root, name, exports)
             if sys.platform == "linux" and os.uname().machine == "x86_64":
                 libraries = case / "libs"
                 libraries.mkdir()

docs/dev/debt/relinker.md

@@ -10,7 +10,7 @@
 
 - [Executable guest imports](../../../core/relinker/relinker/src/guest/GuestModuleBuilder.cpp) - on Linux and macOS, a relocated executable import resolved to one guest provider requires that module and its dependencies at startup, even when it was discovered through `--module-dir` without a DT_NEEDED entry. Unreferenced symbols and no-op relocations do not require an otherwise deferred module. Console module-start ordering is unmeasured.
 
-- [Runtime-loaded modules](../../../core/relinker/relinker/src/guest/GuestModuleBuilder.cpp) (`--module-dir`) can be mapped early for bindings. Linux defers their initialization to an explicit load of each module; standard SDK folders keep their mapping initializer, including through the browser conversion API. On Windows and macOS, a module another module needs is initialized at startup without start arguments; console dependency-start ordering is unmeasured.
+- [Runtime-loaded modules](../../../core/relinker/relinker/src/guest/GuestModuleBuilder.cpp) (`--module-dir`) can be mapped early for bindings. Linux defers their initialization to an explicit load of each module; standard SDK folders keep their mapping initializer, including through the browser conversion API. Windows defers such a module too, unless a module outside these directories or in a standard SDK folder needs it, directly or through other modules; then it starts at startup without start arguments. On both, loading a module does not start the deferred modules it needs: each starts when the title loads it, and one the title never loads gets no `module_start` and no initializers. On macOS, a module another module needs is initialized at startup without start arguments; console dependency-start ordering is unmeasured.
 - [Runtime-loaded modules the executable needs](../../../core/relinker/relinker/src/guest/GuestModuleBuilder.cpp) (`--module-dir`) are mapped at startup on Windows and started when the title loads them with `sceKernelLoadStartModule`, with their start arguments, as a Unity player loads `mono-ps4.sprx`. A title that calls such a module before loading it finds its initializers not run; whether the console starts it earlier is not measured. A module the executable only imports from and the title never loads itself gets no `module_start` and no initializers at all under this deferral. The [entry stub](../../../core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp) prints `Deferred module <path>` at startup for each such module that an import of the executable names, whether or not the title loads it later; an import without a module name is not reported. macOS still starts it at startup.
 - The executable file that [relinker](../../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens a console when launched, unless it is relinked with `--windows-gui`, which selects the GUI subsystem. A GUI build started from Explorer has no standard output or error handle; the [entry stub](../../../core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp) gives each missing one `anyps5.log` beside the executable, replacing the previous run's file (#3282). Errors about the executable path or about waiting for the previous LoadExec process are written before the log is opened and are lost when the standard error handle is missing. If the log cannot be created, output to the missing handles is lost.
 - [Guest import registries](../../../core/relinker/relinker/src/guest/GuestModuleBuilder.cpp) - declared host imports of bundled modules are recorded with their module name and relocation target, even when another guest module exports the same stripped NID. No-op relocations are omitted. Registries describe import bindings, not implementation correctness or runtime reachability.

docs/user/USAGE.md

@@ -82,7 +82,7 @@ All switches are disabled by default. `unused-filter` defaults to `0`; `--rpath`
 | `--rpath <path>`              | Set the system library search path. Quote `$ORIGIN` to prevent shell expansion, for example `--rpath '$ORIGIN/libs'` in Bash or PowerShell. On Linux every colon-separated entry must be nonempty, and with guest modules each must be absolute or begin with `$ORIGIN`. Windows requires a nonempty ASCII path and supports `$ORIGIN` as the executable directory.                                                                                                         |
 | `--autorun`                   | Run the output after conversion, print its exit code, and wait for Enter. Requires the target OS and prepared runtime layout.                                                                                                                                                                                                                                                                                                                                               |
 | `--sce-module-path <path>`    | Parent of the module directories, not the module directory itself. Default: input ELF parent; relative input paths resolve from the process working directory. Explicit absolute path: used directly. Explicit relative path (including `.` and `..`): relative to the process working directory. Output path has no effect. Requires one nonempty value; repeats, nonexistent or nondirectory paths, access failures, and combination with `--skip-sce-module` are errors. |
-| `--module-dir <dir>`          | Relink the ELF modules of a directory under the module parent directory, besides `sce_module`/`sce_modules`/`prx`; outputs keep the relative path. Needed modules are mapped at startup. On Linux, modules in these extra directories initialize only when explicitly loaded, with the arguments from `sceKernelLoadStartModule` (for example Unity plugins in `Media/Plugins`). A module another module needs initializes at startup on Windows and macOS; one only the executable needs initializes when explicitly loaded on Windows and at startup on macOS. Other modules initialize when loaded. Repeatable; `.` is the parent itself, without the input ELF. Conflicts with `--skip-sce-module`.  |
+| `--module-dir <dir>`          | Relink the ELF modules of a directory under the module parent directory, besides `sce_module`/`sce_modules`/`prx`; outputs keep the relative path. Needed modules are mapped at startup. On Linux, modules in these extra directories initialize only when explicitly loaded, with the arguments from `sceKernelLoadStartModule` (for example Unity plugins in `Media/Plugins`). On Windows, such a module initializes when explicitly loaded unless a module outside these directories needs it, directly or through other modules; then it initializes at startup. On macOS, a module another module or the executable needs initializes at startup. Other modules initialize when loaded. Repeatable; `.` is the parent itself, without the input ELF. Conflicts with `--skip-sce-module`.  |
 | `--executable <path>`         | Also convert another executable of the title, which it starts with `sceSystemServiceLoadExec`; the path is relative to the module parent directory, for example `picker/picker.bin`. It is written beside the output, named after the path with `/` replaced by `.` and given the output's extension (`picker.picker.bin.exe`), and uses the same `libs/`, `app0/` and guest modules. It is an error if a guest module converts differently for it. Repeatable. Conflicts with `--skip-sce-module`. |
 | `--patches <file>`            | Apply the byte patches of this file during conversion, see [game patches](#game-patches). Overrides `relinker-patches.toml` discovery. Conflicts with `--no-patches`. |
 | `--no-patches`                | Do not apply patches, and do not load `relinker-patches.toml` from the input directory. |