fix(recompiler): hash shader cache sources with normalized line endings

The shader disk cache source version hashed raw file bytes, so one commit
checked out with CRLF in one worktree and LF in another produced two
versions and emptied the cache and pipeline journal. Hash the text with
CRLF folded to LF.
This commit is contained in:
memob committed 2026-10-21 06:40:50 -04:00
1 parent 158d8ad34c
commit 106fa6b6be
3 files changed
+22 -1

No files matched your search

@@ -39,7 +39,9 @@ foreach(source IN LISTS sources)
if(NOT EXISTS "${source}")
continue()
endif()
file(SHA256 "${source}" digest)
file(READ "${source}" text)
string(REPLACE "\r\n" "\n" text "${text}")
string(SHA256 digest "${text}")
file(RELATIVE_PATH relative "${ROOT}" "${source}")
string(APPEND manifest "${relative} ${digest}\n")
if(NOT source MATCHES "\\.(cpp|hpp|h)$")
@@ -51,6 +51,24 @@ class ShaderCachePathTests(unittest.TestCase):
def test_cjk(self):
self.check_source_change('\u6e2c\u8a66 path')
def test_line_endings_keep_version(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary) / 'line endings'
source, listing, output = self.fixture(root)
text = '#include <cstdint>\nconst char* mode = "APS5_TEST_PATH";\nint value = 1;\n'
source.write_bytes(text.encode('utf-8'))
result = self.run_script(root, listing, output)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
unix = output.read_bytes()
source.write_bytes(text.replace('\n', '\r\n').encode('utf-8'))
result = self.run_script(root, listing, output)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertEqual(unix, output.read_bytes())
source.write_bytes(text.replace('= 1;', '= 2;').replace('\n', '\r\n').encode('utf-8'))
result = self.run_script(root, listing, output)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertNotEqual(unix, output.read_bytes())
def test_unicode_guard_rejects_unhashed_include(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary) / '\u0441\u0431\u043e\u0440\u043a\u0430'
+1
View File
@@ -28,6 +28,7 @@
- [Windows cache housekeeping](../../../core/shader/recompiler/ShaderDiskCache.cpp): eviction of old versions and temporary files is skipped for `\\?\` and `\\.\` cache roots because the supported WinLibs directory iterator can enumerate the process working directory for long prefixed paths. The active version's `last-used` stamp update still runs; it fails for long `\\.\` aliases with the supported toolchain. Ordinary Windows directory enumeration remains subject to MAX_PATH.
- [f64 compares](../../../core/shader/recompiler/Translation/src/CompareInstructions.cpp) flush denormal inputs to signed zero when bit 0 of the `FLOAT_MODE` f16/f64 denormal field is clear (modes 0 and 2); modes 1 and 3 and an unknown mode keep them. This follows the RDNA2 ISA section 6.4 and LLVM's sign-preserving AMDGPU denormal modes. The gfx1036 audit in [#981](https://github.com/boykopovar/AnyPS5/issues/981#issuecomment-6070046381) reports mode-sensitive `VopcCompareF64` results. Class compares still inspect the original bits. The full four-mode matrix has not been newly measured on AMD hardware or the PS5's gfx1013.
- [Shader disk cache source version](../../../core/shader/recompiler/ShaderCacheVersion.cmake) - the version that names `shader_cache/<version>` and stamps `pipelines.journal` hashes every file under `core/shader/recompiler` (line endings normalized), `HostThreadLocal.hpp`, the SPIR-V and glslang submodule commits, and the compiler, build type and flags. Any edit there changes it and empties the cache, including edits that cannot change a translation (logging, the disk cache's own I/O, the recompiler's tests), and so does a rebuild with another compiler or flags. Driver sources outside the recompiler are not hashed: their recompiler calls are only checked against `shaderCacheReviewedNames`, so a driver change that alters a `RecompileRequest` field's meaning without changing its value keeps a stale cache.
- [Unlinked calls](../../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) - the `s_swappc_b64` a table call's compare chain falls back to, when its target matches no linked copy, records the `UnlinkedCall` fault with the target and the program counter and stops the invocation, or continues after the call in programs whose faults do not stop invocations (see `bdaStopsInvocations`). The driver lists those calls in the request (`linkedCallFallbacks`), and it reports the fault like a BDA fault, naming the target: a dispatch that checks its faults when it completes throws, one whose check runs in a deferred write-back logs it and continues. The hardware would jump to the target, which is not modelled. Any other `s_swappc_b64` whose target is not resolved still throws when the shader is compiled.
- [Cached source IR](../../../core/shader/recompiler/Recompiler.cpp) releases lazily allocated SSA construction tables after resource preparation and reclaims values unreachable from instructions, metadata and resource plans when retaining a pending source program. Immediate shader preparation skips reclamation because compilation consumes the program. Reclamation must account for every new IR pointer owner; a remaining external use throws. Live IDs are retained. Source and variant entries remain cached without an eviction bound. A pending source program is released after its first variant is obtained; memory beyond the measured title checkpoint and Linux performance remain unmeasured.