ShaderCacheVersion.cmake resolved every source's real path twice and
every include's real path once per include line: 1102 REALPATH calls
per run, about 1.6 ms each on Windows. Reuse the source paths from the
first pass and resolve each include path once. The generated version
file is unchanged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The shader disk cache source version hashed raw file bytes, so one commit
checked out with CRLF in one worktree and LF in another produced two
versions and emptied the cache and pipeline journal. Hash the text with
CRLF folded to LF.
ShaderCacheVersion.cmake wrote the source version and the APS5_* switch
list into a generated ShaderCacheVersion.hpp that ShaderDiskCache.cpp
included. The version hashes every recompiler source, so any content
change under core/shader/recompiler rewrote that header and recompiled
ShaderDiskCache.cpp next to the file actually edited.
The script now writes ShaderCacheVersion.cpp, which defines
Generated::SourceVersion and Generated::RecompilerSwitches (a span over
the same sorted switch list) with constinit. They are declared in the
new ShaderCacheSourceVersion.hpp in the source tree, which
ShaderDiskCache.cpp includes; the generated directory is no longer an
include directory. The header has a different name from the old
generated one so that a build directory that still holds the stale
ShaderCacheVersion.hpp next to the generated .cpp cannot pick it up.
What is hashed is unchanged: the same manifest (recompiler sources,
HostThreadLocal.hpp, submodule commits, compiler, build type,
architecture and flags), the same checks and the same rewrite-only-
when-different step, so the version still changes exactly when it did.
Running the upstream/main script and this one with the same arguments
over this tree gives the same version (0x7907101dc4d8c62e) and the same
26 switches, and a second ninja run has no work to do. The value
differs from the one main 682ef0c17 generates (0xdce3a0ce03bf030a) only
because this commit edits hashed files, as any recompiler change does.
Tested with ninja -d explain -j 4 shader_recompiler_agc_driver,
CCACHE_DISABLE=1, after appending a line to
IntermediateRepresentation/src/IrType.cpp, three runs on main and then
three runs on this commit back to back, with the machine at 100% CPU
from other builds. On main ShaderCacheVersion.hpp and
ShaderDiskCache.cpp.obj are dirty and the rebuild takes 22.0/23.9/22.2 s.
With this change only ShaderCacheVersion.cpp.obj and IrType.cpp.obj are
compiled, ShaderDiskCache.cpp is absent from the explain output, and the
rebuild takes 12.2/14.7/15.0 s. The shader and recompiler ctest set
(26 tests) passes.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Codex <[email protected]>
SourceVersion hashed every libSceAgcDriver source, so any driver-only
change sent the shader disk cache cold: 88 of the last 200 merges on main
did; with the inputs below, 51 of them would have. It now covers:
- core/shader/recompiler/** (sources, headers, CMake);
- prx/libc/include/HostThreadLocal.hpp, the one header the recompiler
includes from outside its directory (checked against the ninja deps);
- the checked-out commits of SPIRV-Headers and glslang, plus SPIRV-Tools
when ANYPS5_ENABLE_SPIRV_TOOLS is on (the submodule HEAD files are build
dependencies, so moving a submodule regenerates the version);
- the compiler, build type, architecture and C++ flags.
Everything else the driver passes to the recompiler arrives in the
RecompileRequest, which the per-entry key covers. Two build-time checks keep
that true, failing the version step with a message naming the file:
- a recompiler source that includes a project header outside the hashed set;
- a driver source that calls or using-declares a ShaderRecompiler name not
on the reviewed list, pulls in the whole namespace, or sets an environment
variable (which could flip a recompiler APS5_* switch behind the key).
APS5_* switches are collected from every hashed source. The version script
sets cmake_minimum_required: without it CMake 3.x runs it with CMP0054 OLD
and reads the quoted section names in its if() tests as variables.
Every run recompiled every shader: the recompiler's variant cache and
the device's VkPipelineCache only lived in memory, and the first frames
of a title spent hundreds of milliseconds per large program in the
front end, the emitter and the optimizer.
Compiled variants (the SPIR-V plus the compiled shader info and binding
allocation that per-snapshot binding population needs) are now stored
on disk, one file per key, written atomically by a background thread,
and a later run loads them instead of compiling. The key holds every
input the in-memory variant cache matches on: the in-memory recompile
key (stage, code hash, stage registers, user data count, target), every
code word, the host subgroup size, the binding layout, the resource
specialization, and the values of the APS5_* switches the recompiler
sources read (found at build time; switches that only report are left
out). Entries live under a directory per source version, a build-time
hash of the recompiler and AGC driver sources and the build
configuration, and carry their key and checksums: an entry whose
format, version, key, size or checksum does not match, or whose payload
does not decode exactly, is ignored and recompiled. Directories no
build used for two weeks and stale temporary files are removed. Requests
under the debug probe bypass the disk cache.
The device's VkPipelineCache is loaded at creation when the file's
checksum and Vulkan header match the device (vendor, device and
pipelineCacheUUID; the driver version is in the file name), saved every
10 s while it grew, and saved at teardown.
Location: a shader_cache folder beside the running executable (not the
working directory), so the relinked game stays a self-contained folder;
ANYPS5_SHADER_CACHE_DIR overrides it. ANYPS5_NO_SHADER_CACHE=1 turns both caches off; the AGC
tests run with it set so they never write to the user's cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
