On NVIDIA's proprietary driver, function arrays that live in local memory
and are read under a data-dependent branch return wrong values in
subgroups 1-3 of a mesh workgroup; subgroup 0 is correct. A wave64
geometry program whose relative addressing keeps its VGPRs in the vector
window therefore loses register writes in every wave but the first. In
the clipping geometry program of Sony's PS5 PSP emulator (PPSA07550) the
empty waves never cleared their candidate primitive slots and wrote
degenerate triangles over wave 0's, so the 3D menu drew spikes.
On that driver a mesh program's window now lives in a device-local
storage buffer of 256 workgroup slots of 65536 dwords, claimed through a
per-slot lock and released after the final barrier. The pool is created
before the first draw that uses it records anything, and its header is
zeroed in a batch of its own that is submitted and waited for; the pool
is published only after that wait, and a failure releases it. Direct
register accesses keep one function variable per register; a direct
store also writes the buffer when a buffer read can see it, relative
reads load the buffer, relative writes store only to it, and a direct
read that may follow a relative write loads the buffer and refreshes the
variable. A buffer entry is zeroed only when a buffer read can see it
before a write. APS5_NO_MESH_WINDOW_BUFFER=1 keeps the arrays.
SPIRV-Cross emits the lock's acquire compare-exchange and release
exchange as relaxed Metal atomics, so the plain LDS loads and stores of
a locked 64-bit update were not ordered against the lock and contended
ds_add_u64 lost updates on an Apple M5 Pro. On MoltenVK the driver now sets
SpirvTarget::ldsLockFences, and the lock brackets its critical section
with workgroup-scope acquire and release OpMemoryBarriers, which become
threadgroup atomic_thread_fence with no execution barrier.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
MoltenVK exposes VK_KHR_fragment_shader_barycentric but its MSL step
rejects PerVertexKHR, so the extension is disabled there and a pixel
shader whose I/J VGPRs outlive their v_interp_p1/p2 pairs takes the
generated barycentric stages, which accept only triangle lists without
primitive restart on a device without geometry shaders.
A new target flag, fragmentBarycentricBuiltins, enables the extension
for its built-ins only. The host interpolation check then keeps the
leftover I/J reads (all perspective or all linear, at the fragment's
default position) and reads them from BaryCoordKHR or
BaryCoordNoPerspKHR, while the interpolated attributes stay hardware
varyings. Raw vertex parameters and I/J at an offset or a sample
position still take the generated stages. Apple GPUs return
unnormalized perspective weights unless the fragment function reads
[[position]], so each perspective read keeps a neutral use of
gl_FragCoord.w, loaded as a float. Devices with the full extension are
unaffected; the request serialization moves to version 45.
Co-authored-by: yripper <[email protected]>
TargetMerge gains a TargetBlend (factors, operations, write mask, constants, signedness) that the request serializer, the cache key and the variant hash take only when it is enabled, so every request without it keeps its bytes, key and hash.
A shader with an image table map read its fault buffer back only when a
table record was poisoned, an SRT read was poisoned, it had window lane
guards or the loop guard was on. A DescriptorFault only set the fault
buffer, so with a valid table and no copied write nothing ran the fault
check, and a lane that took a folded descriptor web path or selected an
undefined arm read arm 0 or zeros with no report.
The compiled artifact now counts its DescriptorFault instructions
(descriptorFaults, stored in the shader disk cache, format 48), and the
readback gate requests the fault check when it is nonzero. The result a
capture materializes copies the count, and also windowLaneGuards, which
that copy had dropped, so the gate's window lane guard condition never
saw it. sizeof(CompiledShaderArtifact) is 288 and sizeof(RecompileResult)
552 with the new field.
agc_driver_descriptor_web_faults adds a wave32 shader that reads an image
table and then the descriptor web, storing only to LDS so no copied write
forces completion: it binds an image table and no BDA address table, a
valid arm and an unselected undefined arm stay clean, the folded path and
a selected undefined arm fault at the read, and nothing faults with EXEC
empty. Without the gate change the folded case does not fault.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
NVIDIA's proprietary driver flushes f32 add, multiply and fma denormals by default but not the inputs of floor and ceil, so its startup probe failed only on those two. Drop their software flush only where the device declares DenormFlushToZero 32.
Every shader source keeps up to 256 materialized results of
Recompile(request, capture) in its result memo, and as many patch
recipes in its masked memo, with no limit on their bytes. In Ghost of
Tsushima's opening the memos held about 500 MiB after 5 minutes and
kept growing, at a 13 % hit rate.
Each memo entry now counts the bytes its result's vectors keep (the
bindings' vectors included; the module is shared with the prepared
module cache), plus its patch recipe in the masked memo. A result held
by both tables counts once in each. Past a budget, 256 MiB unless
APS5_RESULT_MEMO_MB sets another (0 sets none), the least recently used
entries of both tables of all sources are dropped until three quarters
of the budget is left. The recency stamps come from one clock for all
sources and both tables so they compare across them. The profile line
also reports the trimmed and held entries and their MiB.
The trim takes its lock blocking and repeats, re-reading the bytes held,
until they are within the budget, so the insertion that pushed them
over has its trim run after any other; it stops when nothing is left to
drop.
SetResultMemoBudget and ResultMemoBytes let agc_shader_result_memo_budget
check that a memo over its budget keeps the most recently used results
and drops the oldest one, with the masked memo off. A one-shot hook
inside the trim (SetResultMemoTrimHook) lets
agc_shader_result_memo_budget_threads insert two results from other
threads after the trim has read the bytes held: the memo must still end
within its budget. agc_shader_result_memo_budget_masked, with the masked
memo on, checks that a first result counts its masked entry too and that
a trim empties the masked memo.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Crimson Desert (PPSA25286) draws with VGT_SHADER_STAGES_EN 0x60212d:
local, hull, domain and NGG geometry stages, the hull halves merged in one
registered type-3 program, the domain and geometry halves in one type-2
program. The domain half runs as the tessellation evaluation stage and
captures each vertex's ESGS item, and a mesh shader runs the geometry half
on the captured items.
Registered type-3 merged hulls come from #6679; this adds 16-bit LDS
reads and writes in tessellation stages, which Crimson Desert's merged
hull uses.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The specialized module caches (by variant and constants, by binding plan
and by registration key) and the SPIR-V interning table held every
module's words for the rest of the run, after the results that used them
were gone. They now keep a module's interface (bindings, push data,
specialization ID and content hash) and only a weak reference to its
words; a lookup after the words were released builds them again from the
same key, checks the content hash, and keeps the specialization ID that
names the module's pipelines. The interning table keeps weak references
and drops expired ones from the bucket it searches.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A shader source loading its resource plan from the disk cache decoded
its own copy, also when another source had already loaded the same
plan: the same code registered under another plan key (another host
subgroup size or interface) stores byte-identical plan entries. In Ghost
of Tsushima's cache, 14,043 plan entries hold 8,170 distinct payloads.
Loaded plans now go through a table of weak references keyed by two
hashes of the payload, its size and the shader address the plan carries,
so equal payloads for one shader decode once and share the plan, its
compact form and its compiled SRT tape while any source uses it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The recompiler orders an LDS atomic, and the compare-exchange loop that
writes a byte or a halfword of LDS (ds_write_b8/b16), with a
workgroup-scope OpMemoryBarrier after the update. AMD's proprietary
Windows driver 32.0.21042.62 (Radeon 0x13c0, RDNA 2) compiles that
memory barrier to s_barrier, as the code objects in its pipeline cache
show. Such updates usually sit in branches that only some waves of a
workgroup take, so those waves execute extra s_barriers, which pair
with the other waves' barriers, and the other waves read LDS before it
is written. An OpMemoryBarrier is not an execution barrier and SPIR-V
allows it in non-uniform control flow, so the SPIR-V is valid; a
device-scope OpMemoryBarrier compiles to GL0 and GL1 cache
invalidations without s_barrier there.
PS5LM Plus (PPSA99581, llama.cpp's Vulkan backend) drifted into
garbage on that driver with flash attention on. Its flash-attention
shader stores two halfwords to LDS in a branch that two of its four
waves take, inside the KV loop. The driver's code had 23 s_barrier for
the guest's 21; a replay of the dispatch differed from NVIDIA in 159
words, up to 1.7e38, and now has 21 and differs only in the last bit
of 9 words.
On that driver the fence is now device scope, which orders everything
the workgroup-scope fence did. Other drivers keep the workgroup scope.
The flag is part of the shader cache key, and agc_shader_replay takes
--device-scope-lds-memory-barriers to replay a request the way that
driver compiles it. The LDS update helper already fences, so the
second, identical fence after it in ds_min/max_f32 and the other
read-modify-write LDS operations is dropped.
agc_driver_lds_subword_store_branch stores a halfword to LDS in a
branch that waves 0-1 of four take, delays those waves and then
exchanges LDS slots across one s_barrier. Before, thread 64 read a
stale slot on the AMD driver; it passes on AMD and NVIDIA now.
agc_subgroup_barrier checks the driver quirk and that ds_write_b16,
ds_add_u32 and ds_max_f32 each emit one fence of the expected scope.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BM2kfDbr6MydxsfTckZJDR
The slot counter is reset only after earlier fragment writes, and the overflow flag stays until WaitIdle. The storage buffer is limited to NVIDIA's proprietary driver, which is where the large function array does not finish.
The recompiler marks every guest f32 add, subtract and multiply
NoContraction, because the guest rounds each one separately.
SPIRV-Cross turns each of them into a call to a helper that Metal
compiles without optimization, which dominates fragment time on
MoltenVK, and its multiply helper, fma(a, b, +0.0), turns a -0.0
product into +0.0.
On MoltenVK they are now GLSL Fma whose second step is exact, which
gives the separately rounded result: a * b is fma(a, b, -0.0), a + b
is fma(1.0, a, b) and a - b is fma(-1.0, b, a). The sin, cos and log2
series and the offset image coordinates and gather texel scale use the
same forms. The choice is a SpirvTarget flag set for
VK_DRIVER_ID_MOLTENVK, part of the shader cache key and of the request
serialization (version 44); other drivers are unchanged.
agc_driver_f32_separate_rounding checks a multiply feeding an add or a
subtract on rows where fusing them changes the result. It,
agc_driver_f32_arithmetic_nan and agc_driver_f32_denormal_flush also
run the fused forms on devices that do not use them, so lavapipe checks
them against the same rows.
Co-authored-by: Ycaro Gabriel <[email protected]>
v_add_f32, v_sub_f32, v_subrev_f32, v_mul_f32 and v_mul_legacy_f32
follow the FLOAT_MODE f32 denormal field with f32 round to nearest
even, as measured on RDNA2 (gfx1030): mode 1 keeps denormal inputs and
flushes a denormal result, and mode 2 flushes denormal inputs and keeps
a denormal result. Both need the device to keep the f32 denormals the
shader keeps, so they translate only when the module declares
DenormPreserve 32 and throw otherwise. The result flush now applies
when only the result flush bit is set; it was skipped unless the input
flush bit was set too. A constant factor no longer skips the tiny
product check while denormal inputs are kept, as a kept denormal times
such a constant can still be tiny. Directed rounding in these modes
still throws.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Adrià