SPIRV-Cross emits the lock's acquire compare-exchange and release
exchange as relaxed Metal atomics, so the plain LDS loads and stores of
a locked 64-bit update were not ordered against the lock and contended
ds_add_u64 lost updates on an Apple M5 Pro. On MoltenVK the driver now sets
SpirvTarget::ldsLockFences, and the lock brackets its critical section
with workgroup-scope acquire and release OpMemoryBarriers, which become
threadgroup atomic_thread_fence with no execution barrier.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Main took request serialization version 48 for the fragment barycentric
built-ins flag (#9193). The depth export clamp bounds are now written at
version 49 and read only for versions from 49, so a version 48 request as main
writes it still decodes, with default bounds. Tests: version 49 is the
emitted prefix, 50 the first unsupported one, a depth exporting request
without its bounds decodes at 48, 47 and 46 (the last two without main's
barycentric byte), and a version 48 two-sample dithered request decodes as
written.
Main took request serialization version 47 for the two-sample dithered
alpha-to-coverage value. The depth export clamp bounds are now written at
version 48 and read only for versions from 48, so a version 47 request as main
writes it still decodes, with default bounds. Tests: version 48 is the
emitted prefix, 49 the first unsupported one, and a version 47 two-sample
dithered request (with and without depth export) decodes as written.
MoltenVK exposes VK_KHR_fragment_shader_barycentric but its MSL step
rejects PerVertexKHR, so the extension is disabled there and a pixel
shader whose I/J VGPRs outlive their v_interp_p1/p2 pairs takes the
generated barycentric stages, which accept only triangle lists without
primitive restart on a device without geometry shaders.
A new target flag, fragmentBarycentricBuiltins, enables the extension
for its built-ins only. The host interpolation check then keeps the
leftover I/J reads (all perspective or all linear, at the fragment's
default position) and reads them from BaryCoordKHR or
BaryCoordNoPerspKHR, while the interpolated attributes stay hardware
varyings. Raw vertex parameters and I/J at an offset or a sample
position still take the generated stages. Apple GPUs return
unnormalized perspective weights unless the fragment function reads
[[position]], so each perspective read keeps a neutral use of
gl_FragCoord.w, loaded as a float. Devices with the full extension are
unaffected; the request serialization moves to version 45.
Co-authored-by: yripper <[email protected]>
Crimson Desert (PPSA25286) draws with VGT_SHADER_STAGES_EN 0x60212d:
local, hull, domain and NGG geometry stages, the hull halves merged in one
registered type-3 program, the domain and geometry halves in one type-2
program. The domain half runs as the tessellation evaluation stage and
captures each vertex's ESGS item, and a mesh shader runs the geometry half
on the captured items.
Registered type-3 merged hulls come from #6679; this adds 16-bit LDS
reads and writes in tessellation stages, which Crimson Desert's merged
hull uses.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Bugsnax (PPSA01503) draws with DB_ALPHA_TO_MASK 0xe701 (quad offsets
3, 1, 2, 3 without OFFSET_ROUND) on a two-sample target and stopped on
the refusal of every dither but the eight-sample 0x18701. The pixel
shader now applies an eight-level per-quad table for it, measured
natively on gfx1032, and the request's dither byte selects that table
(serialization version 47). It is accepted only for two matched raster
and DB_EQAA samples, the two measured location pairs and FP16_ABGR or
32_ABGR exports.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Every recompile cache key appended the target's capability list and
each SPIR-V extension name one byte per 64-bit word, so on a device that
enables a few dozen extensions most of every key was the same device
description: built and compared on every dispatch and draw, and kept in
every in-memory key and disk cache entry. The two lists now enter the key
as a 128-bit digest over the capabilities and the length and bytes of
each extension name, eight bytes per step.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The recompiler orders an LDS atomic, and the compare-exchange loop that
writes a byte or a halfword of LDS (ds_write_b8/b16), with a
workgroup-scope OpMemoryBarrier after the update. AMD's proprietary
Windows driver 32.0.21042.62 (Radeon 0x13c0, RDNA 2) compiles that
memory barrier to s_barrier, as the code objects in its pipeline cache
show. Such updates usually sit in branches that only some waves of a
workgroup take, so those waves execute extra s_barriers, which pair
with the other waves' barriers, and the other waves read LDS before it
is written. An OpMemoryBarrier is not an execution barrier and SPIR-V
allows it in non-uniform control flow, so the SPIR-V is valid; a
device-scope OpMemoryBarrier compiles to GL0 and GL1 cache
invalidations without s_barrier there.
PS5LM Plus (PPSA99581, llama.cpp's Vulkan backend) drifted into
garbage on that driver with flash attention on. Its flash-attention
shader stores two halfwords to LDS in a branch that two of its four
waves take, inside the KV loop. The driver's code had 23 s_barrier for
the guest's 21; a replay of the dispatch differed from NVIDIA in 159
words, up to 1.7e38, and now has 21 and differs only in the last bit
of 9 words.
On that driver the fence is now device scope, which orders everything
the workgroup-scope fence did. Other drivers keep the workgroup scope.
The flag is part of the shader cache key, and agc_shader_replay takes
--device-scope-lds-memory-barriers to replay a request the way that
driver compiles it. The LDS update helper already fences, so the
second, identical fence after it in ds_min/max_f32 and the other
read-modify-write LDS operations is dropped.
agc_driver_lds_subword_store_branch stores a halfword to LDS in a
branch that waves 0-1 of four take, delays those waves and then
exchanges LDS slots across one s_barrier. Before, thread 64 read a
stale slot on the AMD driver; it passes on AMD and NVIDIA now.
agc_subgroup_barrier checks the driver quirk and that ds_write_b16,
ds_add_u32 and ds_max_f32 each emit one fence of the expected scope.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BM2kfDbr6MydxsfTckZJDR
The slot counter is reset only after earlier fragment writes, and the overflow flag stays until WaitIdle. The storage buffer is limited to NVIDIA's proprietary driver, which is where the large function array does not finish.
The DB clamps a pixel shader's exported depth to PA_SC_VPORT_ZMIN_0 and
ZMAX_0 unless DB_RENDER_OVERRIDE.DISABLE_VIEWPORT_CLAMP is set. Vulkan only
clamps with depthClamp, which also drops clipping, so the exported value
reached the depth attachment unclamped.
The pixel stage info now carries the two bounds when the shader exports
depth (0 and 1 otherwise), and the exported value is NClamped to them
before it is stored to gl_FragDepth. With
DISABLE_VIEWPORT_CLAMP the value is stored as exported; non-finite or
inverted bounds are rejected. DB_RENDER_OVERRIDE joins the draw key
registers. The bounds
are part of the recompile cache key (only for depth-exporting shaders, so
other keys do not change), the draw key and its register check, and
recompile requests (serialization version 47).
Tests: the pixel stage decode reads the bounds, passes the disable flag through
and rejects bad bounds, and leaves a draw without depth export at 0 and 1; a
recompiled depth export stores the NClamped value, and its bounds survive
serialization and key the cache.
Request version 47: main is at 44 (#8612) and open PRs also claim 44 to 46 (#8564, #7888, #9193 and the #9020, #9335, #9748 stack). Whichever of these lands second renumbers.
The recompiler marks every guest f32 add, subtract and multiply
NoContraction, because the guest rounds each one separately.
SPIRV-Cross turns each of them into a call to a helper that Metal
compiles without optimization, which dominates fragment time on
MoltenVK, and its multiply helper, fma(a, b, +0.0), turns a -0.0
product into +0.0.
On MoltenVK they are now GLSL Fma whose second step is exact, which
gives the separately rounded result: a * b is fma(a, b, -0.0), a + b
is fma(1.0, a, b) and a - b is fma(-1.0, b, a). The sin, cos and log2
series and the offset image coordinates and gather texel scale use the
same forms. The choice is a SpirvTarget flag set for
VK_DRIVER_ID_MOLTENVK, part of the shader cache key and of the request
serialization (version 44); other drivers are unchanged.
agc_driver_f32_separate_rounding checks a multiply feeding an add or a
subtract on rows where fusing them changes the result. It,
agc_driver_f32_arithmetic_nan and agc_driver_f32_denormal_flush also
run the fused forms on devices that do not use them, so lavapipe checks
them against the same rows.
Co-authored-by: Ycaro Gabriel <[email protected]>
A point list routed through the vertex path was refused, and so was
PA_CL_VS_OUT_CNTL.USE_VTX_POINT_SIZE on it. OliSe Player (PPSA01153)
stops on the latter with two of its MilkDrop presets.
The vertex path now draws point lists as Vulkan points. The vertex stage
writes PointSize: the misc export's X clamped to PA_SU_POINT_MINMAX when
USE_VTX_POINT_SIZE is set, and the PA_SU_POINT_SIZE height otherwise.
The point state travels in the recompile request (version 31) and the
cache and draw keys. Non-square register sizes, inverted clamps, point
sprite texture coordinates and, without largePoints, sizes other than
one pixel throw.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
main 7ce3b1df3e took disk-cache format 43 (#8295) for its own layout, so this
branch's format moves to 44. The request serializer stays at 42: main
still writes 41.
The recompiler asked for DenormPreserve only for 16-bit and 64-bit floats,
so a device that flushes 32-bit denormals by default (RADV, MoltenVK) flushed
them whatever the guest's float mode said. target.float32DenormPreserve is set
from shaderDenormPreserveFloat32 when denormBehaviorIndependence allows a 32-bit
mode, is part of the cache key and of the request serialization (version 41),
and makes the module declare DenormPreserve 32 unless the 32-bit flush is
already taken in hardware. APS5_NO_HW_F32_PRESERVE=1 turns it off.
The refusal of a merging shader that writes guest memory under late
tests fired in two cases where the early tests change nothing.
The first was a draw with no depth, stencil or depth-bounds test. Early
tests reject no pixel there, so every write the console makes still
happens. The target merge now carries whether the draw has such a
test, and the refusal applies only then. This adds one byte to the
version 40 merge fields of the recompile request.
The second was any GDS binding, including a shader that only reads GDS.
The refusal now looks for GDS writes, atomics, appends and consumes.
Tests: the depth and stencil decode checks the new flag, and the
static shader key depends on it. A counter-adding shader under late
tests compiles without a test and throws with one. A late-test merge
that reads GDS compiles, and one that adds to GDS throws.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A packed 4_4_4_4 partial write merges inside the pixel shader after early
depth and stencil tests. Moving the tests ahead of a pixel kill only changes
the result when the draw writes depth or stencil, so the target merge now
carries whether the draw does (depth write, or stencil with a write mask and
a non-keep operation), and the late-test kill refusal applies only then.
The flag is part of the static shader key, the serialized request (now
version 40) and the pixel input info.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A CB_TARGET_MASK that writes fewer than four components of a 4_4_4_4
target leaves the other nibbles of each packed word as they were, which
an R16_UINT attachment cannot do. Such draws now close the attachment's
color write mask, and the pixel shader merges the written nibbles into
the target through a storage image: inside a pixel interlock it reads
the word, keeps the bits the mask leaves alone and stores it back.
The merge (export index, word mask, sample count) is part of the static
shader key, request version 40 and shader disk cache format 42; the
storage image is appended to the shader's images with a single runtime
mode, and its T# is encoded from the target's surface at draw time, so
the draw resolves it to the target's resident image, which Draw
requires. Merging shaders declare early fragment tests; shaders that
kill under late tests, export depth, stencil or coverage, use
alpha-to-coverage or a quad sample mask are refused, as are
multisampled, mipmapped, volume and array targets and more than one
merged target per draw.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A guest wave's lanes see each other's LDS writes without a barrier, so
the recompiler adds a subgroup-scope OpControlBarrier between an LDS
write and the next LDS access when the guest wave is one host subgroup
(WaveLdsScope). AMD's proprietary Windows driver 32.0.21042.62 (Vulkan
driver 2.0.353, "AMD proprietary shader compiler", Radeon 0x13c0)
compiles that barrier to s_barrier, as the pipeline's internal ISA
shows (26 s_barrier for the guest's 9 in a small LDS reduction). In a
workgroup of several waves, the waves that enter a branch holding LDS
accesses then execute extra s_barriers, which pair with the other
waves' barriers, and those waves read LDS before it is written. LLPC
compiles a subgroup-scope control barrier to a wave barrier instead.
PS5LM Plus (PPSA99581) answered with garbage on that driver: the first
compute dispatch of its first graph, an RMS norm with a 512-thread
wave32 LDS tree reduction, scaled every row except its first 32
elements by a partial sum, while NVIDIA matched a NumPy reference and
the inputs were identical.
On that driver the barrier is now a subgroup-scope OpMemoryBarrier,
which it compiles to no instruction. A 32-lane subgroup is one hardware
wave there, which runs its lanes in lockstep and issues its LDS
instructions in order; Vulkan does not guarantee that, so it is
recorded as debt. Other drivers keep the control barrier. The flag is
part of the shader cache key, and agc_shader_replay takes
--wave-lds-memory-barriers to replay a request the way that driver
compiles it.
agc_driver_lds_tree_reduction sums over eight wave32 waves the way the
guest does, with its last four steps inside one wave and no s_barrier
between them: before, thread 32 read 1936 instead of 32896 on the AMD
driver; it passes on AMD and NVIDIA now. agc_subgroup_barrier checks
the driver quirk and the emitted barriers on every runner. PS5LM Plus
now answers " Paris. It is the largest city in Europe ..." on the AMD
iGPU without flash attention, as on NVIDIA and the CPU.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A registered type-3 hull program holds the local and the hull part of a
tessellation draw in one binary, addressed through SPI_SHADER_PGM_LO_LS
(SH 0x148) with no SPI_SHADER_PGM_LO_HS: the local part runs on the lanes
below s3[7:0] and writes LDS, an s_barrier follows, and the hull part runs
on the lanes below s3[15:8] and may store the registers the local part
filled. The draw decode asked 0x148 for a type-5 local half, and
PPSA11386 (007 First Light) stopped in its level load with "registered
program refers to an incompatible shader binary type".
The draw decode now gives a registered type-3 program both the local and
the hull role and marks the tessellation configuration mergedHull. The
local stage only passes gl_VertexIndex and gl_InstanceIndex in a 16-byte
record. The hull stage runs the whole program, one invocation per output
control point: v2 and v5 come from the record of control point
min(gl_InvocationID, inputs - 1), v3 is the relative vertex id, and the
LDS becomes a per-control-point output array that a control point writes
in its own record and reads in any record of its patch after the barrier.
Registers live across the barrier stay in the invocation.
The hull stage of a merged program runs at one lane per invocation on
every device (HostSubgroupSize 1). Its lanes are gl_InvocationID, not host
subgroup invocations, so the s_cbranch_execz around the local part tests
the invocation's own lane. With the device's subgroup it took a subgroup
ballot, and lavapipe, whose subgroup stages lack the tessellation control
stage, rejected the draw ("subgroup capability 61 is unsupported for
shader stage 2").
The lowering refuses more input than output control points, LDS writes
outside the own record, reads outside the patch's records, barriers that
some invocations do not reach or that sit in a loop, reads or writes of
another lane's register, outputs beyond the device limits, and fetch
subroutines. Type 3 is prepared with its stage group like type 5 and
takes its float mode from RSRC1_HS. The hull user-data address (SH
0x102/0x103) is optional for a merged program: s[0:1] take it, unchecked,
when the draw sets it and are zero otherwise.
Both stages refer to one binary, so a draw lists each code and header
region once (AppendProgramMemory); a request's memory regions must not
overlap. An indirect draw's record words go into every program whose user
data holds the register, so the hull stage of a merged program reads
them too.
mergedHull is part of the recompile cache key, the draw decode comparison
and the request serialization (version 39).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A pixel shader that reads barycentrics on a device without
VK_KHR_fragment_shader_barycentric used to need a vertex shader so a
generated geometry stage could follow it, which a mesh pipeline cannot
have. The mesh program is now compiled a second time with the layout
its pixel shader needs: it writes its clip position as a varying and the
three corner positions of each primitive as per-primitive outputs, and
the pixel shader solves for the weights, for both smooth and linear
interpolation.
The layout is part of the mesh cache key, the stored artifact and the
captured request. Without VK_EXT_mesh_shader the draw still throws, now
saying so. Per-vertex parameters on the mesh path still throw.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
With DB_SHADER_CONTROL.STENCIL_OP_VAL_EXPORT_ENABLE the pixel shader
supplies the value that REPLACE_OP and the add and subtract operations
use in place of DB_STENCILREFMASK.STENCILOPVAL. The bit was refused as a
whole, which stopped Onimusha: Way of the Sword (PPSA27836) at a
DB_SHADER_CONTROL 0x45 draw with DB_STENCIL_CONTROL 0x40040 (REPLACE_OP
on depth pass, both faces), write mask 0xff and compare ALWAYS.
The export is accepted with the 32_GR and 32_ABGR Z formats. When no
reachable operation through a non-zero write mask reads the value, it
cannot change the result and is ignored. Without a test value export, a
reachable REPLACE_OP becomes REPLACE with the exported value as
FragStencilRefEXT (or the bounded replay on devices without the
extension), provided both faces compare without the test value and use
no ONES or REPLACE_TEST; early tests throw. The add and subtract
operations, and REPLACE_OP with a test value export, still throw.
The value is read from bits 8-15 of the second MRTZ channel: the
title's pixel shader shifts it left by 8 right before exp mrtz
(v_lshlrev_b32 v0, 8, v0). That is the only evidence for the layout and
it is listed as technical debt. The pixel stage info carries the new
flag into the cache key, the draw keys and request serialization
version 31.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
#6503 took request serialization 36 for its float32 flush flag, so this branch's sample order moves to 37. The version relabels in ShaderMemory.cpp and RequestSerializerTests.cpp drop both trailing flags and the sample order.
Devices with DenormFlushToZero for f32 run the mixed denormal-flush programs with and
without the hardware mode at startup; when every result matches, add, sub, mul, fma,
floor and ceil in the default rounding mode drop their software flush.
#4650 took request serialization version 35 for its depth remap flag, so this branch's sample order moves to 36. The serialization tests drop #4650's trailing byte from every older version they build, as main's do, along with the iteration fields and the sample order.
Combines this branch's sample order with #6470's once-per-pixel and once-per-sample inputs. The pixel request keeps both sides' fields: #6470's iteration, position location and sample locations are serialized at version 34, and the sample order after them at version 35. The serialization tests move the sample-order bytes after the iteration fields and drop both from older versions. The pixel info collector adds sample order to the SampleLocationsVary inputs, and the draw keys compare and hash both. EmitSampleOffset's iterated index already goes through GuestSampleIndex, since the SampleId builtin does. The shader cache stays at FormatVersion 40, and the sample-location debt entry takes #6470's sentence on the position inputs.
10_11_11_UNORM (format 30) images are read and written through an
R32_UINT view, so the hardware sampler cannot filter them. image_sample
of a 2D or 2D array view now fetches the one or four texels at LOD 0,
decodes them and blends them in the shader, with bilinear weights on
the 8-bit grid, for point or bilinear filtering with wrap or
clamp-to-edge addressing.
A shader gets the converted sample mode only after a converted
descriptor reaches one of its sampled images: that capture throws
ConvertedSampleRequired and the driver prepares the program again
with GuestContext::convertedSamples.
The R10_G11_B11 float format is sampled natively since #7271 and keeps
its refusal on the converted path.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
With robustBufferAccess2 and a robustStorageBufferAccessSizeAlignment of 1, the device
returns zero for a storage buffer load past the descriptor range, so the shader's own
OpArrayLength comparison against the same range is redundant. The guest V# bound check
stays. A guest buffer's and a draw snapshot's descriptor range of 4 bytes or more is
rounded down to whole dwords so both zeros agree. APS5_NO_EXACT_ROBUST_LOADS keeps the
comparison. The flag is part of the shader cache key.
Only the remap mode changes what the vertex shader compiles to. Scale and
offset are specialization constants that each materialization takes from
its own request, so they no longer split the compiled variant per depth
transform. The result memo hashes them with the snapshot instead, so one
prepared vertex shader does not hand the module specialized for one
transform to a draw with another. sameVertexInfo repeated its first
comparison without paClVsOutCntl; the repeat is dropped.
A test pins the version 27 layout (the remap fields are the last bytes
of a vertex request), a version 25 replay, truncation and trailing-data
refusal, and that scale and offset leave the cache key alone while the
mode does not. agc_depth_range_remap materializes one prepared vertex
shader with two transforms and checks that each gets its own module.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Without VK_EXT_depth_range_unrestricted a viewport depth range outside
[0, 1] threw. Minecraft (PPSA17221) draws its world with scale 1 and
offset 0 in negative-one-to-one clip space, so its depth is the raw z/w
clamped by PA_SC_VPORT_ZMIN/ZMAX. On such a device the vertex shader now
scales clip z so z/w is the console's depth, the pipeline clamps depth at
the console's bounds, and two clip distances restore the near and far
planes that depth clamping removes. Mode, scale and offset travel as
pipeline specialization constants from the draw's vertex stage info, in
both the per-draw and the registration-module path; mesh and tessellation
draws with such a range still throw.
The console's depth clamp bounds are clipped to [0, 1]: exact for a
normalized depth target or none, and reported once for a tested or
written float depth target, where a depth below 0 is stored as 0
(Minecraft clamps its float depth target to [-1, 1]). The shader
validator accepts the remap's two extra clip distances next to the one
a program may already export.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
A load selected instead of branched still accesses memory, so its fallback
element must be one no write can race with: scratch keeps element 0, LDS in
workgroup or function memory reads a sentinel dword appended after the
program's LDS and lock, and storage buffers read element 0xfffffffe, which
robustBufferAccess2 turns into a zero load without a memory access. The
device enables robustBufferAccess2 when it has it and the target carries the
flag; without it, and for LDS in device memory, buffer loads keep their
selections. The aligned high word of a buffer read re-reads the low word's
element instead of word 0. agc_guarded_memory_loads traces the emitted loads
in both device modes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Takes #6451's merge of main. Main's spanned-waves flag took request
serialization version 32, so the sample order moves to version 33; a
version 32 request keeps its spanned-waves flag and reads no sample order.
Multisampled color targets now gain storage usage on their first storage
binding; a target whose samples may be reordered still gets it at creation,
and the recorder's storage-usage checks expect that.
Dean