FindUserDataCalls required the s_mov copy of the user-data callee pointer to
sit in the call's basic block, so a compute shader that copies s[0:1] at entry
and calls after an execz guard was left with an unresolvable s_swappc_b64.
Fall back to reachingWriter, as FindTableCalls already does.
The UserDataCallAfterBranch execution test dispatches such a shader with two
callees and checks every lane's result.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Main took request serialization version 48 for the fragment barycentric
built-ins flag (#9193). The depth export clamp bounds are now written at
version 49 and read only for versions from 49, so a version 48 request as main
writes it still decodes, with default bounds. Tests: version 49 is the
emitted prefix, 50 the first unsupported one, a depth exporting request
without its bounds decodes at 48, 47 and 46 (the last two without main's
barycentric byte), and a version 48 two-sample dithered request decodes as
written.
Main took request serialization version 47 for the two-sample dithered
alpha-to-coverage value. The depth export clamp bounds are now written at
version 48 and read only for versions from 48, so a version 47 request as main
writes it still decodes, with default bounds. Tests: version 48 is the
emitted prefix, 49 the first unsupported one, and a version 47 two-sample
dithered request (with and without depth export) decodes as written.
MoltenVK exposes VK_KHR_fragment_shader_barycentric but its MSL step
rejects PerVertexKHR, so the extension is disabled there and a pixel
shader whose I/J VGPRs outlive their v_interp_p1/p2 pairs takes the
generated barycentric stages, which accept only triangle lists without
primitive restart on a device without geometry shaders.
A new target flag, fragmentBarycentricBuiltins, enables the extension
for its built-ins only. The host interpolation check then keeps the
leftover I/J reads (all perspective or all linear, at the fragment's
default position) and reads them from BaryCoordKHR or
BaryCoordNoPerspKHR, while the interpolated attributes stay hardware
varyings. Raw vertex parameters and I/J at an offset or a sample
position still take the generated stages. Apple GPUs return
unnormalized perspective weights unless the fragment function reads
[[position]], so each perspective read keeps a neutral use of
gl_FragCoord.w, loaded as a float. Devices with the full extension are
unaffected; the request serialization moves to version 45.
Co-authored-by: yripper <[email protected]>
A block that ended in s_code_end fell through to the next block. With
undecodable words skipped after s_code_end, that made a later branch
target a fall-through successor of s_code_end. The graph now starts a
block at the next decoded instruction after s_code_end and ends the
s_code_end block with a return, as it does for s_endpgm.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The RDNA program decode sweeps linearly up to the last s_endpgm past the
furthest direct branch target, so it also decoded words that no path
reaches and threw on any it could not decode. A word directly after
s_branch, s_endpgm or s_code_end that fails to decode is now skipped;
the decode still throws that word's error when a direct branch or
s_call_b64 targets it, and a failure on a word reachable by fall-through
is unchanged. The control-flow graph starts the block after s_branch and
s_endpgm at the next decoded instruction instead of the next word.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Bugsnax (PPSA01503) draws with DB_ALPHA_TO_MASK 0xe701 (quad offsets
3, 1, 2, 3 without OFFSET_ROUND) on a two-sample target and stopped on
the refusal of every dither but the eight-sample 0x18701. The pixel
shader now applies an eight-level per-quad table for it, measured
natively on gfx1032, and the request's dither byte selects that table
(serialization version 47). It is accepted only for two matched raster
and DB_EQAA samples, the two measured location pairs and FP16_ABGR or
32_ABGR exports.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The slot counter is reset only after earlier fragment writes, and the overflow flag stays until WaitIdle. The storage buffer is limited to NVIDIA's proprietary driver, which is where the large function array does not finish.
The DB clamps a pixel shader's exported depth to PA_SC_VPORT_ZMIN_0 and
ZMAX_0 unless DB_RENDER_OVERRIDE.DISABLE_VIEWPORT_CLAMP is set. Vulkan only
clamps with depthClamp, which also drops clipping, so the exported value
reached the depth attachment unclamped.
The pixel stage info now carries the two bounds when the shader exports
depth (0 and 1 otherwise), and the exported value is NClamped to them
before it is stored to gl_FragDepth. With
DISABLE_VIEWPORT_CLAMP the value is stored as exported; non-finite or
inverted bounds are rejected. DB_RENDER_OVERRIDE joins the draw key
registers. The bounds
are part of the recompile cache key (only for depth-exporting shaders, so
other keys do not change), the draw key and its register check, and
recompile requests (serialization version 47).
Tests: the pixel stage decode reads the bounds, passes the disable flag through
and rejects bad bounds, and leaves a draw without depth export at 0 and 1; a
recompiled depth export stores the NClamped value, and its bounds survive
serialization and key the cache.
Request version 47: main is at 44 (#8612) and open PRs also claim 44 to 46 (#8564, #7888, #9193 and the #9020, #9335, #9748 stack). Whichever of these lands second renumbers.
The recompiler marks every guest f32 add, subtract and multiply
NoContraction, because the guest rounds each one separately.
SPIRV-Cross turns each of them into a call to a helper that Metal
compiles without optimization, which dominates fragment time on
MoltenVK, and its multiply helper, fma(a, b, +0.0), turns a -0.0
product into +0.0.
On MoltenVK they are now GLSL Fma whose second step is exact, which
gives the separately rounded result: a * b is fma(a, b, -0.0), a + b
is fma(1.0, a, b) and a - b is fma(-1.0, b, a). The sin, cos and log2
series and the offset image coordinates and gather texel scale use the
same forms. The choice is a SpirvTarget flag set for
VK_DRIVER_ID_MOLTENVK, part of the shader cache key and of the request
serialization (version 44); other drivers are unchanged.
agc_driver_f32_separate_rounding checks a multiply feeding an add or a
subtract on rows where fusing them changes the result. It,
agc_driver_f32_arithmetic_nan and agc_driver_f32_denormal_flush also
run the fused forms on devices that do not use them, so lavapipe checks
them against the same rows.
Co-authored-by: Ycaro Gabriel <[email protected]>
unchangedUntil skipped a write by the reading instruction itself for every shader, so an RDNA s_load that overwrites its own address pair was taken as a user-data pointer load where main refused it. The skip now applies only to PS4 (GCN) shaders, where it is needed; PS5 shaders take main's path.
Each GCN instruction is re-encoded to its RDNA form and decoded by the
existing RDNA decoder, so the PS4 path shares the recompiler. Covers the
legacy VS stage (program at the VS slot, user SGPRs from s0, vertex and
instance index in v0-v3), fetch shaders linked as user-data calls, MIMG
(image_sample and friends), PS4 buffer descriptors (GCN DATA_FORMAT /
NUM_FORMAT, 44-bit base) and T# image descriptors (GCN tiling in word 7).
The GCN switch is keyed into the shader disk cache. PS5 behaviour is
unchanged: every PS4-only rule is gated on GcnGuestShaders().
hasStructuredConstructs tested every block of the graph against each merge header, and searched every block for a block's innermost loop, with dominator binary searches each time. Under APS5_FORCE_STRUCTURIZER_FALLBACK, Ghostrunner's compute shader 0x200c520000 becomes a 32,490-block reach-flag graph with 6,647 merge headers, and the check took 44 to 50 s of the 49 to 55 s.
The innermost loop is now searched among the loop headers only. Every other block failed the loop-header test first, so the candidates and their order are unchanged.
When constructsFollowDominance holds for the structural graph, the member checks visit only the blocks dominatedBlocks finds from the header, stopping at its merge, and for a loop also from its continue block, in block order. Only a block in the header's construct or in the loop's continue construct can fail a member check. A construct block is dominated by the header and not by the merge, so some path from the entry reaches it without the merge, and from the last header on that path every block is dominated by the header, so the walk reaches it. A continue-construct block is dominated by the continue block and is reached from it the same way. The walk finds every such block, and the blocks it skips passed every check, so the result is unchanged. Otherwise every block is checked as before, and APS5_NO_PERF_DOMINATOR_TREE keeps the old loop.
On an M4 Pro the check takes 0.27 s instead of 44 to 50 s on that graph, and the forced fallback takes 5.1 to 5.4 s instead of 49 to 55 s. The default structurization of the shader makes one check, which takes 0.03 s instead of 0.36 s.
After each split, every selection header that needs no work still walks its construct (dominatedBlocks) to learn which predecessors of its merge lie inside. On Ghostrunner's compute shader 0x200c520000 that is 1.7 million walks.
When every block is reachable from the entry and the predecessor lists match the successor lists, and the merge M neither is the header H nor dominates it, the walk finds exactly the blocks that H dominates and M does not, other than M. A block the walk reaches is dominated by H and reached from the entry on a path that avoids M, because some path to H avoids M. A block that H dominates and M does not lies on a path from the entry that avoids M, and from the last H on that path every block is dominated by H, so the walk reaches it. splitOneSelectionMerge checks those conditions once per pass and then asks the dominator sets directly; otherwise it walks as before. Only membership in the construct is used, so the split and its order are unchanged.
On an M4 Pro the shader's structurization takes 12 to 15 s instead of 18 s, and 176 s before this series.
computeStronglyConnectedComponents visited blocks recursively, one stack frame per block on the deepest path. The reach-flag fallback turns Ghostrunner's compute shader 0x200c520000 into a graph of 32,490 blocks, and with APS5_FORCE_STRUCTURIZER_FALLBACK the visit overflowed agc_shader_replay's 2 MiB stack after 4 s.
The visit now keeps its frames in a vector. It enters successors in the same order and closes components in the same order, so the components, their entry blocks and the irreducibility verdict are unchanged.
recomputeAnalyses runs after every merge split, and its dominator and post-dominator fixpoints intersected a bitset of every block for every block until nothing changed. They now build the tree with the Cooper-Harvey-Kennedy iteration over a reverse post-order from a virtual root above the entry block, or above every exit for post-dominators, and write each block's sorted set as its parent's set plus the block itself.
The sets are the same. When every block is reachable from the virtual root, the greatest fixpoint of the old equations is the dominance relation of that rooted graph, which the tree describes. When a block is unreachable from the entry, or cannot reach an exit, the old fixpoint runs as before.
The post-dominator tree also keeps each block's immediate post-dominator, and FindNearestCommonPostDominator climbs it when both blocks have one: the sets are the tree's paths, so the deepest common block is the one the set intersection found. Blocks added or cloned after the analysis have no parent and take the set path, as before.
APS5_NO_PERF_DOMINATOR_TREE turns both off, and the structurizer and control-flow tests also run with it set. On an M4 Pro the structurization of Ghostrunner's compute shader 0x200c520000 takes 18 s instead of 37 s.
Merge splitting re-examines every selection header after each split, so the lookups behind each examination run millions of times on a large graph. On Ghostrunner's compute shader 0x200c520000 (4,750 blocks growing to 10,170 over 1,302 splits) agc_shader_replay spent 173 s in splitSharedMergeBlocks. A profile put most of it in linear membership tests and in FindNearestCommonPostDominator, not in the analyses.
Loop bodies, construct blocks, selection regions and strongly connected components are sorted, so membership in them is now a binary search. The traversals that tested membership in the list they were building (selectionRegion, reachableWithin, naturalLoopBody) and externalEntryEdges test a bitset instead, and selectionRegion reads its sorted result from the bitset. dominatedBlocks tests the header once and pushes a successor only before it is visited. FindNearestCommonPostDominator skips a candidate whose post-dominator set is smaller than the other common post-dominators and checks the rest in one pass over the set. remapIds sorts only when the map broke the order.
Each change keeps the same sets in the same order and the same first candidate, so the structured graph is unchanged. On an M4 Pro the shader's structurization takes 37 s instead of 176 s.
main took request versions 33 to 42 for other fields, the last of them
the pixel sample order, so requests of those versions from main have no
point-list tail. The point fields now come with version 43, after them,
and the tests expect 43 and reject 44.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Dean