A block that ended in s_code_end fell through to the next block. With
undecodable words skipped after s_code_end, that made a later branch
target a fall-through successor of s_code_end. The graph now starts a
block at the next decoded instruction after s_code_end and ends the
s_code_end block with a return, as it does for s_endpgm.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The RDNA program decode sweeps linearly up to the last s_endpgm past the
furthest direct branch target, so it also decoded words that no path
reaches and threw on any it could not decode. A word directly after
s_branch, s_endpgm or s_code_end that fails to decode is now skipped;
the decode still throws that word's error when a direct branch or
s_call_b64 targets it, and a failure on a word reachable by fall-through
is unchanged. The control-flow graph starts the block after s_branch and
s_endpgm at the next decoded instruction instead of the next word.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Bugsnax (PPSA01503) draws with DB_ALPHA_TO_MASK 0xe701 (quad offsets
3, 1, 2, 3 without OFFSET_ROUND) on a two-sample target and stopped on
the refusal of every dither but the eight-sample 0x18701. The pixel
shader now applies an eight-level per-quad table for it, measured
natively on gfx1032, and the request's dither byte selects that table
(serialization version 47). It is accepted only for two matched raster
and DB_EQAA samples, the two measured location pairs and FP16_ABGR or
32_ABGR exports.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The slot counter is reset only after earlier fragment writes, and the overflow flag stays until WaitIdle. The storage buffer is limited to NVIDIA's proprietary driver, which is where the large function array does not finish.
The recompiler marks every guest f32 add, subtract and multiply
NoContraction, because the guest rounds each one separately.
SPIRV-Cross turns each of them into a call to a helper that Metal
compiles without optimization, which dominates fragment time on
MoltenVK, and its multiply helper, fma(a, b, +0.0), turns a -0.0
product into +0.0.
On MoltenVK they are now GLSL Fma whose second step is exact, which
gives the separately rounded result: a * b is fma(a, b, -0.0), a + b
is fma(1.0, a, b) and a - b is fma(-1.0, b, a). The sin, cos and log2
series and the offset image coordinates and gather texel scale use the
same forms. The choice is a SpirvTarget flag set for
VK_DRIVER_ID_MOLTENVK, part of the shader cache key and of the request
serialization (version 44); other drivers are unchanged.
agc_driver_f32_separate_rounding checks a multiply feeding an add or a
subtract on rows where fusing them changes the result. It,
agc_driver_f32_arithmetic_nan and agc_driver_f32_denormal_flush also
run the fused forms on devices that do not use them, so lavapipe checks
them against the same rows.
Co-authored-by: Ycaro Gabriel <[email protected]>
unchangedUntil skipped a write by the reading instruction itself for every shader, so an RDNA s_load that overwrites its own address pair was taken as a user-data pointer load where main refused it. The skip now applies only to PS4 (GCN) shaders, where it is needed; PS5 shaders take main's path.
Each GCN instruction is re-encoded to its RDNA form and decoded by the
existing RDNA decoder, so the PS4 path shares the recompiler. Covers the
legacy VS stage (program at the VS slot, user SGPRs from s0, vertex and
instance index in v0-v3), fetch shaders linked as user-data calls, MIMG
(image_sample and friends), PS4 buffer descriptors (GCN DATA_FORMAT /
NUM_FORMAT, 44-bit base) and T# image descriptors (GCN tiling in word 7).
The GCN switch is keyed into the shader disk cache. PS5 behaviour is
unchanged: every PS4-only rule is gated on GcnGuestShaders().
hasStructuredConstructs tested every block of the graph against each merge header, and searched every block for a block's innermost loop, with dominator binary searches each time. Under APS5_FORCE_STRUCTURIZER_FALLBACK, Ghostrunner's compute shader 0x200c520000 becomes a 32,490-block reach-flag graph with 6,647 merge headers, and the check took 44 to 50 s of the 49 to 55 s.
The innermost loop is now searched among the loop headers only. Every other block failed the loop-header test first, so the candidates and their order are unchanged.
When constructsFollowDominance holds for the structural graph, the member checks visit only the blocks dominatedBlocks finds from the header, stopping at its merge, and for a loop also from its continue block, in block order. Only a block in the header's construct or in the loop's continue construct can fail a member check. A construct block is dominated by the header and not by the merge, so some path from the entry reaches it without the merge, and from the last header on that path every block is dominated by the header, so the walk reaches it. A continue-construct block is dominated by the continue block and is reached from it the same way. The walk finds every such block, and the blocks it skips passed every check, so the result is unchanged. Otherwise every block is checked as before, and APS5_NO_PERF_DOMINATOR_TREE keeps the old loop.
On an M4 Pro the check takes 0.27 s instead of 44 to 50 s on that graph, and the forced fallback takes 5.1 to 5.4 s instead of 49 to 55 s. The default structurization of the shader makes one check, which takes 0.03 s instead of 0.36 s.
After each split, every selection header that needs no work still walks its construct (dominatedBlocks) to learn which predecessors of its merge lie inside. On Ghostrunner's compute shader 0x200c520000 that is 1.7 million walks.
When every block is reachable from the entry and the predecessor lists match the successor lists, and the merge M neither is the header H nor dominates it, the walk finds exactly the blocks that H dominates and M does not, other than M. A block the walk reaches is dominated by H and reached from the entry on a path that avoids M, because some path to H avoids M. A block that H dominates and M does not lies on a path from the entry that avoids M, and from the last H on that path every block is dominated by H, so the walk reaches it. splitOneSelectionMerge checks those conditions once per pass and then asks the dominator sets directly; otherwise it walks as before. Only membership in the construct is used, so the split and its order are unchanged.
On an M4 Pro the shader's structurization takes 12 to 15 s instead of 18 s, and 176 s before this series.
computeStronglyConnectedComponents visited blocks recursively, one stack frame per block on the deepest path. The reach-flag fallback turns Ghostrunner's compute shader 0x200c520000 into a graph of 32,490 blocks, and with APS5_FORCE_STRUCTURIZER_FALLBACK the visit overflowed agc_shader_replay's 2 MiB stack after 4 s.
The visit now keeps its frames in a vector. It enters successors in the same order and closes components in the same order, so the components, their entry blocks and the irreducibility verdict are unchanged.
recomputeAnalyses runs after every merge split, and its dominator and post-dominator fixpoints intersected a bitset of every block for every block until nothing changed. They now build the tree with the Cooper-Harvey-Kennedy iteration over a reverse post-order from a virtual root above the entry block, or above every exit for post-dominators, and write each block's sorted set as its parent's set plus the block itself.
The sets are the same. When every block is reachable from the virtual root, the greatest fixpoint of the old equations is the dominance relation of that rooted graph, which the tree describes. When a block is unreachable from the entry, or cannot reach an exit, the old fixpoint runs as before.
The post-dominator tree also keeps each block's immediate post-dominator, and FindNearestCommonPostDominator climbs it when both blocks have one: the sets are the tree's paths, so the deepest common block is the one the set intersection found. Blocks added or cloned after the analysis have no parent and take the set path, as before.
APS5_NO_PERF_DOMINATOR_TREE turns both off, and the structurizer and control-flow tests also run with it set. On an M4 Pro the structurization of Ghostrunner's compute shader 0x200c520000 takes 18 s instead of 37 s.
Merge splitting re-examines every selection header after each split, so the lookups behind each examination run millions of times on a large graph. On Ghostrunner's compute shader 0x200c520000 (4,750 blocks growing to 10,170 over 1,302 splits) agc_shader_replay spent 173 s in splitSharedMergeBlocks. A profile put most of it in linear membership tests and in FindNearestCommonPostDominator, not in the analyses.
Loop bodies, construct blocks, selection regions and strongly connected components are sorted, so membership in them is now a binary search. The traversals that tested membership in the list they were building (selectionRegion, reachableWithin, naturalLoopBody) and externalEntryEdges test a bitset instead, and selectionRegion reads its sorted result from the bitset. dominatedBlocks tests the header once and pushes a successor only before it is visited. FindNearestCommonPostDominator skips a candidate whose post-dominator set is smaller than the other common post-dominators and checks the rest in one pass over the set. remapIds sorts only when the map broke the order.
Each change keeps the same sets in the same order and the same first candidate, so the structured graph is unchanged. On an M4 Pro the shader's structurization takes 37 s instead of 176 s.
main took request versions 33 to 42 for other fields, the last of them
the pixel sample order, so requests of those versions from main have no
point-list tail. The point fields now come with version 43, after them,
and the tests expect 43 and reject 44.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A point list routed through the vertex path was refused, and so was
PA_CL_VS_OUT_CNTL.USE_VTX_POINT_SIZE on it. OliSe Player (PPSA01153)
stops on the latter with two of its MilkDrop presets.
The vertex path now draws point lists as Vulkan points. The vertex stage
writes PointSize: the misc export's X clamped to PA_SU_POINT_MINMAX when
USE_VTX_POINT_SIZE is set, and the PA_SU_POINT_SIZE height otherwise.
The point state travels in the recompile request (version 31) and the
cache and draw keys. Non-square register sizes, inverted clamps, point
sprite texture coordinates and, without largePoints, sizes other than
one pixel throw.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The recompiler asked for DenormPreserve only for 16-bit and 64-bit floats,
so a device that flushes 32-bit denormals by default (RADV, MoltenVK) flushed
them whatever the guest's float mode said. target.float32DenormPreserve is set
from shaderDenormPreserveFloat32 when denormBehaviorIndependence allows a 32-bit
mode, is part of the cache key and of the request serialization (version 41),
and makes the module declare DenormPreserve 32 unless the 32-bit flush is
already taken in hardware. APS5_NO_HW_F32_PRESERVE=1 turns it off.
The refusal of a merging shader that writes guest memory under late
tests fired in two cases where the early tests change nothing.
The first was a draw with no depth, stencil or depth-bounds test. Early
tests reject no pixel there, so every write the console makes still
happens. The target merge now carries whether the draw has such a
test, and the refusal applies only then. This adds one byte to the
version 40 merge fields of the recompile request.
The second was any GDS binding, including a shader that only reads GDS.
The refusal now looks for GDS writes, atomics, appends and consumes.
Tests: the depth and stencil decode checks the new flag, and the
static shader key depends on it. A counter-adding shader under late
tests compiles without a test and throws with one. A late-test merge
that reads GDS compiles, and one that adds to GDS throws.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A packed 4_4_4_4 partial write merges inside the pixel shader after early
depth and stencil tests. Moving the tests ahead of a pixel kill only changes
the result when the draw writes depth or stencil, so the target merge now
carries whether the draw does (depth write, or stencil with a write mask and
a non-keep operation), and the late-test kill refusal applies only then.
The flag is part of the static shader key, the serialized request (now
version 40) and the pixel input info.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A CB_TARGET_MASK that writes fewer than four components of a 4_4_4_4
target leaves the other nibbles of each packed word as they were, which
an R16_UINT attachment cannot do. Such draws now close the attachment's
color write mask, and the pixel shader merges the written nibbles into
the target through a storage image: inside a pixel interlock it reads
the word, keeps the bits the mask leaves alone and stores it back.
The merge (export index, word mask, sample count) is part of the static
shader key, request version 40 and shader disk cache format 42; the
storage image is appended to the shader's images with a single runtime
mode, and its T# is encoded from the target's surface at draw time, so
the draw resolves it to the target's resident image, which Draw
requires. Merging shaders declare early fragment tests; shaders that
kill under late tests, export depth, stencil or coverage, use
alpha-to-coverage or a quad sample mask are refused, as are
multisampled, mipmapped, volume and array targets and more than one
merged target per draw.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
CB_COLOR_INFO format 19 with UNORM and COMP_SWAP 2 lists A, B, G, R from the
least significant bits. No Vulkan device is required to render into the 4-bit
formats and NVIDIA exposes none of them as color attachments, so the target
is an R16_UINT image and the pixel shader packs the exports into the guest
layout, as the 10_11_11 target does. Writes of fewer than four components,
blending, DCC and truncating rounding are refused.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Adrià