Dean Galvin b57ab5a0e9 feat(recompiler): structurize externally entered selection regions by cloning or route variables ...

A selection whose region (the blocks reachable from its arms before its
merge) is also entered from outside the header cannot be expressed as a
SPIR-V selection construct, and the structurizer gave up with "semantic
block cloning is disabled". Pixel shaders hit this through a shared
discard tail (s_mov_b64 exec, 0 / exp null / s_endpgm) that several
selections branch to, compute shaders through short-circuit conditions
that jump into a shared body, and ray-traced compute passes through an
s_cbranch_execz that skips past a join into the middle of a selection.

Such a region is made single-entry in one of two ways, chosen per region
by its estimated translated size:

- Cloning (node splitting): the blocks the outside edges reach inside the
  region are copied and the outside edges redirected to the copies. When
  the header's merge is a shared linear return tail, giving the header a
  private copy of that tail is tried first. Each block now carries an
  estimate of the SPIR-V words it translates to. The per-instruction
  weights come from a fit of the recompiler's per-block output (before
  spirv-opt, phis excluded) over 8,300 blocks of captured shaders: VALU
  60, buffer and flat 120, SMEM 180, image 220, LDS 400. SALU (8) and
  exports (20) fit near zero and keep a small weight. Lane ops (120) and
  image_bvh_intersect_ray (2000) were too rare there to fit and are
  weighted for the wave and traversal expansions they need. A region is cloned only when it is at most 1024 estimated
  words and at most an eighth of the program (at least 256), and while
  all clones stay within a quarter of the program.
- Routing through a junction, for larger regions: every edge into the
  entered part (from outside, and from the header's own arms) and every
  edge from the arms to the merge is sent through a new empty junction
  block. Each rerouted edge first passes empty blocks that assign
  Boolean route variables (the IR's goto variables); the junction tests
  them in order to reach the edge's original target. The entered part
  then follows the junction and is emitted once. Branch conditions are
  still evaluated only at the end of their original blocks: a route
  variable records the edge a block already took, so a later SCC, VCC or
  EXEC write cannot change the route, and in the two-lane wave64 layout
  the source branch is combined across both halves as before while the
  route variable holds the same constant in both. Routing is refused
  when a rerouted edge is a back edge or leaves an indirect branch, when
  the region wraps around to a dominating block, or when it would change
  the natural loops or make the graph irreducible; cloning bounded by the
  program's estimated size is the fallback, and otherwise it throws with
  the header, the entered block and both reasons.

Selections whose arms only meet again after private return tails had no
post-dominator merge. When the existing rules find none, the merge is
now looked up with edges into private return tails ignored, or taken as
the other arm when one arm only returns; the candidate must still exit
the construct only through the merge. applyBlockOrder also remaps
indirect branch targets.

agc_shader_control_flow_tests (under ctest) covers hand-assembled
gfx1030 programs: the cloned shapes above, and routed ones (an
s_cbranch_execz into an expensive region with a loop, a region inside a
loop, nested entered regions, a shared body reached after SCC is
overwritten, early returns beside an entered region, an arm that only
returns). It checks that every selection construct exits only through
its merge, follows route variables to check that the structured graph
runs the same instruction ranges under the same branch conditions as
the original, bounds the cloned instructions, recompiles each program
for 64- and 32-lane subgroups, and keeps the routed execz program within
10% of the SPIR-V of the same program without the entering branch.
Given .req files it checks their graphs instead. agc_shader_replay --cfg
prints a request's graph, now with each block's estimate, before and
after structurization.

2026-10-01 08:36:11 +13:00

..

ShaderReplay.cpp

feat(recompiler): structurize externally entered selection regions by cloning or route variables

2026-10-01 08:36:11 +13:00