A registered type-3 hull program holds the local and the hull part of a
tessellation draw in one binary, addressed through SPI_SHADER_PGM_LO_LS
(SH 0x148) with no SPI_SHADER_PGM_LO_HS: the local part runs on the lanes
below s3[7:0] and writes LDS, an s_barrier follows, and the hull part runs
on the lanes below s3[15:8] and may store the registers the local part
filled. The draw decode asked 0x148 for a type-5 local half, and
PPSA11386 (007 First Light) stopped in its level load with "registered
program refers to an incompatible shader binary type".
The draw decode now gives a registered type-3 program both the local and
the hull role and marks the tessellation configuration mergedHull. The
local stage only passes gl_VertexIndex and gl_InstanceIndex in a 16-byte
record. The hull stage runs the whole program, one invocation per output
control point: v2 and v5 come from the record of control point
min(gl_InvocationID, inputs - 1), v3 is the relative vertex id, and the
LDS becomes a per-control-point output array that a control point writes
in its own record and reads in any record of its patch after the barrier.
Registers live across the barrier stay in the invocation.
The hull stage of a merged program runs at one lane per invocation on
every device (HostSubgroupSize 1). Its lanes are gl_InvocationID, not host
subgroup invocations, so the s_cbranch_execz around the local part tests
the invocation's own lane. With the device's subgroup it took a subgroup
ballot, and lavapipe, whose subgroup stages lack the tessellation control
stage, rejected the draw ("subgroup capability 61 is unsupported for
shader stage 2").
The lowering refuses more input than output control points, LDS writes
outside the own record, reads outside the patch's records, barriers that
some invocations do not reach or that sit in a loop, reads or writes of
another lane's register, outputs beyond the device limits, and fetch
subroutines. Type 3 is prepared with its stage group like type 5 and
takes its float mode from RSRC1_HS. The hull user-data address (SH
0x102/0x103) is optional for a merged program: s[0:1] take it, unchecked,
when the draw sets it and are zero otherwise.
Both stages refer to one binary, so a draw lists each code and header
region once (AppendProgramMemory); a request's memory regions must not
overlap. An indirect draw's record words go into every program whose user
data holds the register, so the hull stage of a merged program reads
them too.
mergedHull is part of the recompile cache key, the draw decode comparison
and the request serialization (version 39).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Capture evaluated every flattened scalar slot even when a clean scalar
branch skipped it, so an inaccessible direct user-data root refused the
draw. Record flat-slot uses on resource control-flow blocks and evaluate
only the reachable ones; keep refusing taken and unresolved paths.
Fixes#8272.
Back 1D and 1D-array depth comparison textures with 2D Vulkan images and views of height 1. Drivers such as AMD RADV reject 1D depth images (VK_ERROR_FORMAT_NOT_SUPPORTED). The shader recompiler maps 1D depth comparison samplers to 2D image descriptors, and pins the normalized vertical coordinate to 0.5 to sample the center of row 0 without hitting borders.
A ds_ordered_count with offset1 bit 4 set swaps the first active lane's value into the GDS counter instead of adding it. Emit it as one device-scope atomic exchange through the same path as the ordered add, and return the counter's previous value to every active lane.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
SPI_PS_INPUT_ENA/ADDR bit 14 (SAMPLE_COVERAGE_ENA) gives the pixel shader one
VGPR, after ANCILLARY, holding the samples the primitive covers. The decoder
refused it and the recompiler never loaded the VGPR. It is now loaded from
gl_SampleMaskIn[0] (sharing the input the partial quad sample mask already
declares). On a multisampled draw the hardware word is the whole pixel's
coverage while gl_SampleMaskIn holds only the invocation's samples when the
pixel shader runs per sample, so that case, and sample-rate or ancillary inputs
beside it, are refused.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The data-hit verification keeps this branch's pending-span query on its ShaderMemory and takes #5958's CaptureInvocation, which retries a capture with converted samples.
Keeps this branch's tagVariantRuns declaration next to main's dumpRequest suffix parameter in Driver.hpp, and reads #6232's pointer-chain test words from its table's snapshot, where this branch keeps them.
Resolves the shader debt file bullet by bullet: main's EmitSetAttribute entry, and this branch's PERSP_SAMPLE/LINEAR_SAMPLE and pixel-position entries.
#7527's selected-record limit test reads a record's descriptor words through the
table snapshot's shared words list; this branch keeps a snapshot's words per column,
so the test reads them from its column instead.
10_11_11_UNORM (format 30) images are read and written through an
R32_UINT view, so the hardware sampler cannot filter them. image_sample
of a 2D or 2D array view now fetches the one or four texels at LOD 0,
decodes them and blends them in the shader, with bilinear weights on
the 8-bit grid, for point or bilinear filtering with wrap or
clamp-to-edge addressing.
A shader gets the converted sample mode only after a converted
descriptor reaches one of its sampled images: that capture throws
ConvertedSampleRequired and the driver prepares the program again
with GuestContext::convertedSamples.
The R10_G11_B11 float format is sampled natively since #7271 and keeps
its refusal on the converted path.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Every IR value kept two lists of its users: uses (the users) and
operandUses (the users with their operand index). AddArgument,
ReplaceArgument and ReplaceAllUsesWith updated both, and each value
allocated and freed both. Only operandUses is kept now. Uses() is a view
of the users in operandUses, and HasUses, UseCount and Invalidate read
it. sizeof(IrValue) drops from 152 to 128 bytes; the plan value encoder
writes no use lists, so only its size check changes.
The two lists held the same users in the same order, except after
ReplaceArgument on a user that took the value in more than one
operand: uses dropped the user's first entry and operandUses the
replaced operand's entry. Uses() now follows operandUses there too.
shader_ir_use_rewrite counted the allocations of both lists: its
allocation failure check now fails the one reserve ReplaceAllUsesWith
makes, and its growth check allows the allocations of one list, which
main exceeds.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Every IR value grew its argument vector one AddArgument at a time from
empty, so a value with three arguments allocated three times (capacity
1, 2 and then 4) and freed the first two. The constructor now reserves
IrOpcodeOperandCount(opcode) arguments, the count IrBuilder::Emit
already requires, so a value built with its opcode's arguments
allocates once. Opcodes without fixed arguments (constants, phis)
reserve nothing, as before. The IR and the SPIR-V are unchanged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
IrProgram::ReclaimUnusedValues put every live value into an
unordered_set, one allocation each, and looked every value of the
program up in it again. Values enter the program only through
CreateValue, which numbers them with nextValueId, so an id names at
most one of them. The marks of those values now live in a vector
indexed by id, after a check that the id's owner is the same pointer.
A value the program does not own, such as one in the resource plan's
value storage, which may reuse an id, still goes into a set. The values
kept and removed are unchanged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Compute LDS beyond the device's shared-memory limit lives in a storage
buffer, and its 64-bit atomics took a lock in that buffer. AMD's Windows
driver compiles the lock's acquire and release to s_barrier, which inside
the divergent lock loop pairs with the program's own barriers, so the
device-memory case of agc_driver_lds_lock_budget read half its sum.
With shaderBufferInt64Atomics the device-memory LDS now uses a 64-bit view
of the buffer instead: 64-bit atomics are compare-exchange loops on it and
32-bit atomics stay native, so no lock is taken. Without the feature the
lock is kept. The per-workgroup slice is rounded up to an even dword count
so that the 64-bit view stays aligned.
#6243's new head merges main f1af6c5915, whose #7325 adds
verifyChainedTableOffset next to the selected-record tests this
branch carries from #6243. That test reads the record words from the
snapshot's shared word list; this branch keeps the words with each
table's snapshot, so the merge reads them from the table it checks.
Main's #7325 adds verifyChainedTableOffset to the image table tests
where this branch adds its selected-record tests. Keep both, with
main's call after this branch's.
An implicit-LOD or gradient sample with a texel offset through an
unnormalized S# was refused at bind time with "unnormalized guest sampler
is used by an implicit-LOD sample". PPSA01317 aborts on boot on such a
pixel image_sample_o. The base-level path already samples these through
OpImageSampleExplicitLod with LOD 0, which Vulkan allows for unnormalized
samplers, but it dropped the offset. It now adds the offset to the
coordinates in that branch, as the explicit-LOD offset fold does, so a
sampler whose uses also include offsets takes the base-level path. A
normalized S# keeps the offset operand.
AI-assisted: yes
Moves a few include lines and the ResolveImageTablesShared declaration by a line or two, so they no longer share an insertion point with other open pull requests. No behaviour change.
The ancillary input's sample number (bits 11:8) was the SampleId
builtin in every pixel shader. Vulkan enables sample shading for a
shader that uses SampleId, so a shader that reads the sample field or
the whole ancillary word ran once per sample on a multisampled draw
even when the guest runs it once per pixel (PS_ITER_SAMPLE clear), and
FragCoord, and with it POS_X_FLOAT and POS_Y_FLOAT, moved to the
sample. The sample number is now SampleId only when the request says
the guest iterates the shader, and otherwise 0, the iterated sample
number of a pixel the hardware does not iterate.
The other inputs that depend on the sample follow where the console
evaluates them:
- PERSP_SAMPLE and LINEAR_SAMPLE are interpolated with
InterpolateAtOffset at sample 0's offset in a shader the guest does
not iterate, and in an iterated one at the offset of the host's
SampleId, read from a table of the guest's locations, because Vulkan
leaves plain barycentrics under sample shading at an
implementation-defined position (lavapipe uses the pixel centre).
- PERSP_CENTER and LINEAR_CENTER are interpolated at offset 0 in an
iterated shader.
- POS_X_FLOAT and POS_Y_FLOAT follow SPI_BARYC_CNTL: the centre is
floor(FragCoord) + 0.5 under sample shading, a sample position in a
shader the guest does not iterate is the centre plus sample 0's
offset, and POS_FLOAT_ULC gives the upper-left corner.
The offsets need InterpolationFunction (sampleRateShading) and the
fragment-shader barycentric path; devices without them keep the host's
default positions. The shader disk cache format goes to 39.
Brings the series onto main 4e9a2a871 (#1193, #1201, #1202, #6060,
#6333 and the descriptor word tracking of 682b07eb0).
Resolutions:
- ShaderMemory: the column memo's supplied and spilled blocks now sit
on #1193's page vector, page pool and word masks. Pages are found by
binary search, since a spilled block can add many pages, and a block
placed into a lazily read page fills the rest of the 16-word blocks it
touches, so a later lazy read never overwrites placed words.
- ResourceMaterializer and SharedValues: the selected-record path and
the words index carry availableDwordMask, as the snapshot hash does.
A selected record's dwords past the end of its buffer count as read
zeros, which is what a scalar buffer load returns there, so a key past
the heap still selects a null image without reading the heap.
- DescriptorCache: kept table sets live beside #1201's recycled sets.
- Tests and CMake: both sides' tests are kept.
A capture that reused a memoized image table column still copied it: each record's state and code
and the distinct T#s went from the memo into the snapshot, the snapshot rebuilt its codes from the
states, and the column's per-record vectors were allocated before the memo was even looked up. Every
Recompile then hashed the codes and the distinct T#s word by word for the result memo. On a
32,768-record column of distinct T#s that was about 155 us of copying per capture and 410 us of
hashing per Recompile here, for a column nothing wrote.
A column is now built once into immutable shared values (SharedValues: its codes, its distinct T#s
and its read ranges) and the memo keeps that column. A capture that reuses it takes a reference, so
two captures of an unchanged table share one column and copy nothing, and the per-record vectors
are allocated only when the column is walked. The codes index the column's own distinct T#s
(ImageTableColumnSnapshot::words) instead of a list shared by the shader's columns, so every column
of a shader is shared whole, not only the first. A column read again after a write is a new one:
the snapshots holding the old one keep it unchanged. The result memo hashes each shared value once
and keeps the hash in it; equal contents hash equal whether they are shared or not.
ImageTableReuseCounts gains the reuses that shared a column and the columns built.
The per-draw capture of a shader's memory (the SRT walk and the guest pages it reads) cost about
15 us per draw packet in GTA V Performance mode; it now costs about 6.6.
- The SRT evaluator memoizes per plan value in a thread's reused, stamped array instead of a hash
table allocated and grown on every walk (evaluate 5.3 -> 3.6 us per capture).
- ExtractPlan also stores the plan's values as 32-byte records with argument ids, and the walk
evaluates that compact copy: the IR walk spent most of its time in cache misses (3.6 -> 2.7 us).
The evaluator is a template over the node view, so both walks share every opcode's semantics, memo,
cycle and failure rules. A plan that cannot be expressed compactly keeps the IR walk.
APS5_VERIFY_SRT_COMPACT=1 walks both and throws on any difference.
- ShaderMemory's valid/read/recent marks are arrays of 64-bit words scanned with countr_zero instead
of bitsets tested bit by bit (about 1 us per draw packet).
- Captured pages come from a per-thread pool, are found through a short vector with a last-page
shortcut, and a page no recorded GPU work writes gets the flush hook a whole-page read would make,
then has each 64-byte block copied when a word of it is first read (about 0.3 us per packet).
Pages with pending GPU writes and unmapped pages are read as before.
- APS5_PROFILE_DRAW splits the capture time into SRT evaluation and materialization, and prints the
[capture] line once per 10 s instead of every 500 captures (it cost about 2 us per capture).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Gjon Gojcevic <[email protected]>
Since #757, a T# loaded from a buffer heap at a runtime offset is an
image table column, and every capture snapshots the whole heap, up to
65,536 records, even when user data or an SRT read fixes the offset.
When the T#'s eight dwords and its offset are runtime values the SRT
walker evaluates, the column is now selected: the offset is kept as a
descriptor source, the shader indexes entry 0, and the snapshot reads
only the record at the evaluated offset, with the heap base moved to
that record.
The column keeps a table's semantics. An invalid T#, an unmapped record
or an offset that carries past 32 bits is poison that faults only when a
lane selects it; an offset the walker cannot read faults the table
instead of throwing; the record's mode is picked per entry, so no
pipeline variant is added; and a dispatch that writes the record faults
it as written by its dispatch. Dwords past the heap read zeros, as the
scalar loads would, and a record past the key cap is read rather than
poisoned.
A column stays a whole table when its offset is chosen per lane, or when
the shader may write the memory the offset is read from at run time: the
buffer it is read from (by V#), or any memory when the read or a write
goes through a pointer or a GPU-only descriptor. Samplers are unchanged:
an S# the walker evaluates is already a direct sampler. The plan stores
the selection, so the disk cache format is 36.
Five agc_shader_memory and agc_shader_bindless checks asserted the old
planning and now assert the selected record:
- "image table SRT key": a key read through the SRT narrowed a 4-key
table to records 0 and 2. It now selects record 2 alone, captures no
other record, and a changed key moves the capture without changing the
module or its disk key.
- "image table key union" and "image table late key": two keys of one
heap shared one column with two value key domains. Each key now
selects its own record, as two one-record tables.
- "image table unkeyed access": a read keyed through the SRT shared a
whole column with a read keyed per lane. The first now selects its
record; the second keeps the whole, unnarrowed table.
- "uniform table image": the T# at a user data offset was one key of a
full table. It is now the one selected record, and the record the user
data does not select is not read.
A value key domain only arises for a key the walker evaluates, so it no
longer narrows an image column; key buffer domains (keys loaded per
lane) still do.
