The [dispmove] line printed refused 0 next to a rebuild count that came mostly from PrepareDispatch dropping the hit. The line now reports the refusals at stage A and at record separately, each with its reasons.
Read-only addresses are not part of the dispatch key, so the fresh build after a refused
move replaced the template that serves the other address, and the two addresses
rebuilt each other. A refused dispatch now uses or builds an entry under the exact key
(read-only bases included) and leaves the movable template alone. The [dispmove] line
counts the rebuilds after a refusal, the builds inserted beside the template and the
dispatches served by an exact entry. Restores the MarkGpuWrites comment.
A read-only V# the build clamps (size above the descriptor range limit cut to the
mapped end, stride overhang cut to the allocation end) was compared with the template
by its raw size, so an unchanged buffer counted as moved, was refused over the limit
and the dispatch rebuilt every time; a moved one with an overhang was snapshotted past
the allocation end. The build and the move proof now take the range from one function
(ShaderResources::guestBufferBytes), compare clamped with clamped and snapshot the
clamped range.
An overlay without a descriptor set is refused like one that could not be made, and
recording one throws instead of binding the template's set.
The content key of a compute stage leaves the base address and size of its
read-only guest buffers out, as a draw's key does. A template hit whose such
buffers moved is proved the way a draw's is (ShaderResources::MovedDispatchBuffers:
read-only, read in place when built, in reach, nothing pending over the new range)
and binds snapshots of the new ranges in a descriptor set of its own, kept with the
batch that records the dispatch. A refused proof builds the dispatch fresh, as
before. APS5_NO_DISPATCH_MOVE=1 turns the path off, APS5_DISPMOVE_MAX_KIB caps the
bytes snapshotted per dispatch.
A template whose storage image lies over memory that a depth surface covers is no
longer proved from stamps (fastRevalidate): cachedStorageTexture answers such a
binding with the plane's colour mirror, copied in from the depth image for every use,
and nothing stamps the appearance of a depth surface after the build. The full walk
decides, and a stale image can no longer be copied over the depth image.
Tests: Graphics.cpp (verdicts of the proof), execution DispatchMove (real device:
moved, exact, shortened, stored-over, own-output and written-range refusals) and
DispatchMoveDepth (a depth surface appearing over a template's plain storage image,
planes inside and outside the write watch, imports watched and unwatched).
On Windows the atexit handlers of a DLL run after the process has ended
its other threads, so the logger thread can be gone with lines still
queued and the unbounded Flush waits forever (guest_shader_link timed
out on 4 of the last 7 completed main builds since #6371). Run the bounded
FlushAtEnd there instead.
Co-Authored-By: Claude <[email protected]>
APS5_SPANNED_WAVE is now read once by the driver, which sets
SpirvTarget::spannedWaves only on a device with 32-lane compute subgroup
control and computeFullSubgroups. The recompiler no longer reads the
environment. The field is part of the cache key and of the request
serialization (version 32), so a cached or replayed module keeps the mode
it was built in.
A spanned module numbers its lanes SubgroupId * 32 +
SubgroupLocalInvocationId, which needs two full 32-lane subgroups. The
artifact records this as fullSubgroups (disk cache and pipeline identity
included), and the driver creates those pipelines with
REQUIRE_FULL_SUBGROUPS. Compute pipeline captures accept the flag with a
required subgroup size.
The two-lane fallback now catches only SpannedWaveUnsupported, thrown for
a shuffle wider than 32 bits; other emit errors surface as before.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A program that stores to a 2D multisampled texture writes the samples of
its host multisampled color target through a storage view. Targets are
created without storage usage; the first storage binding that resolves
to a target moves it to an image with storage usage, copying its samples
once, and keeps the previous image until the target is released. Storage
usage needs the device's shaderStorageImageMultisample feature and its
format at the target's sample count. A surface no draw rendered to is
created from its uniform color memory, as for sampling; other color
memory still throws.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Restore the shaderDrawParameters feature guard while adding shaderTerminateInvocation support. Cover enabled and disabled DrawParameters profiles, and pass the termination feature to the correct shader validation argument.
The device queries and enables shaderTerminateInvocation when available, advertises SPV_KHR_terminate_invocation to the recompiler and validation, and the shader device profile rejects a target that advertises the extension without the enabled feature.
Based on the approach in poonasor/AnyPS5@73969879.
Refs #2009
(cherry picked from commit 1866a3f70e7ac1c34c8cd87fc79a973bd9ba6947)
With pipelined draws, a DISPATCH_DIRECT on queue 0 whose dispatch-cache entry hits with a
recipe runs as an ordered commit item instead of draining the pipeline first. Misses, reads
that overlap writes still pending in the pipeline, verified data hits and writes that cannot
be ranged still drain. A dispatch that is resolved ahead (or adopted from a group resolved
ahead) and copy kernels keep the drained path; with the switch on, queue 0 does not resolve
dispatch groups ahead, since that drains the pipeline at every group. Off by default.
agc_driver_pipelined_dispatch counts drains per opcode (each dispatch drains with the switch
off); agc_driver_pipelined_dispatch_item checks that repeated dispatches do not drain, and that
a dispatch whose descriptor table a pending fill retargets drains and reads the new buffer.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
A GPU hang or a crash in a title leaves no record of what the queues
were last given: APS5_TRACE_GPU prints every submission (too much to
leave on), and the WAIT_REG_MEM timeout message shows only the packet
before the wait on the waiting queue.
Each queue now keeps a ring of its last 16 submissions
(APS5_PM4_RING=<n>, 0 to 4096, 0 disables): the TraceMs time in tenths
of a millisecond, serial, size, guest address and a copy of the first
512 dwords after indirect buffers are inlined, recorded under the
driver lock next to the serial, REWIND tails included. The ring of a
queue and the failure report registration are set up at the top of
Submit, before any output is reserved, so recording under the lock
does not allocate. Disabled, Submit only tests a static flag; enabled,
it costs one bounded copy per submission.
DumpSubmissionRings decodes the kept dwords with Pm4::PacketWords and
the new Pm4::KnownName, which returns the decoder's names as views of
static strings (empty for an unknown opcode, printed as UNKNOWN_0x%02x);
Pm4::Name now builds its std::string from it. Every line is formatted
with integer and string conversions only into a 512-byte stack buffer
and written with one WriteFile / write(2) on the stderr handle, so the
dump does not allocate from the C runtime heap, lines from other
threads do not interleave and the CRT stream lock is not taken (the
rule of #3871). The header gives the number of queues and the ring
size, and each queue line the submissions recorded and kept.
libc gets a small registry of failure reports
(LibcRegisterFailureReport_nid_no_patch, Unregister, Run); Run runs
the registered reports the first time it is called and returns at once
after that. The Windows crash report runs it after the thread dump of
an unhandled exception, after SIGABRT (which also covers the
APS5_HANG_DUMP_SECS timed dump, since it aborts) and after an invalid
CRT parameter, so an abort after a reported exception does not print
the rings twice. The driver registers its dump at the first submission
and unregisters at static destruction; the dump there waits a bounded
time for each ring lock. What the crash-path dump cannot do (a ring
held by the crashed thread, rings corrupted by the crash, Linux) is
recorded in technical debt.
APS5_STALL_WATCHDOG_SECS=<n> (0 to 86400, 0 disables) makes a waiter
that has not progressed for n seconds dump the rings once per wait: a
WAIT_REG_MEM (progress as for APS5_GPU_WAIT_TIMEOUT_MS: another packet
runs or completes), a REWIND waiting for the title, and a compute
submission held behind queue 0. The last one prints under the driver
lock, so submissions and the other queues wait for it (USAGE.md says
so). Both variables are read at the top of Submit, so an invalid value
of either throws at the first submission and never later inside a GPU
worker. The order wait behind queue 0 releases its order hold on scope
exit, so a throwing dump cannot leave it counted.
No packet semantics change; packet sizes and names are those of the
existing decoder (PM4 type-3 header layout as in Mesa
src/amd/common/sid.h). The queue bound of Submit and of the rings is
one constant, SubmissionQueueEnd.
Tests: agc_driver_pm4_tests gets ten modes, each its own ctest:
agc_driver_pm4_ring (APS5_PM4_RING=4, six submissions on queue 0 and
two on 0x20: the dump holds #3..#6 and #1..#2 in order, nothing
older), agc_driver_pm4_ring_empty (one line), _ring_disabled (one
line), _ring_invalid and agc_driver_pm4_stall_watchdog_invalid
(APS5_STALL_WATCHDOG_SECS=1x: the submission throws and is not
executed), _ring_report (LibcRunFailureReports_nid_no_patch dumps the
rings), _ring_report_heap (Windows: with ucrtbase.dll's HeapAlloc and
HeapReAlloc imports counting calls from the reporting thread, the
failure report of a submission holding INDEX_BUFFER_SIZE names it and
allocates nothing, while Pm4::Name of the same header is seen
allocating), and with APS5_STALL_WATCHDOG_SECS=1 a WAIT_REG_MEM
(agc_driver_pm4_stall_watchdog), a REWIND (_stall_watchdog_rewind) and
a compute submission held behind a queue 0 rendering wait
(_stall_watchdog_order), each held for about 2.5 s, dump once and
complete when released. The PM4 catalog test checks KnownName against
every opcode and custom name. windows_backtrace_failure_report checks
that abort() runs the registered reports once and not an unregistered
one; windows_backtrace_failure_report_once that abort() does not run
them again after an earlier run. With the dump decoding through
Pm4::Name, agc_driver_pm4_ring_report_heap fails with "the failure
report allocated from the C runtime heap"; with Run resetting its
flag, windows_backtrace_failure_report_once prints the report twice.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A draw whose rasterizer has more samples than its color or depth targets
threw "a color target's sample count differs from the rasterizer's" or
"the depth target's sample count differs from the rasterizer's". When
every target has one sample, MSAA_ENABLE is set, and there is no
alpha-to-coverage, no PS_ITER_SAMPLE and no sample-dependent pixel input,
only sample 0's coverage reaches the targets, so draw it with one sample
at sample 0's location in each pixel of the grid. One-sample depth images
are created sample-location compatible when the device allows it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Reapplies #5791 (reverted by #6552). agc_driver_mesh_index_cache now skips
on llvmpipe like agc_driver_mesh does: llvmpipe aborts compiling these mesh
programs (LLVM ERROR: Cannot emit physreg copy), which is what turned main's
linux job red.
Co-Authored-By: Claude <[email protected]>
agc_driver_mesh_index_cache aborts on the linux job (lavapipe, LLVM 20.1.2:
"LLVM ERROR: Cannot emit physreg copy instruction") on every main push build
since the merge (first red: 38777177302). The PR was merged on a
windows-only run after a linux-red batch was split.
Co-Authored-By: Claude <[email protected]>
A geometry program with VGT_GS_OUT_PRIM_TYPE 1 was refused as an
unsupported output assembly. Each primitive export of an NGG line-strip
program is one line whose indices are packed like a triangle's, so the
mesh module now declares OutputLinesEXT with PrimitiveLineIndicesEXT for
it. A subgroup gets VGT_GS_MAX_VERT_OUT - 1 lines per input primitive and
at least GE_MAX_OUTPUT_PER_SUBGROUP line slots; fast launch keeps using
PRIM_AMP_FACTOR. The output size is part of the cache, draw, validation
and pipeline keys, and the request serialization version becomes 31.
Triangle-strip geometry is unchanged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
With APS5_VERIFY_DRAW_ENTRIES=1 every hit is captured again, so drawHit is false and no relocated hit counted for its shader set's trial, which judged every set unproductive and turned the relocation off under the very mode meant to check it.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Lines queued behind the logger were lost when the process ended through
abort(), std::terminate, the Windows crash report or a guest _Exit,
because only driver failure, Driver::stop and a normal exit flushed the
queue.
libc keeps one end flush (LibcSetEndFlush_nid_no_patch) and runs it from
those paths. Its first registration chains a SIGABRT handler in front of
the previous one, so every abort() and std::terminate in the process is
covered, on Linux as well; with the default disposition the handler
restores it and raises again, so raise(SIGABRT) still ends the process.
Guest abort() and _Exit, and ReportCrash for the first fatal exception,
run the flush directly. AsyncLog registers a bounded flush (FlushFor, 2 s)
that polls two counters and takes no lock, so a dying process cannot hang
on a blocked stream or on the queue lock; after its first timeout it
returns at once. Flush counts queued and finished entries instead of
watching the queue and the writer flag, so it waits for the lines queued
before the call and not for a moment in which no thread is logging.
agc_async_log_fatal runs the test executable as a child that queues lines
behind a held stream and ends in abort(), std::terminate(), raise(SIGABRT)
(POSIX) or an access violation (Windows), and once more with the stream
never released. The lines of the first four must arrive in order, the last
child must end after the bound, and every child has a watchdog.
agc_async_log gains the bounded flush case.
Flush must not return while the logger still waits for a stream that
another thread holds (the regression reported on the pull request: it gave
up after two seconds), and a full queue must block its caller instead of
dropping a line. AsyncLog::Capacity becomes public so the test sizes its
load from it.
Texture creation logged one stdout line per texture and per mip through
WriteLogLine, which formatted with the mingw printf and flushed every
line on the queue worker (about 786k lines in a 25 minute cave run).
AsyncLog keeps a bounded queue and a logger thread. AGC_LOG_DEFERRED
captures numeric arguments on the caller and formats them on the logger
thread; WriteLogLine hands finished lines to a per-module sink in
libc/LogMacros.hpp that AgcDriver installs, so all AgcDriver stdout
lines keep their order and are written one flush per batch. A full queue
blocks the caller, so no line is dropped. Flush runs on driver failure,
shutdown and at exit. APS5_LOG_SYNC=1 restores the synchronous path.
Identical "deferred write-back failed" notes are printed three times per
10 s and then counted into one "repeated N more times" line. A
"[log] 10 s" line reports lines, bytes, caller time and logger time
(APS5_LOG_STATS=0 hides it).
The lazy mode of agc_driver_prepared_shaders only checked dispatch output and failure reporting, which eager preparation also produces. Read the registered snapshot through Driver::RegisteredShaders: registration leaves it deferred with no prepared artifact, and the first dispatch prepares it.
APS5_LAZY_SHADER_PREPARE=1 keeps registration validation and decoding but skips queueing the preparation. Draws and dispatches prepare a deferred snapshot on first use and cache it; explicit ABI resolution still prepares its stages. Titles that register thousands of shaders at boot no longer hold every prepared shader in memory before the first frame.
Cover lazy dispatch, metadata and failure cases in both asynchronous and synchronous modes.
A draw whose user-data words differ only in pointers the capture read through
looks up an alias key built from the contents behind them, shifts the cached
variant's runs by the pointer delta, validates them by value and patches the
forwarded shader-data words.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The headless device is replaced by the windowed one at window attach and then only
lingers for in-flight workers, but it parsed its own full copy of the cache file.
It now starts empty (it still merges what it compiles into the file). Load reuses
the file buffer as the initial data instead of copying it, and save() writes the
live data directly when the file still carries the header this object loaded or
wrote, skipping the read-back and the merge cache.
The kernel maps a dma-buf import's pages for the GPU at the first submission that uses it. On RADV
that took about 2 s for a 3 GiB import, in the second frame of Hades (PPSA03355), and its staging
pool ran out while the frame was held.
Linux now makes the settle queue from #6060 when the device imports dma-buf memory, and settles
every dma-buf import of 1 MiB or more right after it is made with settleImport's 4-byte copy. The
settle queue needs neither the GPU mutex nor the main queue's work to finish. A dma-buf import's
buffer is shared with the settle queue's family instead of aliased, since a sparse import is bound
to many memory objects. Where the sparse queue already takes that family, the settle queue is a
second queue of it.
The dispatch look-ahead stopped before every DISPATCH_INDIRECT whose
arguments an earlier dispatch of the run may write, or whose earlier
dispatches have unknown writes. That stop protects a CPU read of the
arguments at the resolve, which only the thread-dimensions, misaligned and
APS5_NO_GPU_INDIRECT paths make. A GPU-read indirect dispatch never reads
its arguments on the CPU, so the stop left it and every dispatch behind it
to be captured when the worker reached them, after the GPU mutex waits
between dispatches. That is the late read the group capture exists to
avoid: by then the title had re-armed the gate and reused its table slots,
and the capture read descriptors the title had not written yet. The
look-ahead now stops only before an indirect dispatch whose arguments it
would read on the CPU; the path choice is shared with dispatchIndirect.
AgcDriverDispatchesAdoptedAhead_nid_postfix counts the dispatches adopted
from the look-ahead so a test can tell a gate-time capture from a late one.
Two tests check through it that both dispatches of a gated group are
captured at the gate: one whose writer stores the indirect arguments
through a registered binding, and one whose writer stores them through a
binding misaligned in its host import.
AlxcBTW