Restore the shaderDrawParameters feature guard while adding shaderTerminateInvocation support. Cover enabled and disabled DrawParameters profiles, and pass the termination feature to the correct shader validation argument.
The device queries and enables shaderTerminateInvocation when available, advertises SPV_KHR_terminate_invocation to the recompiler and validation, and the shader device profile rejects a target that advertises the extension without the enabled feature.
Based on the approach in poonasor/AnyPS5@73969879.
Refs #2009
(cherry picked from commit 1866a3f70e7ac1c34c8cd87fc79a973bd9ba6947)
The validator only checked SPV_KHR_terminate_invocation when it saw the
OpExtension, so a module using OpTerminateInvocation without declaring
the extension, or in a non-fragment stage, was accepted. Require the
fragment stage, the enabled shaderTerminateInvocation feature and the
declared extension at the instruction itself.
Refs #2009
Shader validation accepts the extension only in fragment shaders and only when the device enabled shaderTerminateInvocation. The flag is part of the validation memo key.
Refs #2009
Pixel invocations stopped by a BDA fault now end with OpTerminateInvocation when the target offers SPV_KHR_terminate_invocation, and keep OpKill otherwise. OpKill is not enough on AMD driver branch 21045, where a guest atomic after the kill still lands once per faulting pixel.
Based on the approach in poonasor/AnyPS5@73969879.
Refs #2009
With pipelined draws, a DISPATCH_DIRECT on queue 0 whose dispatch-cache entry hits with a
recipe runs as an ordered commit item instead of draining the pipeline first. Misses, reads
that overlap writes still pending in the pipeline, verified data hits and writes that cannot
be ranged still drain. A dispatch that is resolved ahead (or adopted from a group resolved
ahead) and copy kernels keep the drained path; with the switch on, queue 0 does not resolve
dispatch groups ahead, since that drains the pipeline at every group. Off by default.
agc_driver_pipelined_dispatch counts drains per opcode (each dispatch drains with the switch
off); agc_driver_pipelined_dispatch_item checks that repeated dispatches do not drain, and that
a dispatch whose descriptor table a pending fill retargets drains and reads the new buffer.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Flushing at shutdown built the singleton and its writer thread on a run that never recorded anything; on Windows that throws std::system_error while the process is exiting.
Pipelines are now shared by every program that recompiles to the same
module, and a leading s_nop recompiles to the same one, so the second
dispatch reused the first pipeline. The new program also stores to the
second word.
Co-Authored-By: Claude <[email protected]>
The replayed stencil reference, the reversed blend, the quad sample masks
and the tessellation instance patches and draw parameters change the
pipeline a draw creates, so the recipe carries them; the file format is
bumped and the layout guards follow the new sizes.
Co-Authored-By: Claude <[email protected]>
MinGW's emulated TLS runs thread_local destructors from the module's TLS
callback, which corrupts the heap when a replay thread or queue worker that
ran CachedPipeline exits (0xC0000374, prx_thread_local_destructors).
Co-Authored-By: Claude <[email protected]>
#5861 added both to the pipeline state and its store key; the recipe dropped them, so a
replayed pipeline keyed another configuration.
Co-Authored-By: Claude <[email protected]>
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
Resolves#4943's draw relocation against this branch: tagVariantRuns, which now builds a draw variant's runs and words, also records the captured words it shares with a table's supplied blocks, per region as before. A variant whose runs relocation splits keeps sharing for the draw cache's word compare, and skips the vouched validation, which already requires one run per region.
The self-hosted runners share one 32-thread host and were fixed at one job each, so
a build with many ccache misses compiled them one after another (686 misses took 37 min).
Pick jobs from the free CPUs and available memory (1 to 8, ctest at most 4) and pass a
load limit to ninja so a busy host backs off.
Co-Authored-By: Claude <[email protected]>
The line said FormatVersion 36 while the code is at 37, as main's says 33 against 36. Without the number it no longer goes stale on each bump or conflicts between PRs that bump it.
The sched_get_priority_max/min entry was rewritten by each fix that
matched a PS5 measurement, until it described only behaviour that now
matches the console. What remains is the gap no entry named: the
priorities are recorded and reported but never reach the host
scheduler.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
An in-place draw input flushed pending GPU writes over its range on the
CPU before the bind, a submit-and-wait of the open batch when a dispatch
in it wrote the buffer. The CPU never reads a vertex buffer's bytes, so a
vertex input under a pending write is bound in place behind a barrier from
shader and transfer writes to vertex attribute reads, recorded before the
draw's rendering begins; index inputs keep the flush, since the highest
index is read on the CPU. Minecraft generates a vertex buffer with a
compute dispatch every frame and drew from it one CPU wait later.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
guest_time_conversion now converts nine times, from the year 1 to 1950,
with localtime and back with mktime in America/Los_Angeles,
Europe/Berlin, Asia/Kolkata and America/Argentina/Buenos_Aires, once
with the daylight flag localtime gave and once with -1. The zones'
local mean time and early standard offsets are covered, including the
first day of 1900. It runs on Linux and macOS; the Windows CRT does not
read region zone names.
A local time before 1900 was converted 400-year cycles later, where the
zone's modern rules apply. macOS's localtime_r gives such a time the
zone's historical offset, local mean time before its first transition,
so mktime(localtime(t)) no longer returned t in a region zone: +422 s in
America/Los_Angeles (LMT -7:52:58), -392 in Europe/Berlin, +1408 in
Asia/Kolkata (-530 under its MMT offset, from 1870 through 1 January
1900), -3228 in America/Argentina/Buenos_Aires (-4608 under its CMT
offset). jotitadev found this in review.
On macOS mktime now solves a local time before 2 January 1900, where the
host's mktime no longer applies, directly: it subtracts the offset
localtime_r gives for a candidate time from the local fields and repeats
until the offset holds, at most four times. Windows and local times from
2 January 1900 on are unchanged.
A GPU hang or a crash in a title leaves no record of what the queues
were last given: APS5_TRACE_GPU prints every submission (too much to
leave on), and the WAIT_REG_MEM timeout message shows only the packet
before the wait on the waiting queue.
Each queue now keeps a ring of its last 16 submissions
(APS5_PM4_RING=<n>, 0 to 4096, 0 disables): the TraceMs time in tenths
of a millisecond, serial, size, guest address and a copy of the first
512 dwords after indirect buffers are inlined, recorded under the
driver lock next to the serial, REWIND tails included. The ring of a
queue and the failure report registration are set up at the top of
Submit, before any output is reserved, so recording under the lock
does not allocate. Disabled, Submit only tests a static flag; enabled,
it costs one bounded copy per submission.
DumpSubmissionRings decodes the kept dwords with Pm4::PacketWords and
the new Pm4::KnownName, which returns the decoder's names as views of
static strings (empty for an unknown opcode, printed as UNKNOWN_0x%02x);
Pm4::Name now builds its std::string from it. Every line is formatted
with integer and string conversions only into a 512-byte stack buffer
and written with one WriteFile / write(2) on the stderr handle, so the
dump does not allocate from the C runtime heap, lines from other
threads do not interleave and the CRT stream lock is not taken (the
rule of #3871). The header gives the number of queues and the ring
size, and each queue line the submissions recorded and kept.
libc gets a small registry of failure reports
(LibcRegisterFailureReport_nid_no_patch, Unregister, Run); Run runs
the registered reports the first time it is called and returns at once
after that. The Windows crash report runs it after the thread dump of
an unhandled exception, after SIGABRT (which also covers the
APS5_HANG_DUMP_SECS timed dump, since it aborts) and after an invalid
CRT parameter, so an abort after a reported exception does not print
the rings twice. The driver registers its dump at the first submission
and unregisters at static destruction; the dump there waits a bounded
time for each ring lock. What the crash-path dump cannot do (a ring
held by the crashed thread, rings corrupted by the crash, Linux) is
recorded in technical debt.
APS5_STALL_WATCHDOG_SECS=<n> (0 to 86400, 0 disables) makes a waiter
that has not progressed for n seconds dump the rings once per wait: a
WAIT_REG_MEM (progress as for APS5_GPU_WAIT_TIMEOUT_MS: another packet
runs or completes), a REWIND waiting for the title, and a compute
submission held behind queue 0. The last one prints under the driver
lock, so submissions and the other queues wait for it (USAGE.md says
so). Both variables are read at the top of Submit, so an invalid value
of either throws at the first submission and never later inside a GPU
worker. The order wait behind queue 0 releases its order hold on scope
exit, so a throwing dump cannot leave it counted.
No packet semantics change; packet sizes and names are those of the
existing decoder (PM4 type-3 header layout as in Mesa
src/amd/common/sid.h). The queue bound of Submit and of the rings is
one constant, SubmissionQueueEnd.
Tests: agc_driver_pm4_tests gets ten modes, each its own ctest:
agc_driver_pm4_ring (APS5_PM4_RING=4, six submissions on queue 0 and
two on 0x20: the dump holds #3..#6 and #1..#2 in order, nothing
older), agc_driver_pm4_ring_empty (one line), _ring_disabled (one
line), _ring_invalid and agc_driver_pm4_stall_watchdog_invalid
(APS5_STALL_WATCHDOG_SECS=1x: the submission throws and is not
executed), _ring_report (LibcRunFailureReports_nid_no_patch dumps the
rings), _ring_report_heap (Windows: with ucrtbase.dll's HeapAlloc and
HeapReAlloc imports counting calls from the reporting thread, the
failure report of a submission holding INDEX_BUFFER_SIZE names it and
allocates nothing, while Pm4::Name of the same header is seen
allocating), and with APS5_STALL_WATCHDOG_SECS=1 a WAIT_REG_MEM
(agc_driver_pm4_stall_watchdog), a REWIND (_stall_watchdog_rewind) and
a compute submission held behind a queue 0 rendering wait
(_stall_watchdog_order), each held for about 2.5 s, dump once and
complete when released. The PM4 catalog test checks KnownName against
every opcode and custom name. windows_backtrace_failure_report checks
that abort() runs the registered reports once and not an unregistered
one; windows_backtrace_failure_report_once that abort() does not run
them again after an earlier run. With the dump decoding through
Pm4::Name, agc_driver_pm4_ring_report_heap fails with "the failure
report allocated from the C runtime heap"; with Run resetting its
flag, windows_backtrace_failure_report_once prints the report twice.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The NeighbourCopied wait accepted any pending result once the surface
image was live, so a pending result of the source surface could end it.
It now waits for a pending result overlapping the surface's range.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Yunus Shaikh