Without VirtualQuery, a range no cached page table covers is looked up
line by line in /proc/self/maps. A cursor in a gap between mappings never
matched a line, so the scan parsed the whole file and then reported the
rest of the range unmapped, including mapped pages after the gap. The
scan now stops at the first mapping above the cursor, reports the gap up
to it as inaccessible and continues from there, as the VirtualQuery path
does with a free region.
PPSA21564 queries near-null addresses (0x18, 0x30, 0x58) about 1500 times
a second during resource capture; each of those parsed the whole maps
file (about 5700 lines here) instead of stopping at the first mapping.
Tests: agc_driver_graphics_tests describes three host pages with the
middle one unmapped as two ranges, a query that starts in the gap as the
page after it, and a near-null address as inaccessible.
PPSA21564 draws small indexed triangle fans (VGT_PRIMITIVE_TYPE 5, six
indices, restart enabled) into a real ES + GS (VGT_SHADER_STAGES_EN
0x2030). DecodeShaderStages took only point, line, triangle list and strip
input, so these were skipped as "unsupported geometry input or output
assembly".
- A fan is accepted as geometry input with an input step of 1, like a
strip: a subgroup of P triangles takes P + 2 vertices, in the decode,
ShaderMeshInputInfo and the draw's group count.
- The mesh entry shades the draw's vertex 0 (the hub) in ES thread 0 of
every subgroup and the subgroup's vertices from firstPrimitive + 1 in
the others; GS thread p gets ES threads (p + 1, p + 2, 0). That is the
order the hardware hands a GS with the first vertex provoking (as
Vulkan orders fans), so a triangle keeps the winding and flat color of
its rim vertex.
- The mesh path does not restart primitives: an indexed fan with restart
enabled is refused when its indices hold the restart index (all ones).
Tests: agc_driver_graphics decodes PPSA21564's fan registers (30
triangles in a 32-vertex subgroup) and rejects short subgroups;
agc_shader_memory checks the fan subgroup sizes; agc_driver_mesh_tests
(GPU, run by hand) draws non-indexed and indexed fans over three
subgroups, checks each triangle takes its first rim vertex's flat color
and the fan is not closed, and refuses a restart index.
A DMA_DATA too large to resolve as a store drained the device and then
copied on the submitting worker's thread. For a large range that copy takes
milliseconds, and meanwhile another queue's worker records, submits and runs
work that reads the destination in place. PPSA21564 copies each 12 MiB
frame of its intro video into one of two textures with a single DMA_DATA on
one queue and samples it on queue 0 without waiting for that queue; the
GPU's own DMA finishes in microseconds, so the title never needed to. Here
queue 0's upload of the texture ran while the CPU copy was still writing
it, and a frame showed the new frame's luma over the chroma of the frame
two before.
Such a copy (both addresses memory and incrementing, ranges disjoint and
non-empty) now goes through VulkanDevice::CopyBuffer like the copy kernel:
a transfer between the host imports, recorded in order, so everything
recorded after the packet reads the whole copy. The drain and CPU copy
remain the fallback when either range is not host-imported.
A wave64 compute or mesh program on a 32-wide host runs two lanes per
invocation, and the emitter emitted every IR value once per lane, scalar ones
included: each SGPR value, scalar memory load and loop phi existed twice in
the module, raising register pressure for every such program.
WaveUniformValues (SpirvAnalysis) finds the values every lane of the wave
computes alike: user data, constants, resources, ballots, readlane and
readfirstlane results, scalar loads through a wave-uniform address, and what
follows from them alone. Lane ids, builtins other than the workgroup id, lane
exchanges and every other memory or image read differ by lane, as does
anything computed from them. For such a value without side effects, the
second lane reuses the first lane's definition. Both lanes are one
invocation, so they share control flow and the definition dominates every
use of the second lane.
agc_shader_memory checks the analysis and that a two-lane module computes a
scalar product as often as the one-lane module. The agc_driver_wave_uniform
execution test runs a wave64 program with a scalar loop, a ballot and a
readfirstlane and checks every lane's result. It skips on a subgroup
narrower than 32, such as lavapipe's.
With APS5_FLIP_INFLIGHT (default 1) a presentation trailed on the GPU and
the flip completed as soon as its blit was queued behind the frame's
batches. The title reads that completion as "the GPU is done with the
frame" and rewrites the frame's memory, but the recorded GPU work still reads
guest memory in place when it runs. In PPSA21564 the per-draw constant
ring of the next frames landed under vertex programs that had not run
yet: a program read an integer sample count as float 1.0 from another
draw and stepped a loop by its reciprocal, about 1e9 iterations, and
other draws got wrong matrices.
The presenter now queues the present and then waits for the frame's own
fence, outside the GPU mutex, before reporting the flip, so the title
cannot reuse the frame's memory while the GPU still reads it.
Test: video_out_flip_tests aftergpu (needs a GPU and a display) submits a
busy compute dispatch, an end-of-pipe label without an interrupt (which
does not drain the device) and a flip, and checks that the flip
completes no earlier than the label lands.
The [present] line, printed under APS5_PROFILE_DRAW, takes its GPU busy and
idle gaps from the whole-batch timestamps in each batch's completion record.
Those stamps were only written under APS5_PROFILE_GPU, which also times every
command. Under APS5_PROFILE_DRAW alone every batch finished with zero stamps,
so the line read "GPU busy 0.0 ms, idle gaps 0.0 ms" with no missing records.
Under APS5_PROFILE_DRAW each batch now carries just its two batch stamps in a
two-query pool, reused across batches like the command buffers.
APS5_PROFILE_GPU is unchanged, and nothing is written with profiling off.
agc_driver_recorder_tests (GPU, by hand) checks that three consecutive
batches' completion records carry ordered stamps exactly when batch stamps
are on, and none with profiling off.
The APM leaves bits 127:64 of the destination undefined after EXTRQ and
INSERTQ, but hardware zeroes them, and the register-form INSERTQ stub
already does the same (#185). The other paths kept the old high qword, so
a following register-form INSERTQ that reads its length and index from
that qword computed the wrong low qword too.
Every stub path that left the high qword now ends with movq dst, dst.
The in-place PSRLQ, PMOVZX and PBLENDW forms leave it non-zero and have
no room for the movq, so those sites now take a stub. The two in-place
forms that emitted nothing now emit movq dst, dst.
Co-Authored-By: Claude <[email protected]>
DecodeInstruction returned before the ModRM byte for VEX and EVEX and
read the third opcode byte of 0F 38/0F 3A as ModRM, so their
RIP-relative operands were never reported. --to-intel then moved such an
instruction after a short EXTRQ/INSERTQ into the stub with its
displacement unchanged, and the strict unused-import filter missed its
references.
Co-Authored-By: Claude <[email protected]>
SHA1RNDS4, SHA1NEXTE, SHA1MSG1 and SHA1MSG2 passed through --to-intel
unchanged, so a relinked title that executes them faults on Intel CPUs
without SHA-NI. Their register forms are now lowered out of line with
SSE2 only through the shared stub builder; short sites share stubs via
MatchSequence. Memory forms fail the relink with the file offset, as for
SHA-256.
Co-Authored-By: Claude <[email protected]>
The Linux entry stub built a copy of the entry parameter block holding
argc and a pointer to the host argv array, so the guest read that pointer
as argv[0] and uninitialised stack as argv[1..]. The process stack at
entry already has the guest's layout (argc as a qword, then the argv
pointers and a null), so pass rsp itself.
Test: linux_entry_argv relinks a fixture whose entry checks argc and
argv and runs the output on Linux x86-64.
The draw resource key holds each read-only buffer's V# base and size and every
ShaderData/FlattenedSrt word, so draws whose buffers move per frame (constant and
vertex rings) never hit a template. The key now leaves those out; a hit checks every
moved read-only buffer (accessible, no pending or shadowed writes over it) and
rebinds it from its new address with the template's push adjustment, and data words
that changed get a fresh data buffer with the template's patches. A template whose
buffers cannot be rebound counts as a miss and is built fresh.
APS5_NO_MOVED_BUFFER_TEMPLATES=1 keys the words as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The source-handle memo poisoned only compute failures, so a draw stage whose resource
walk cannot resolve was recompiled, and threw, on every draw. Draw stages are now
poisoned too, keyed with the linked programs, and a memoized failure rejects the draw
without an exception.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Batch jobs were stored as a 72-byte header plus 16 bytes per buffer, so one decode
job took 104 bytes. GTA V initializes its MP3 batches with 0x40 bytes, every
sceAjmBatchJobDecode returned SCE_AJM_ERROR_OUT_OF_RESOURCES and its streamed
music and speech stayed silent. A job is now a 32-byte record, its buffers, and
its parameters only when it has any; the MP3 test decodes through a 64-byte batch.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
processFlip set lastFlipVblank to the vblank count after the host present
returned. When the acquire, the wait for an earlier presentation or the
present itself crossed the next vblank, the next flip's target moved out
by one vblank even when that flip was already queued: a frame ready in
17 ms was shown for 33 ms, and a title that advances its logic by a fixed
step per presented frame ran in slow motion.
A flip takes effect at the vblank it is released at, so that vblank is
now the one the next flip's interval counts from, recorded when the flip
is released. Flip events, flipStatus and flipPendingNum are unchanged.
Test: video_out_flip_tests pacing (needs a GPU and a display) submits
the first flip at flip rate 2, whose presentation opens the device and
spans several vblanks, and checks that lastFlipVblank is 3.
EmitSetAttribute read state.inputInfo.vertex->clipSpace for every
position export. A pixel shader has no vertex input info, so a pixel
program that exports a position dereferenced a null pointer and crashed
the process; three words (0xf80008cf, 0, s_endpgm) are enough.
EmitSetAttribute now throws "vertex input info is missing for a position
export", as EmitAuxPositionExport already does for auxiliary positions.
Found by fuzzing the recompiler with mutated pixel shaders.
agc_pixel_position_export recompiles that program and expects the error;
it segfaults without the change.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01R5WJS4ZdQynAJesCZKH5hb
V_WRITELANE_B32 writes the selected lane regardless of EXEC, but the
translation routed it through the exec-masked VGPR write, so a spill to an
inactive lane was dropped. The WriteLane test now writes lane 5 while only
lane 0 is active.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Every draw stage's capture walks the shader's SRT plan with an Evaluator
whose per-value memo was a std::unordered_map: a node allocation per
evaluated value, and out-of-line calls for Opcode, IsIdentity, Resolve
and Argument on every step. The walk took ~17 us of every capture at
PPSA21564's title (about 1.25 s of the queue worker's 10 s).
The memo is now an open-addressing table keyed by the value pointer,
and the accessors are inline. Evaluation order, results and failures
are unchanged. The capture walk drops to ~10 us.
A sampler (or texture) chosen by scalar control flow reaches the image
operation as a Phi of two ReadConst values. The SRT walker only accepts
invariant Phis, so GetHandle threw "GetSamplerResource dword 0 is not a
valid runtime value" and the draw was skipped (G-buffer and HDR draws
in PPSA21564).
Before tracking, a read-only image operation (sample, gather, LOD query,
size query, load) whose descriptor dwords are Phis of one block is
copied once per distinct incoming descriptor. Each copy gets its own
memory entry and descriptor handles whose dwords are the edge's values,
recomputed before the operation (SRT reads, user data, the shader base
and pure uniform arithmetic over them only). The results are selected
by a Bool Phi of the edge that entered the Phi block. The copies run
where the original did, so implicit derivatives are unchanged, and the
host binds every edge's descriptor. Any other shape (a nested Phi, a
dynamic read on an edge, a store or atomic) is left alone and still
throws.
A selection whose one arm only returns (kill or s_endpgm) and whose other
arm branches straight to the merge of an enclosing selection had no
structured merge, so the draw was skipped. PPSA21564's BotEye G-buffer
pixel shaders (eight variants) have this shape.
When no other split applies, the edge from such a header to the shared
join is now split into an empty block, which becomes the header's merge.
A RELEASE_MEM with an interrupt select drained the device: the queue
worker submitted everything recorded and waited for the GPU before it
stored the label and delivered the interrupt. PPSA21564 issues about 300
of them per frame, so the worker and the GPU took turns.
The label is now recorded like any other (on the GPU behind the work, or
as a completion action behind pending write-backs), and the delivery
becomes a completion action of the batch holding the work before the
packet (Recorder::AfterRecordedWork, after the batch's earlier
completions). It is counted with the completion labels, so idle workers
and pollers reap it, and it starts the label flush deadline. With draws
or dispatches recorded since the last submit the batch is submitted at
once. A batch holding only label stores stays open and goes out with the
next work, at the label flush deadline, at a wait, at a flip or at the
end of the submission, like any other pending label. A label the GPU
cannot store, or one that does not decode, still drains.
The interrupt so never arrives before the work ahead of it completed and
its label landed; it can only arrive later, by at most the flush
deadline plus the work recorded into the same batch.
Every vertex attribute copied its own range of guest memory, so a draw
over an interleaved buffer copied it once per attribute (seven or eleven
times for PPSA21564's level meshes). Attributes with the same stride and
input rate whose first elements lie within one record now share one copy
of their union, each bound at its offset into it.
Every SET_*_REG packet stored each register into a std::map, and every
draw read dozens of them back through map lookups (the decoders, the
draw key's range walks, the prechecks). PPSA21564 sends ~350k register
packets per 10 s at the title, and the map's node allocations and tree
walks were a visible share of the graphics queue worker's time.
Registers is now a flat value array with a presence bitmap per bank. It
keeps the map's interface and semantics: absent registers stay absent,
iteration and lower_bound walk the present registers in offset order,
emplace does not overwrite and at() throws for an unset register.
PPSA21564's final composite renders into 4K 10-bit scanout buffers
(COLOR_2_10_10_10 with the alternate swap, A2R10G10B10). No guest texture
format matched that color format, so the target stayed non-resident:
every frame's composite draw ran synchronously (about 11 ms, 143 per 10 s
at the title) and the flip read the frame back through guest memory.
The blue-low 10-bit color target now keeps its bits in the storage image
of its surface's red-low 2-10-10-10 format, as B8G8R8A8 targets map to
R8G8B8A8.
Presenting it: DisplayTexelFormat names a display buffer's texel order as
the guest-memory path reads it (R8G8B8A8 base: red low, else blue low;
the 10-bit variant: A2B10G10R10 or A2R10G10B10). ResidentPresentPath
blits a resident image only when it holds the display's 8-bit texel
format (an exact copy). A 10-bit image (whose blit would round where the
guest path drops the low two bits), the other 8-bit order or an sRGB type
is converted instead: GpuColorTransfer::DetileImage copies the image's
texels raw and runs the guest path's own conversion shader on them, so
both paths present the same pixels.
The image's write-back is unchanged: its results stay pending like any
resident target's and are stored when a reader of the memory flushes
them.
Tests: agc_driver_recorder_tests checks the format decision for both
8-bit and both 10-bit orders, and converts an image in each order,
comparing it with the CPU decode and the GPU detile of the same words in
guest memory.
Dean Galvin