mirror of
https://github.com/boykopovar/AnyPS5.git
synced 2026-10-27 00:28:14 +00:00
feat(agc): run registered merged hull programs (binary type 3)
A registered type-3 hull program holds the local and the hull part of a
tessellation draw in one binary, addressed through SPI_SHADER_PGM_LO_LS
(SH 0x148) with no SPI_SHADER_PGM_LO_HS: the local part runs on the lanes
below s3[7:0] and writes LDS, an s_barrier follows, and the hull part runs
on the lanes below s3[15:8] and may store the registers the local part
filled. The draw decode asked 0x148 for a type-5 local half, and
PPSA11386 (007 First Light) stopped in its level load with "registered
program refers to an incompatible shader binary type".
The draw decode now gives a registered type-3 program both the local and
the hull role and marks the tessellation configuration mergedHull. The
local stage only passes gl_VertexIndex and gl_InstanceIndex in a 16-byte
record. The hull stage runs the whole program, one invocation per output
control point: v2 and v5 come from the record of control point
min(gl_InvocationID, inputs - 1), v3 is the relative vertex id, and the
LDS becomes a per-control-point output array that a control point writes
in its own record and reads in any record of its patch after the barrier.
Registers live across the barrier stay in the invocation.
The hull stage of a merged program runs at one lane per invocation on
every device (HostSubgroupSize 1). Its lanes are gl_InvocationID, not host
subgroup invocations, so the s_cbranch_execz around the local part tests
the invocation's own lane. With the device's subgroup it took a subgroup
ballot, and lavapipe, whose subgroup stages lack the tessellation control
stage, rejected the draw ("subgroup capability 61 is unsupported for
shader stage 2").
The lowering refuses more input than output control points, LDS writes
outside the own record, reads outside the patch's records, barriers that
some invocations do not reach or that sit in a loop, reads or writes of
another lane's register, outputs beyond the device limits, and fetch
subroutines. Type 3 is prepared with its stage group like type 5 and
takes its float mode from RSRC1_HS. The hull user-data address (SH
0x102/0x103) is optional for a merged program: s[0:1] take it, unchecked,
when the draw sets it and are zero otherwise.
Both stages refer to one binary, so a draw lists each code and header
region once (AppendProgramMemory); a request's memory regions must not
overlap. An indirect draw's record words go into every program whose user
data holds the register, so the hull stage of a merged program reads
them too.
mergedHull is part of the recompile cache key, the draw decode comparison
and the request serialization (version 39).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
ef0eed9b31
commit
8b8219dcef
25 files changed
+605
-69
No files matched your search
@@ -11,6 +11,7 @@
|
||||
#include <cstdint>
|
||||
#include <list>
|
||||
#include <memory>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
#include "prx/libc/include/general/AtomicSharedPtr.hpp"
|
||||
|
||||
@@ -45,6 +46,9 @@ void ReadUserWords(const QueueState& queue, DrawProgram& program, bool staticAbi
|
||||
void InitializeMerged(const QueueState& queue, DrawProgram& program, std::uint32_t pointerBase, bool pointerRequired, bool staticAbi);
|
||||
void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const ShaderRegistry& registry, bool staticAbi, bool includeFragment);
|
||||
|
||||
void AppendProgramMemory(std::vector<ShaderRecompiler::MemoryRegion>& memory, const std::vector<DrawProgram>& programs);
|
||||
std::vector<std::pair<std::size_t, std::size_t>> UserDataWords(const std::vector<DrawProgram>& programs, const std::vector<ShaderRecompiler::ProgramRole>& roles, std::uint32_t location);
|
||||
|
||||
struct PreparedGraphicsStage {
|
||||
std::shared_ptr<const ShaderSnapshot> snapshot;
|
||||
PreparedShaders::Entry entry;
|
||||
|
||||
@@ -216,13 +216,9 @@ DrawVerdict Driver::draw(QueueState& queue, std::span<const std::uint32_t> packe
|
||||
phaseTiming.Phase(DrawRowDecode);
|
||||
|
||||
const auto locate = [&](std::uint32_t location) -> std::optional<std::pair<std::size_t, std::size_t>> {
|
||||
if (location == 0x280u) return std::nullopt;
|
||||
for (std::size_t i = 0; i < programs.size(); ++i) {
|
||||
if (roles[i] == Role::Fragment || roles[i] == Role::GeometryBack || location < programs[i].userDataBase) continue;
|
||||
const auto word = location - programs[i].userDataBase + (8u - programs[i].firstUserSgpr);
|
||||
if (word < programs[i].userData.size()) return std::make_pair(i, static_cast<std::size_t>(word));
|
||||
}
|
||||
return std::nullopt;
|
||||
const auto words = UserDataWords(programs, roles, location);
|
||||
if (words.empty()) return std::nullopt;
|
||||
return words.front();
|
||||
};
|
||||
if (drawParameters.indirect) {
|
||||
auto& indirect = *drawParameters.indirect;
|
||||
@@ -237,13 +233,11 @@ DrawVerdict Driver::draw(QueueState& queue, std::span<const std::uint32_t> packe
|
||||
}
|
||||
std::vector<ShaderRecompiler::MemoryRegion> memory;
|
||||
memory.reserve(2 * programs.size() + 32);
|
||||
AppendProgramMemory(memory, programs);
|
||||
std::vector<ShaderRecompiler::LinkedProgram> linked;
|
||||
linked.reserve(programs.size());
|
||||
for (std::size_t i = 0; i < programs.size(); ++i) {
|
||||
const auto& program = programs[i];
|
||||
for (const auto& region : program.memory) {
|
||||
if (!region.bytes.empty()) memory.push_back(region);
|
||||
}
|
||||
linked.push_back({roles[i], program.binary, program.userDataBase, program.firstUserSgpr, program.userData});
|
||||
}
|
||||
timing.Mark("prepare");
|
||||
@@ -544,9 +538,9 @@ DrawVerdict Driver::draw(QueueState& queue, std::span<const std::uint32_t> packe
|
||||
std::vector<Pm4::DrawArguments> records;
|
||||
for (std::uint32_t record = 0; record < count; ++record) records.push_back(Pm4::ReadDrawArguments(indirect, record));
|
||||
Graphics::CountIndirectDraw(*indirectCpu, std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - readStart).count());
|
||||
const auto baseVertexWord = locate(indirect.baseVertexLocation);
|
||||
const auto startInstanceWord = locate(indirect.startInstanceLocation);
|
||||
const auto drawIndexWord = indirect.drawIndexEnabled ? locate(indirect.drawIndexLocation) : std::nullopt;
|
||||
const auto baseVertexWords = UserDataWords(programs, roles, indirect.baseVertexLocation);
|
||||
const auto startInstanceWords = UserDataWords(programs, roles, indirect.startInstanceLocation);
|
||||
const auto drawIndexWords = indirect.drawIndexEnabled ? UserDataWords(programs, roles, indirect.drawIndexLocation) : std::vector<std::pair<std::size_t, std::size_t>>{};
|
||||
const auto drawable = [&](const Pm4::DrawArguments& arguments) { return arguments.count != 0 && arguments.instances != 0 && (!drawParameters.indexed || arguments.firstVertexOrIndex < drawParameters.indexCount); };
|
||||
std::size_t lastDrawable = records.size();
|
||||
for (std::size_t record = 0; record < records.size(); ++record) {
|
||||
@@ -557,14 +551,15 @@ DrawVerdict Driver::draw(QueueState& queue, std::span<const std::uint32_t> packe
|
||||
if (traceIndirect) std::fprintf(stderr, "[draw] record %u: count %u instances %u first %u vertexOffset %u startInstance %u\n", record, arguments.count, arguments.instances, arguments.firstVertexOrIndex, arguments.vertexOffset, arguments.firstInstance);
|
||||
if (arguments.count == 0 || arguments.instances == 0) continue;
|
||||
std::set<std::size_t> patched;
|
||||
const auto patch = [&](const std::optional<std::pair<std::size_t, std::size_t>>& word, std::uint32_t value) {
|
||||
if (!word) return;
|
||||
writablePrograms[word->first].userData[word->second] = value;
|
||||
patched.insert(word->first);
|
||||
const auto patch = [&](const std::vector<std::pair<std::size_t, std::size_t>>& words, std::uint32_t value) {
|
||||
for (const auto& [program, word] : words) {
|
||||
writablePrograms[program].userData[word] = value;
|
||||
patched.insert(program);
|
||||
}
|
||||
};
|
||||
patch(baseVertexWord, indirect.recordBytes == 20 ? arguments.vertexOffset : arguments.firstVertexOrIndex);
|
||||
patch(startInstanceWord, arguments.firstInstance);
|
||||
patch(drawIndexWord, record);
|
||||
patch(baseVertexWords, indirect.recordBytes == 20 ? arguments.vertexOffset : arguments.firstVertexOrIndex);
|
||||
patch(startInstanceWords, arguments.firstInstance);
|
||||
patch(drawIndexWords, record);
|
||||
if (graphics.stages.tessellation) {
|
||||
for (std::size_t i = 0; i < programs.size(); ++i) {
|
||||
if (programs[i].binary.stage == Stage::TessellationControl || programs[i].binary.stage == Stage::TessellationEvaluation) patched.insert(i);
|
||||
|
||||
@@ -46,11 +46,34 @@ void InitializeMerged(const QueueState& queue, DrawProgram& program, std::uint32
|
||||
const auto address = static_cast<std::uint64_t>(low) | (static_cast<std::uint64_t>(high) << 32u);
|
||||
require(address != 0 || !pointerRequired, "merged shader user-data address is null");
|
||||
if (address == 0) return;
|
||||
GuestMemory::CheckRange(reinterpret_cast<const void*>(address), 8, 4);
|
||||
if (pointerRequired) GuestMemory::CheckRange(reinterpret_cast<const void*>(address), 8, 4);
|
||||
program.userData[0] = low;
|
||||
program.userData[1] = high;
|
||||
}
|
||||
|
||||
void AppendProgramMemory(std::vector<ShaderRecompiler::MemoryRegion>& memory, const std::vector<DrawProgram>& programs) {
|
||||
for (const auto& program : programs) {
|
||||
for (const auto& region : program.memory) {
|
||||
if (region.bytes.empty()) continue;
|
||||
const bool listed = std::any_of(memory.begin(), memory.end(), [&](const ShaderRecompiler::MemoryRegion& known) { return known.guestAddress == region.guestAddress && known.bytes.data() == region.bytes.data() && known.bytes.size() == region.bytes.size(); });
|
||||
if (!listed) memory.push_back(region);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<std::pair<std::size_t, std::size_t>> UserDataWords(const std::vector<DrawProgram>& programs, const std::vector<ShaderRecompiler::ProgramRole>& roles, std::uint32_t location) {
|
||||
using Role = ShaderRecompiler::ProgramRole;
|
||||
require(roles.size() == programs.size(), "draw program roles are incomplete");
|
||||
std::vector<std::pair<std::size_t, std::size_t>> words;
|
||||
if (location == 0x280u) return words;
|
||||
for (std::size_t i = 0; i < programs.size(); ++i) {
|
||||
if (roles[i] == Role::Fragment || roles[i] == Role::GeometryBack || location < programs[i].userDataBase) continue;
|
||||
const auto word = location - programs[i].userDataBase + (8u - programs[i].firstUserSgpr);
|
||||
if (word < programs[i].userData.size()) words.emplace_back(i, static_cast<std::size_t>(word));
|
||||
}
|
||||
return words;
|
||||
}
|
||||
|
||||
void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const ShaderRegistry& registry, bool staticAbi, bool includeFragment) {
|
||||
using Stage = ShaderRecompiler::ShaderStage;
|
||||
using Role = ShaderRecompiler::ProgramRole;
|
||||
@@ -95,11 +118,21 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const
|
||||
programs.push_back(prepare(programAddress(base), type, stage, resources, users));
|
||||
roles.push_back(role);
|
||||
};
|
||||
const auto& graphics = product->state;
|
||||
auto& graphics = product->state;
|
||||
if (graphics.stages.path == Graphics::ShaderPath::Tessellation) {
|
||||
if (!staticAbi || programAddress(0x148) != 0) append(0x148, 5, Stage::Local, 0x10b, 0x10c, Role::Local);
|
||||
append(0x108, 7, Stage::TessellationControl, 0x10b, 0x10c, Role::Hull);
|
||||
InitializeMerged(queue, programs.back(), 0x102, true, staticAbi);
|
||||
const auto localAddress = programAddress(0x148);
|
||||
const auto local = !staticAbi || localAddress != 0 ? ProgramSnapshot(registry, localAddress, {3, 5}) : nullptr;
|
||||
const bool mergedHull = local != nullptr && !local->header.empty() && local->type == 3;
|
||||
require(graphics.stages.tessellation.has_value(), "tessellation draw has no tessellation configuration");
|
||||
graphics.stages.tessellation->mergedHull = mergedHull;
|
||||
if (mergedHull) {
|
||||
append(0x148, 3, Stage::Local, 0x10b, 0x10c, Role::Local);
|
||||
append(0x148, 3, Stage::TessellationControl, 0x10b, 0x10c, Role::Hull);
|
||||
} else {
|
||||
if (local != nullptr) append(0x148, 5, Stage::Local, 0x10b, 0x10c, Role::Local);
|
||||
append(0x108, 7, Stage::TessellationControl, 0x10b, 0x10c, Role::Hull);
|
||||
}
|
||||
InitializeMerged(queue, programs.back(), 0x102, !mergedHull, staticAbi);
|
||||
append(0x0c8, 2, Stage::TessellationEvaluation, 0x08b, 0x08c, Role::Domain);
|
||||
} else if (graphics.stages.path == Graphics::ShaderPath::Geometry) {
|
||||
const auto frontAddress = programAddress(0xc8);
|
||||
|
||||
@@ -28,6 +28,7 @@ void Driver::lookupDraw(const Submission& submission, const std::shared_ptr<Vulk
|
||||
if (graphics.stages.tessellation) {
|
||||
const auto& tess = *graphics.stages.tessellation;
|
||||
for (const auto value : {tess.inputControlPoints, tess.outputControlPoints, tess.domain, tess.partitioning, tess.outputTopology, tess.patchesPerGroup, tess.primitiveGroupSize, static_cast<std::uint32_t>(tess.breakAtInstance)}) mix(value);
|
||||
mix(tess.mergedHull);
|
||||
}
|
||||
mix(graphics.rectList);
|
||||
mix(programs.size());
|
||||
|
||||
@@ -241,7 +241,7 @@ bool Driver::sameDecode(const DrawDecode& a, const DrawDecode& b) {
|
||||
const auto sameTess = [](const std::optional<ShaderRecompiler::TessellationConfiguration>& x, const std::optional<ShaderRecompiler::TessellationConfiguration>& y) {
|
||||
if (x.has_value() != y.has_value()) return false;
|
||||
if (!x) return true;
|
||||
return x->inputControlPoints == y->inputControlPoints && x->outputControlPoints == y->outputControlPoints && x->domain == y->domain && x->partitioning == y->partitioning && x->outputTopology == y->outputTopology && x->patchesPerGroup == y->patchesPerGroup && x->primitiveGroupSize == y->primitiveGroupSize && x->breakAtInstance == y->breakAtInstance;
|
||||
return x->inputControlPoints == y->inputControlPoints && x->outputControlPoints == y->outputControlPoints && x->domain == y->domain && x->partitioning == y->partitioning && x->outputTopology == y->outputTopology && x->patchesPerGroup == y->patchesPerGroup && x->primitiveGroupSize == y->primitiveGroupSize && x->breakAtInstance == y->breakAtInstance && x->mergedHull == y->mergedHull;
|
||||
};
|
||||
if (s.stages.path != t.stages.path || s.stages.registerValue != t.stages.registerValue || s.stages.vertexWaveSize != t.stages.vertexWaveSize || s.stages.fragmentWaveSize != t.stages.fragmentWaveSize || !sameMesh(s.stages.mesh, t.stages.mesh) || !sameTess(s.stages.tessellation, t.stages.tessellation)) return false;
|
||||
if (!sameColor(s.color, t.color) || s.colors.size() != t.colors.size() || s.blends.size() != t.blends.size()) return false;
|
||||
|
||||
@@ -803,7 +803,7 @@ std::optional<ShaderRecompiler::ShaderFloatMode> DecodeFloatMode(const Registers
|
||||
case 0: rsrc1 = 0x212; fp16OverflowBit = 26; break;
|
||||
case 1: rsrc1 = 0x00a; fp16OverflowBit = 29; break;
|
||||
case 2: case 4: case 6: rsrc1 = 0x08a; fp16OverflowBit = 31; break;
|
||||
case 5: case 7: rsrc1 = 0x10a; fp16OverflowBit = 30; break;
|
||||
case 3: case 5: case 7: rsrc1 = 0x10a; fp16OverflowBit = 30; break;
|
||||
default: return std::nullopt;
|
||||
}
|
||||
const auto found = registers.find(rsrc1);
|
||||
@@ -901,7 +901,7 @@ std::unique_ptr<RegisteredPreparation> PlanRegistered(const ShaderSnapshot& snap
|
||||
case 1: stage = Stage::Fragment; programRegister = 0x008; resourceRegister = 0x00b; break;
|
||||
case 2: stage = Stage::Vertex; programRegister = 0x0c8; resourceRegister = 0x08b; firstUser = 8; break;
|
||||
case 4: stage = Stage::Mesh; programRegister = 0x0c8; resourceRegister = 0x08b; break;
|
||||
case 5: return {};
|
||||
case 3: case 5: return {};
|
||||
case 6: stage = Stage::Mesh; programRegister = 0x088; resourceRegister = 0x08b; break;
|
||||
case 7: stage = Stage::TessellationControl; programRegister = 0x108; resourceRegister = 0x10b; break;
|
||||
default:
|
||||
@@ -1350,13 +1350,13 @@ std::vector<PreparedGraphicsStage> PrepareGraphicsStages(const DrawDecode& decod
|
||||
require(expected == decoded.roles, "graphics ABI linked programs do not match the stage routing");
|
||||
std::vector<ShaderRecompiler::LinkedProgram> linked;
|
||||
std::vector<ShaderRecompiler::MemoryRegion> memory;
|
||||
AppendProgramMemory(memory, decoded.programs);
|
||||
for (std::size_t index = 0; index < decoded.programs.size(); ++index) {
|
||||
const auto& program = decoded.programs[index];
|
||||
require(program.snapshot != nullptr && program.codeOffset < program.snapshot->code.size(), "graphics ABI has an invalid registered entry point");
|
||||
const auto code = std::span(program.snapshot->code).subspan(program.codeOffset);
|
||||
require(program.binary.code.data() == code.data() && program.binary.code.size() == code.size() && program.binary.codeAddress == program.snapshot->codeAddress + program.codeOffset * sizeof(std::uint32_t), "graphics ABI entry point differs from its snapshot");
|
||||
linked.push_back({decoded.roles[index], program.binary, program.userDataBase, program.firstUserSgpr, program.userData});
|
||||
memory.insert(memory.end(), program.memory.begin(), program.memory.end());
|
||||
}
|
||||
std::vector<PreparedGraphicsStage> prepared;
|
||||
std::uint32_t pushOffset = 0;
|
||||
|
||||
@@ -4839,7 +4839,7 @@ void verifyPixelRequestSerialization() {
|
||||
minimal.context.pixel->layerExported = true;
|
||||
minimal.context.pixel->ditheredAlphaToCoverage = true;
|
||||
const auto encoded = serializer.Serialize(minimal);
|
||||
require(requestPrefix(encoded, 8u) == "NVNQQSYAAAA=", "new requests did not use serialization version 38");
|
||||
require(requestPrefix(encoded, 8u) == "NVNQQScAAAA=", "new requests did not use serialization version 39");
|
||||
const std::array<std::uint32_t, 2> fallbacks{0x40u, 0x1ac4u};
|
||||
auto withFallbacks = minimal;
|
||||
withFallbacks.context.linkedCallFallbacks = fallbacks;
|
||||
@@ -4957,7 +4957,7 @@ void verifyPixelRequestSerialization() {
|
||||
for (std::size_t bytes = 0; bytes < 45u; ++bytes) {
|
||||
expectFailure([&] { static_cast<void>(serializer.Deserialize(requestPrefix(encoded, mappingOffset + bytes))); }, "truncated data", "a truncated pixel mapping, packing, dual-source, alpha-to-coverage, stencil reference export flag, primitive class, layer export, dither or front-face flag, quad pixel mask or quad sample mask was accepted");
|
||||
}
|
||||
for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQScAAAA="}) {
|
||||
for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQSgAAAA="}) {
|
||||
expectFailure([&] { static_cast<void>(serializer.Deserialize(unsupported)); }, "serialization version", "an unsupported request version was accepted");
|
||||
}
|
||||
static constexpr std::string_view version18Request{
|
||||
@@ -5243,6 +5243,43 @@ void verifyDepthRemapRequests() {
|
||||
require(key != other, "the depth remap mode did not change the shader cache key");
|
||||
}
|
||||
|
||||
void verifyMergedHullConfiguration() {
|
||||
using namespace ShaderRecompiler;
|
||||
static constexpr std::array<std::uint32_t, 1> code{0xbf810000u};
|
||||
const RequestSerializer serializer;
|
||||
RecompileRequest request{};
|
||||
request.shader = {ShaderStage::TessellationControl, 0x10000u, code, 0, {}};
|
||||
request.context.waveSize = 64;
|
||||
request.context.vertex = ShaderVertexStageInfo{};
|
||||
request.graphics = GraphicsCompileContext{0u, {}, std::nullopt, TessellationConfiguration{3u, 3u, 1u, 2u, 2u}, {}};
|
||||
const auto separate = serializer.Serialize(request);
|
||||
std::vector<std::uint64_t> separateKey;
|
||||
RecompileCacheKey::Build(request, separateKey);
|
||||
request.graphics->tessellation->mergedHull = true;
|
||||
const auto merged = serializer.Serialize(request);
|
||||
std::vector<std::uint64_t> mergedKey;
|
||||
RecompileCacheKey::Build(request, mergedKey);
|
||||
const auto replay = serializer.Deserialize(merged);
|
||||
require(replay.request.graphics.has_value() && replay.request.graphics->tessellation.has_value() && replay.request.graphics->tessellation->mergedHull, "a merged hull configuration was lost in serialization");
|
||||
require(!serializer.Deserialize(separate).request.graphics->tessellation->mergedHull, "a separate hull configuration was read as merged");
|
||||
auto legacy = decodeBase64(merged);
|
||||
require(legacy.size() > 52u && legacy[legacy.size() - 52u] == 1u, "the merged hull flag is not the last byte of the tessellation configuration");
|
||||
legacy.erase(legacy.end() - 52);
|
||||
for (const auto version : {30u, 31u, 32u, 34u, 35u, 36u, 37u, 38u}) {
|
||||
auto older = legacy;
|
||||
older[4] = static_cast<std::uint8_t>(version);
|
||||
if (version < 38u) older.erase(older.end() - 31);
|
||||
if (version < 36u) older.pop_back();
|
||||
if (version < 35u) older.resize(older.size() - 5u);
|
||||
if (version < 32u) older.pop_back();
|
||||
const auto upgraded = serializer.Deserialize(encodeBase64(older));
|
||||
const auto what = "a main version " + std::to_string(version) + " tessellation request";
|
||||
require(upgraded.request.graphics.has_value() && upgraded.request.graphics->tessellation.has_value() && !upgraded.request.graphics->tessellation->mergedHull, (what + " read a merged hull flag").c_str());
|
||||
require(serializer.Serialize(upgraded.request) == separate, (what + " misread the fields after its tessellation configuration").c_str());
|
||||
}
|
||||
require(mergedKey != separateKey && RecompileCacheKey::ContextHash(request) != RecompileCacheKey::ContextHash(serializer.Deserialize(separate).request), "the cache keys ignore whether the hull program is merged");
|
||||
}
|
||||
|
||||
void verifyLegacyPixelRequests() {
|
||||
using namespace ShaderRecompiler;
|
||||
static constexpr std::array<std::string_view, 14> legacyPixelRequests{
|
||||
@@ -7412,6 +7449,7 @@ int main(int argc, char** argv) {
|
||||
verifyPixelRequestSerialization();
|
||||
verifyVertexInfoVersion19();
|
||||
verifyDepthRemapRequests();
|
||||
verifyMergedHullConfiguration();
|
||||
verifyPositionExportComponents();
|
||||
verifyLegacyPixelRequests();
|
||||
verifyPixelExportReplay();
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp"
|
||||
#include "Optimization/ResourceProgram.hpp"
|
||||
#include "CompiledVariant.hpp"
|
||||
#include "Optimization/RequestMemoryView.hpp"
|
||||
#include <spirv/unified1/spirv.hpp>
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
@@ -481,6 +482,197 @@ void RetainedPixelMatchesNullProgram(AgcDriver::VulkanDevice& device) {
|
||||
}
|
||||
}
|
||||
|
||||
void CheckMergedHull(AgcDriver::VulkanDevice& device) {
|
||||
using namespace AgcDriver::DriverDetail;
|
||||
using namespace ShaderRecompiler;
|
||||
Fixture merged;
|
||||
Fixture domain;
|
||||
Fixture fragment;
|
||||
merged.Initialize(3u);
|
||||
domain.Initialize(2u);
|
||||
fragment.Initialize(1u);
|
||||
ShaderRegistry registry;
|
||||
for (const auto* fixture : {&merged, &domain, &fragment}) registry.emplace(fixture->snapshot->codeAddress, std::vector<std::shared_ptr<const ShaderSnapshot>>{fixture->snapshot});
|
||||
AgcDriver::QueueState queue{};
|
||||
queue.context[0x8e] = 0xfu;
|
||||
queue.context[0x8f] = 0xfu;
|
||||
merged.Bind(queue, 0x148u, 0x10bu);
|
||||
domain.Bind(queue, 0xc8u, 0x8bu);
|
||||
fragment.Bind(queue, 0x8u, 0xbu);
|
||||
const auto unset = queue;
|
||||
const auto null = [&] {
|
||||
auto state = queue;
|
||||
state.shader[0x102u] = 0u;
|
||||
state.shader[0x103u] = 0u;
|
||||
return state;
|
||||
}();
|
||||
alignas(8) const std::array<std::uint32_t, 2> pointer{};
|
||||
const auto pointerAddress = reinterpret_cast<std::uintptr_t>(pointer.data());
|
||||
queue.shader[0x102u] = static_cast<std::uint32_t>(pointerAddress);
|
||||
queue.shader[0x103u] = static_cast<std::uint32_t>(pointerAddress >> 32u);
|
||||
DrawDecode prepared{};
|
||||
prepared.state.stages.path = AgcDriver::Graphics::ShaderPath::Tessellation;
|
||||
prepared.state.stages.vertexWaveSize = 64u;
|
||||
prepared.state.stages.fragmentWaveSize = 32u;
|
||||
prepared.state.stages.tessellation = TessellationConfiguration{3, 4, 1, 2, 2};
|
||||
prepared.pixel.wave32 = true;
|
||||
prepared.pixel.targetOutputMode[0] = 9;
|
||||
prepared.pixel.targetExportMapping.fill(0xe4u);
|
||||
DecodeGraphicsPrograms(prepared, queue, registry, true, true);
|
||||
DrawDecode draw{};
|
||||
draw.state = prepared.state;
|
||||
draw.state.stages.tessellation->mergedHull = false;
|
||||
draw.pixel = prepared.pixel;
|
||||
DecodeGraphicsPrograms(draw, queue, registry, false, true);
|
||||
for (const auto* decoded : {&prepared, &draw}) {
|
||||
Require(decoded->state.stages.tessellation->mergedHull, "a registered type-3 hull program was not decoded as a merged hull");
|
||||
Require(decoded->roles == std::vector<ProgramRole>{ProgramRole::Local, ProgramRole::Hull, ProgramRole::Domain, ProgramRole::Fragment}, "a merged hull program did not give the local and hull roles");
|
||||
const auto& local = decoded->programs[0];
|
||||
const auto& hull = decoded->programs[1];
|
||||
Require(local.snapshot == merged.snapshot && hull.snapshot == merged.snapshot && local.codeOffset == 64u && hull.codeOffset == 64u, "the local and hull stages of a merged hull program refer to another program");
|
||||
Require(local.binary.stage == ShaderStage::Local && hull.binary.stage == ShaderStage::TessellationControl && local.firstUserSgpr == 8u && hull.firstUserSgpr == 0u && hull.userData.size() == local.userData.size() + 8u, "a merged hull program's stages lost the merged user SGPR layout");
|
||||
}
|
||||
Require(draw.programs[1].userData[0] == static_cast<std::uint32_t>(pointerAddress) && draw.programs[1].userData[1] == static_cast<std::uint32_t>(pointerAddress >> 32u), "a merged hull draw did not pass its user-data address");
|
||||
Require(UserDataWords(draw.programs, draw.roles, 0x10eu) == std::vector<std::pair<std::size_t, std::size_t>>{{0u, 2u}, {1u, 10u}}, "an indirect draw's user-data register does not reach both stages of a merged hull program");
|
||||
std::vector<MemoryRegion> programMemory;
|
||||
AppendProgramMemory(programMemory, draw.programs);
|
||||
Require(programMemory.size() == 6u, "a merged hull draw's program memory does not list each code and header region once");
|
||||
try {
|
||||
const RequestMemoryView view(programMemory);
|
||||
} catch (const std::exception& error) {
|
||||
throw std::runtime_error(std::string("a merged hull draw's program memory is not a valid request memory: ") + error.what());
|
||||
}
|
||||
for (const auto* state : {&unset, &null}) {
|
||||
DrawDecode without{};
|
||||
without.state = prepared.state;
|
||||
without.pixel = prepared.pixel;
|
||||
DecodeGraphicsPrograms(without, *state, registry, false, true);
|
||||
Require(without.roles == draw.roles && without.programs[1].userData.size() == draw.programs[1].userData.size() && without.programs[1].userData[0] == 0u && without.programs[1].userData[1] == 0u, "a merged hull draw without a user-data address was not decoded with zero s[0:1]");
|
||||
}
|
||||
auto unmapped = queue;
|
||||
unmapped.shader[0x102u] = 0x10u;
|
||||
unmapped.shader[0x103u] = 0u;
|
||||
DrawDecode stale{};
|
||||
stale.state = prepared.state;
|
||||
stale.pixel = prepared.pixel;
|
||||
DecodeGraphicsPrograms(stale, unmapped, registry, false, true);
|
||||
Require(stale.programs[1].userData[0] == 0x10u && stale.programs[1].userData[1] == 0u, "a merged hull draw did not pass a user-data address it does not need unchecked");
|
||||
const auto stages = PrepareGraphicsStages(prepared, device.Target());
|
||||
Require(stages.size() == 4u && stages[0].entry.handle != stages[1].entry.handle, "a merged hull program was not prepared as separate local and hull stages");
|
||||
for (const auto& stage : stages) stage.snapshot->prepared->entries.push_back(stage.entry);
|
||||
std::vector<LinkedProgram> linked;
|
||||
std::vector<MemoryRegion> memory;
|
||||
AppendProgramMemory(memory, draw.programs);
|
||||
for (std::size_t index = 0; index < draw.programs.size(); ++index) {
|
||||
const auto& program = draw.programs[index];
|
||||
linked.push_back({draw.roles[index], program.binary, program.userDataBase, program.firstUserSgpr, program.userData});
|
||||
}
|
||||
const auto predicted = merged.snapshot->prepared->entries.size();
|
||||
for (std::size_t index = 0; index < 2u; ++index) {
|
||||
const auto& program = draw.programs[index];
|
||||
const auto vertex = AgcDriver::Graphics::DecodeVertexStageInfo(program.binary.header, program.binary.headerAddress, program.userData, nullptr, true);
|
||||
RecompileRequest request{program.binary, {64u, program.firstUserSgpr, program.userData, {}, std::nullopt, vertex, memory}, device.Target(), {0, 0, 0, 128u}, GraphicsCompileContext{program.firstUserSgpr, linked, std::nullopt, draw.state.stages.tessellation, {0, 3, 4, 1}}};
|
||||
static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request));
|
||||
Require(merged.snapshot->prepared->entries.size() == predicted, "a merged hull stage prepared at ABI resolution was prepared again at its draw");
|
||||
request.graphics->tessellation->mergedHull = false;
|
||||
Require(!MatchesPreparedShader(request, *stages[index].entry.handle), "a separate hull configuration matched a merged hull artifact");
|
||||
}
|
||||
Fixture front;
|
||||
front.Initialize(5u);
|
||||
registry.emplace(front.snapshot->codeAddress, std::vector<std::shared_ptr<const ShaderSnapshot>>{front.snapshot});
|
||||
Fixture back;
|
||||
back.Initialize(7u);
|
||||
registry.emplace(back.snapshot->codeAddress, std::vector<std::shared_ptr<const ShaderSnapshot>>{back.snapshot});
|
||||
auto separate = queue;
|
||||
front.Bind(separate, 0x148u, 0x10bu);
|
||||
back.Bind(separate, 0x108u, 0x10bu);
|
||||
DrawDecode halves{};
|
||||
halves.state = prepared.state;
|
||||
halves.pixel = prepared.pixel;
|
||||
DecodeGraphicsPrograms(halves, separate, registry, false, true);
|
||||
Require(!halves.state.stages.tessellation->mergedHull && halves.programs[0].snapshot == front.snapshot && halves.programs[1].snapshot == back.snapshot, "separate hull halves were decoded as a merged hull program");
|
||||
auto halvesUnset = unset;
|
||||
front.Bind(halvesUnset, 0x148u, 0x10bu);
|
||||
back.Bind(halvesUnset, 0x108u, 0x10bu);
|
||||
DrawDecode halvesWithout{};
|
||||
halvesWithout.state = prepared.state;
|
||||
halvesWithout.pixel = prepared.pixel;
|
||||
Reject([&] { DecodeGraphicsPrograms(halvesWithout, halvesUnset, registry, false, true); }, "merged shader user-data address is null");
|
||||
}
|
||||
|
||||
void CheckMergedHullVertexTables(AgcDriver::VulkanDevice& device) {
|
||||
using namespace AgcDriver::DriverDetail;
|
||||
using namespace ShaderRecompiler;
|
||||
struct Header {
|
||||
Shader shader{};
|
||||
ShaderUserData users{};
|
||||
std::array<std::uint16_t, 11> direct{};
|
||||
ShaderSemantic semantic{};
|
||||
} header;
|
||||
alignas(256) std::array<std::uint32_t, 65> code{};
|
||||
code.fill(0xffffffffu);
|
||||
code.back() = 0xbf810000u;
|
||||
header.shader.type = 3;
|
||||
header.shader.user_data = &header.users;
|
||||
header.direct.fill(0xffffu);
|
||||
header.direct[8] = 10u;
|
||||
header.direct[10] = 12u;
|
||||
header.users.direct_resource_offset = header.direct.data();
|
||||
header.users.direct_resource_count = static_cast<std::uint16_t>(header.direct.size());
|
||||
header.semantic.size_in_elements = 4u;
|
||||
header.shader.input_semantics = &header.semantic;
|
||||
header.shader.num_input_semantics = 1u;
|
||||
auto merged = std::make_shared<ShaderSnapshot>();
|
||||
merged->type = 3;
|
||||
merged->codeAddress = reinterpret_cast<std::uintptr_t>(code.data());
|
||||
merged->headerAddress = reinterpret_cast<std::uintptr_t>(&header);
|
||||
merged->code.assign(code.begin(), code.end());
|
||||
merged->header.resize(sizeof(header));
|
||||
std::memcpy(merged->header.data(), &header, sizeof(header));
|
||||
Fixture domain;
|
||||
Fixture fragment;
|
||||
domain.Initialize(2u);
|
||||
fragment.Initialize(1u);
|
||||
ShaderRegistry registry;
|
||||
registry.emplace(merged->codeAddress, std::vector<std::shared_ptr<const ShaderSnapshot>>{merged});
|
||||
for (const auto* fixture : {&domain, &fragment}) registry.emplace(fixture->snapshot->codeAddress, std::vector<std::shared_ptr<const ShaderSnapshot>>{fixture->snapshot});
|
||||
AgcDriver::QueueState queue{};
|
||||
queue.context[0x8e] = 0xfu;
|
||||
queue.context[0x8f] = 0xfu;
|
||||
const auto entry = merged->codeAddress + 256u;
|
||||
queue.shader[0x148u] = static_cast<std::uint32_t>(entry >> 8u);
|
||||
queue.shader[0x149u] = static_cast<std::uint32_t>(entry >> 40u);
|
||||
queue.shader[0x10bu] = 16u << 1u;
|
||||
domain.Bind(queue, 0xc8u, 0x8bu);
|
||||
fragment.Bind(queue, 0x8u, 0xbu);
|
||||
alignas(16) const std::array<std::uint32_t, 4> bufferTable{};
|
||||
alignas(16) const std::array<std::uint32_t, 4> attributeTable{};
|
||||
const auto bufferAddress = reinterpret_cast<std::uintptr_t>(bufferTable.data());
|
||||
const auto attributeAddress = reinterpret_cast<std::uintptr_t>(attributeTable.data());
|
||||
for (std::uint32_t slot = 2u; slot < 6u; ++slot) queue.shader[0x10cu + slot] = slot % 2u == 0u ? 0x12345600u : 0x00000077u;
|
||||
queue.shader[0x10cu + 10u] = static_cast<std::uint32_t>(bufferAddress);
|
||||
queue.shader[0x10cu + 11u] = static_cast<std::uint32_t>(bufferAddress >> 32u);
|
||||
queue.shader[0x10cu + 12u] = static_cast<std::uint32_t>(attributeAddress);
|
||||
queue.shader[0x10cu + 13u] = static_cast<std::uint32_t>(attributeAddress >> 32u);
|
||||
DrawDecode draw{};
|
||||
draw.state.stages.path = AgcDriver::Graphics::ShaderPath::Tessellation;
|
||||
draw.state.stages.vertexWaveSize = 64u;
|
||||
draw.state.stages.fragmentWaveSize = 32u;
|
||||
draw.state.stages.tessellation = TessellationConfiguration{3, 3, 1, 2, 2};
|
||||
draw.pixel.wave32 = true;
|
||||
draw.pixel.targetOutputMode[0] = 9;
|
||||
draw.pixel.targetExportMapping.fill(0xe4u);
|
||||
DecodeGraphicsPrograms(draw, queue, registry, false, true);
|
||||
Require(draw.state.stages.tessellation->mergedHull && draw.programs[1].firstUserSgpr == 0u, "the merged hull draw with vertex tables was not decoded as a merged hull");
|
||||
for (std::size_t index = 0; index < 2u; ++index) {
|
||||
const auto& program = draw.programs[index];
|
||||
std::vector<AgcDriver::Graphics::DecodeRead> reads;
|
||||
const auto vertex = AgcDriver::Graphics::DecodeVertexStageInfo(program.binary.header, program.binary.headerAddress, program.userData, &reads, false, 8u - program.firstUserSgpr);
|
||||
Require(vertex.fetchEmbedded && vertex.fetchAttribReg == 12u && vertex.fetchBufferReg == 10u && vertex.resourcesNum == 1u, "a merged hull stage did not decode its vertex tables");
|
||||
Require(reads.size() == 2u && reads[0].address == attributeAddress && reads[1].address == bufferAddress, "a merged hull stage read its vertex tables from user-data words other than its slots");
|
||||
}
|
||||
}
|
||||
|
||||
void NullPixelMatchesRegistration(AgcDriver::VulkanDevice& device) {
|
||||
using namespace AgcDriver::DriverDetail;
|
||||
Fixture front;
|
||||
@@ -541,6 +733,8 @@ int main(int argc, char** argv) {
|
||||
Check(*device, AgcDriver::Graphics::ShaderPath::Vertex, dump);
|
||||
Check(*device, AgcDriver::Graphics::ShaderPath::Geometry, dump);
|
||||
Check(*device, AgcDriver::Graphics::ShaderPath::Tessellation, dump);
|
||||
CheckMergedHull(*device);
|
||||
CheckMergedHullVertexTables(*device);
|
||||
std::cout << "prepared graphics ABI tests passed\n";
|
||||
return 0;
|
||||
} catch (const std::exception& error) {
|
||||
|
||||
@@ -326,6 +326,90 @@ alignas(256) constexpr std::array<std::uint32_t, 38> QuadDomainCode{
|
||||
0x7e3402f2, 0xf80008cf, 0x1a191716, 0xf800020f, 0x1a19181a, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 87> MergedHullCode{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001c,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340000, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000, 0x000100c1,
|
||||
0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101, 0x362402ff,
|
||||
0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015, 0x162c2890,
|
||||
0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0781000, 0x02020216, 0x7da82881, 0xbe94047e,
|
||||
0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80048, 0x06000013, 0xd8dc1008, 0x07000013, 0xbf8cc07f,
|
||||
0x7e300306, 0x7e320306, 0x7e340306, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000, 0x0403180a,
|
||||
0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 87> MergedHullRegisters{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001c,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340000, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000, 0x000100c1,
|
||||
0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101, 0x362402ff,
|
||||
0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015, 0x162c2890,
|
||||
0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0741000, 0x02020816, 0x7da82881, 0xbe94047e,
|
||||
0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80048, 0x06000013, 0xd8dc1008, 0x07000013, 0xbf8cc07f,
|
||||
0x7e300306, 0x7e320306, 0x7e340306, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000, 0x0403180a,
|
||||
0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 87> MergedHullForeignRecord{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001c,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340020, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000, 0x000100c1,
|
||||
0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101, 0x362402ff,
|
||||
0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015, 0x162c2890,
|
||||
0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0781000, 0x02020216, 0x7da82881, 0xbe94047e,
|
||||
0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80048, 0x06000013, 0xd8dc1008, 0x07000013, 0xbf8cc07f,
|
||||
0x7e300306, 0x7e320306, 0x7e340306, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000, 0x0403180a,
|
||||
0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 88> MergedHullDivergentBarrier{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001d,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340000, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8a0000, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000,
|
||||
0x000100c1, 0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101,
|
||||
0x362402ff, 0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015,
|
||||
0x162c2890, 0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0781000, 0x02020216, 0x7da82881,
|
||||
0xbe94047e, 0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80048, 0x06000013, 0xd8dc1008, 0x07000013,
|
||||
0xbf8cc07f, 0x7e300306, 0x7e320306, 0x7e340306, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000,
|
||||
0x0403180a, 0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 87> MergedHullOutsidePatch{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001c,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340000, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000, 0x000100c1,
|
||||
0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101, 0x362402ff,
|
||||
0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015, 0x162c2890,
|
||||
0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0781000, 0x02020216, 0x7da82881, 0xbe94047e,
|
||||
0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80060, 0x06000013, 0xd8dc1008, 0x07000013, 0xbf8cc07f,
|
||||
0x7e300306, 0x7e320306, 0x7e340306, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000, 0x0403180a,
|
||||
0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 87> MergedHullReadLane{
|
||||
0xbefe04c1, 0xd7650000, 0x000100c1, 0xd7660000, 0x000200c1, 0x7da206f9, 0x80060000, 0xbf88001c,
|
||||
0xd5480004, 0x02090502, 0xd5480005, 0x02090102, 0xd5480006, 0x02110902, 0xd5480007, 0x02111102,
|
||||
0x7e080d04, 0x7e0a0d05, 0x7e0c0d06, 0x7e0e0d07, 0x10100cff, 0x3f19999a, 0x061010ff, 0xbf666666,
|
||||
0x3e1008ff, 0x3e800000, 0xd54b0009, 0x03c5e105, 0x161406a0, 0xd9340000, 0x0000080a, 0xd8340008,
|
||||
0x0000070a, 0xd8340010, 0x0000070a, 0xbf8cc07f, 0xbefe04c1, 0xbf8cc07f, 0xd7650000, 0x000100c1,
|
||||
0xd7660000, 0x000200c1, 0xbf8a0000, 0x7da206f9, 0x81060000, 0xd5480014, 0x02151101, 0x362402ff,
|
||||
0x000000ff, 0x160028a0, 0xd5430015, 0x040224ff, 0x00000060, 0xdbfc0000, 0x02000015, 0x162c2890,
|
||||
0xd5430016, 0x045a24ff, 0x00000030, 0xbf8cc07f, 0xe0781000, 0x02020216, 0x7da82881, 0xbe94047e,
|
||||
0xb06a0060, 0x162602f9, 0x0086066a, 0xd8d80048, 0x06000013, 0xd8dc1008, 0x07000013, 0xbf8cc07f,
|
||||
0xd7600010, 0x02010106, 0x7e300210, 0x7e360306, 0x34142484, 0xbefe0414, 0xe0781000, 0x0403180a,
|
||||
0x4a162481, 0x16161690, 0x4c1616ff, 0x00008000, 0xe0741000, 0x0202070b, 0xbf810000,
|
||||
};
|
||||
|
||||
alignas(256) constexpr std::array<std::uint32_t, 7> PixelCode{
|
||||
0xc8020002, 0xc8060102, 0xc80a0202, 0xc80e0302, 0xf800180f, 0x03020100, 0xbf810000,
|
||||
};
|
||||
@@ -333,6 +417,7 @@ alignas(256) constexpr std::array<std::uint32_t, 7> PixelCode{
|
||||
using ShaderRecompiler::TessellationConfiguration;
|
||||
|
||||
constexpr TessellationConfiguration Patches{3u, 3u, 1u, 2u, 2u};
|
||||
constexpr TessellationConfiguration MergedPatches{3u, 3u, 1u, 2u, 2u, 256u, 0u, false, 0u, false, true};
|
||||
constexpr std::array<std::uint32_t, 3> PatchLevels{1u, 0u, 5u};
|
||||
|
||||
constexpr std::uint16_t Corner(std::uint32_t patch, std::uint32_t x, std::uint32_t y) {
|
||||
@@ -379,6 +464,7 @@ alignas(256) constexpr std::array<std::uint16_t, 12> NextInputIndices{
|
||||
|
||||
const PatchDraw Triangles{Patches, Indices};
|
||||
const PatchDraw NextInputTriangles{{4u, 3u, 1u, 2u, 2u}, NextInputIndices};
|
||||
const PatchDraw MergedTriangles{MergedPatches, Indices};
|
||||
|
||||
struct Stages {
|
||||
ShaderRecompiler::RecompileResult local;
|
||||
@@ -842,6 +928,31 @@ void RequireSingleControlPointHulls(const ShaderRecompiler::SpirvTarget& target)
|
||||
Require(!CompileHull(target, HullSingleOutput, {1u, 1u, 1u, 2u, 2u}).spirv.empty(), "a hull with one output control point and unguarded patch outputs was not compiled");
|
||||
}
|
||||
|
||||
void RequireMergedHullWithoutSubgroups(const ShaderRecompiler::SpirvTarget& target) {
|
||||
std::uint32_t cursor = 0;
|
||||
const auto hull = Compile(target, ShaderStage::TessellationControl, MergedHullCode, cursor, MergedTriangles);
|
||||
const auto words = hull.spirv.Words();
|
||||
for (std::size_t at = 5; at + 1u < words.size(); at += words[at] >> 16u) {
|
||||
Require((words[at] >> 16u) != 0u, "the merged hull program compiled to a SPIR-V instruction without words");
|
||||
Require((words[at] & 0xffffu) != spv::OpCapability || words[at + 1u] != spv::CapabilityGroupNonUniform, "the hull stage of a merged hull program declares subgroup operations, which a device without tessellation control subgroups rejects");
|
||||
}
|
||||
}
|
||||
|
||||
void RequireRegisteredMergedStages() {
|
||||
const auto low = [](const auto& code) { return static_cast<std::uint32_t>(reinterpret_cast<std::uintptr_t>(code.data()) >> 8u); };
|
||||
const auto high = [](const auto& code) { return static_cast<std::uint32_t>(reinterpret_cast<std::uintptr_t>(code.data()) >> 40u); };
|
||||
RegisteredStage merged;
|
||||
RegisteredStage domain;
|
||||
Describe(merged, MergedHullCode, 3u, {{0x148u, low(MergedHullCode)}, {0x149u, high(MergedHullCode)}, {0x10au, 0u}, {0x10bu, 16u}}, {{0x2dbu, 0x40049u}, {0x286u, 0x42800000u}, {0x287u, 0x3f800000u}, {0x2d6u, 0xc355u}}, 0x105u);
|
||||
Describe(domain, DomainCode, 2u, {{0xc8u, low(DomainCode)}, {0xc9u, high(DomainCode)}, {0x8au, 0u}, {0x8bu, 16u}}, {}, 0x02002008u);
|
||||
for (const auto* stage : {&merged, &domain}) AgcDriverRegisterShader_nid_postfix(&stage->shader);
|
||||
const std::array<const Shader*, 2> stages{&merged.shader, &domain.shader};
|
||||
const std::array<ShaderRegister, 2> context{{{0x2d5u, 0x0200210du}, {0x29bu, 0u}}};
|
||||
const std::array<ShaderRegister, 3> primitive{{{0x25bu, 0u}, {0x262u, 0u}, {0x242u, 9u}}};
|
||||
AgcDriverResolveGraphicsStagesAbi_nid_postfix(stages, context, primitive);
|
||||
AgcDriverResolveGraphicsStagesAbi_nid_postfix(stages, context, primitive);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
int main() {
|
||||
@@ -877,6 +988,14 @@ int main() {
|
||||
RequirePatches(*device, target, HullAndSaveexec, "hull program narrowing exec with s_and_saveexec_b64");
|
||||
RequirePatches(*device, target, HullSaveexecLoop, "hull program restoring exec through loop phis");
|
||||
RequirePatches(*device, target, HullDescendingRecords, "hull program with descending patch records in 128-patch groups", {{3u, 3u, 1u, 2u, 2u, 128u}, Indices});
|
||||
ExpectRefusal(target, ShaderStage::TessellationControl, MergedHullCode, "more input than output control points", {{4u, 3u, 1u, 2u, 2u, 256u, 0u, false, 0u, false, true}, Indices});
|
||||
ExpectRefusal(target, ShaderStage::TessellationControl, MergedHullForeignRecord, "outside its own control point's record", MergedTriangles);
|
||||
ExpectRefusal(target, ShaderStage::TessellationControl, MergedHullDivergentBarrier, "s_barrier that some invocations do not reach", MergedTriangles);
|
||||
ExpectRefusal(target, ShaderStage::TessellationControl, MergedHullOutsidePatch, "reads LDS outside its patch's control point records", MergedTriangles);
|
||||
ExpectRefusal(target, ShaderStage::TessellationControl, MergedHullReadLane, "another lane's register", MergedTriangles);
|
||||
RequireMergedHullWithoutSubgroups(target);
|
||||
RequirePatches(*device, target, MergedHullCode, "merged hull program", MergedTriangles, MergedHullCode);
|
||||
RequirePatches(*device, target, MergedHullRegisters, "merged hull program storing its local part's registers", MergedTriangles, MergedHullRegisters);
|
||||
RequireCounterClockwise(*device, target);
|
||||
RequireLevelClamps(*device, target);
|
||||
RequireIntegerPartitioning(*device, target);
|
||||
@@ -888,6 +1007,7 @@ int main() {
|
||||
RequireHullBdaReads(*device, target);
|
||||
RequireCompareBranches(target);
|
||||
device.reset();
|
||||
RequireRegisteredMergedStages();
|
||||
RequireRegisteredStages();
|
||||
std::puts("Tessellation tests passed");
|
||||
return 0;
|
||||
|
||||
@@ -131,6 +131,7 @@ private:
|
||||
append(key, tessellation != nullptr);
|
||||
if (tessellation == nullptr) return;
|
||||
for (const auto value : {tessellation->inputControlPoints, tessellation->outputControlPoints, tessellation->domain, tessellation->partitioning, tessellation->outputTopology}) append(key, value);
|
||||
if (request.shader.stage != ShaderStage::TessellationEvaluation) append(key, tessellation->mergedHull);
|
||||
if (request.shader.stage == ShaderStage::Local) return;
|
||||
append(key, tessellation->patchesPerGroup);
|
||||
append(key, tessellation->primitiveGroupSize);
|
||||
|
||||
@@ -563,6 +563,7 @@ void writeTessellationConfiguration(Writer& writer, const TessellationConfigurat
|
||||
writer.WriteU32(configuration.patchesPerGroup);
|
||||
writer.WriteU32(configuration.primitiveGroupSize);
|
||||
writer.WriteBool(configuration.breakAtInstance);
|
||||
writer.WriteBool(configuration.mergedHull);
|
||||
}
|
||||
|
||||
TessellationConfiguration readTessellationConfiguration(Reader& reader, std::uint32_t version) {
|
||||
@@ -577,6 +578,7 @@ TessellationConfiguration readTessellationConfiguration(Reader& reader, std::uin
|
||||
configuration.primitiveGroupSize = reader.ReadU32();
|
||||
configuration.breakAtInstance = reader.ReadBool();
|
||||
}
|
||||
if (version >= 39u) configuration.mergedHull = reader.ReadBool();
|
||||
return configuration;
|
||||
}
|
||||
|
||||
@@ -825,7 +827,7 @@ std::string RequestSerializer::Serialize(const RecompileRequest& request) const
|
||||
std::string buffer;
|
||||
Writer writer(buffer);
|
||||
writer.WriteU32(0x41505335u);
|
||||
writer.WriteU32(38u);
|
||||
writer.WriteU32(39u);
|
||||
writeShaderBinary(writer, request.shader);
|
||||
writeGuestContext(writer, request.context);
|
||||
writeSpirvTarget(writer, request.target);
|
||||
@@ -864,7 +866,7 @@ DeserializedRequest RequestSerializer::Deserialize(std::string_view text) const
|
||||
Reader reader(decoded);
|
||||
if (reader.ReadU32() != 0x41505335u) throw std::runtime_error("invalid recompile request signature");
|
||||
const auto version = reader.ReadU32();
|
||||
if (version < 1u || version > 38u) throw std::runtime_error("unsupported recompile request serialization version");
|
||||
if (version < 1u || version > 39u) throw std::runtime_error("unsupported recompile request serialization version");
|
||||
DeserializedRequest result{};
|
||||
result.request.shader = readShaderBinary(reader, result.shaderCode, result.shaderHeader);
|
||||
result.request.context = readGuestContext(reader, result, version);
|
||||
|
||||
+1
@@ -33,6 +33,7 @@ struct IrProgramMetadata {
|
||||
bool vectorWindowIndexedWrites = false;
|
||||
std::uint32_t tessellationControlStride = 0;
|
||||
std::uint32_t tessellationPatchStride = 0;
|
||||
std::uint32_t tessellationSharedStride = 0;
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
+2
-1
@@ -16,7 +16,8 @@ enum class TessellationAttribute {
|
||||
Factor,
|
||||
PatchInput,
|
||||
InstanceOutput,
|
||||
InstanceInput
|
||||
InstanceInput,
|
||||
ControlShared
|
||||
};
|
||||
|
||||
enum class TessellationBaseKind {
|
||||
|
||||
@@ -123,6 +123,7 @@ struct ShaderTessellationInputInfo {
|
||||
bool breakAtInstance = false;
|
||||
std::uint32_t instancePatches = 0;
|
||||
bool drawParameters = false;
|
||||
bool mergedHull = false;
|
||||
};
|
||||
|
||||
struct ShaderVertexInputInfo {
|
||||
|
||||
@@ -3,12 +3,13 @@
|
||||
|
||||
#include "IntermediateRepresentation/IrProgram.hpp"
|
||||
#include "Optimization/ShaderStageInputInfo.hpp"
|
||||
#include "Recompiler.hpp"
|
||||
|
||||
namespace ShaderRecompiler {
|
||||
|
||||
class TessellationLowering {
|
||||
public:
|
||||
void Lower(IrProgram& program, const ShaderTessellationInputInfo& tessellation) const;
|
||||
void Lower(IrProgram& program, const ShaderTessellationInputInfo& tessellation, const TessellationTargetLimits* limits = nullptr) const;
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
@@ -336,10 +336,16 @@ void CollectComputeInputs(const ShaderComputeInputInfo* compute, ShaderInfo& inf
|
||||
}
|
||||
}
|
||||
|
||||
bool OwnControlPointWrite(const IrValue& inst) {
|
||||
if (inst.Opcode() != IrOpcode::SetTessellationAttribute) return false;
|
||||
const auto kind = static_cast<TessellationAttribute>(inst.Argument(0)->ImmediateU32());
|
||||
return kind == TessellationAttribute::ControlOutput || kind == TessellationAttribute::ControlShared;
|
||||
}
|
||||
|
||||
void CollectBuiltinInputs(const IrProgram& program, ShaderInfo& info) {
|
||||
for (const auto& block : program.Blocks()) {
|
||||
for (const IrValue* inst : block->Instructions()) {
|
||||
if (program.Resources().stage == IrShaderStage::TessellationControl && inst->Opcode() == IrOpcode::LaneId) {
|
||||
if (program.Resources().stage == IrShaderStage::TessellationControl && (inst->Opcode() == IrOpcode::LaneId || OwnControlPointWrite(*inst))) {
|
||||
AddInput(info, StageInputKind::InvocationId, 0, 1, "gl_InvocationID");
|
||||
}
|
||||
if (inst->Opcode() != IrOpcode::GetBuiltin) {
|
||||
|
||||
@@ -558,10 +558,16 @@ std::uint32_t ComponentCount(IrOpcode opcode) {
|
||||
|
||||
class Lowering {
|
||||
public:
|
||||
Lowering(IrProgram& program, const ShaderTessellationInputInfo& tessellation)
|
||||
: program(program), tessellation(tessellation), stage(program.Resources().stage), evaluator(stage == IrShaderStage::TessellationControl ? tessellation.outputControlPoints : 0u, stage == IrShaderStage::Local ? LocalVertexIndexCount : static_cast<std::int64_t>(tessellation.patchesPerGroup)) {}
|
||||
Lowering(IrProgram& program, const ShaderTessellationInputInfo& tessellation, const TessellationTargetLimits* limits)
|
||||
: program(program), tessellation(tessellation), limits(limits), stage(program.Resources().stage), merged(stage == IrShaderStage::TessellationControl && tessellation.mergedHull),
|
||||
evaluator(stage == IrShaderStage::TessellationControl ? tessellation.outputControlPoints : 0u, stage == IrShaderStage::Local ? LocalVertexIndexCount : static_cast<std::int64_t>(tessellation.patchesPerGroup)), localStride(program.Metadata().tessellationLocalStride) {}
|
||||
|
||||
void Run() {
|
||||
if (merged) {
|
||||
if (tessellation.inputControlPoints > tessellation.outputControlPoints) Fail("a merged hull program with more input than output control points runs its local part on lanes the hull stage does not have", 0u);
|
||||
requireUniformBarriers();
|
||||
requireOwnLanes();
|
||||
}
|
||||
if (stage == IrShaderStage::TessellationControl) lowerOwnLaneBits();
|
||||
collect();
|
||||
derive();
|
||||
@@ -591,6 +597,8 @@ public:
|
||||
program.Metadata().tessellationLocalStride = localStride;
|
||||
program.Metadata().tessellationControlStride = controlStride;
|
||||
program.Metadata().tessellationPatchStride = patchStride;
|
||||
program.Metadata().tessellationSharedStride = sharedStride;
|
||||
if (merged) requireMergedLimits();
|
||||
}
|
||||
|
||||
private:
|
||||
@@ -626,6 +634,8 @@ private:
|
||||
if (access.components == 0u || (shared != SharedAccess::Read && shared != SharedAccess::Write)) Fail("only 32-bit LDS reads and writes are supported in tessellation stages", flags.pc);
|
||||
if (stage == IrShaderStage::Local && access.write) {
|
||||
access.kind = TessellationAttribute::LocalOutput;
|
||||
} else if (merged) {
|
||||
access.kind = TessellationAttribute::ControlShared;
|
||||
} else if (stage == IrShaderStage::TessellationControl && !access.write) {
|
||||
access.kind = TessellationAttribute::ControlInput;
|
||||
} else {
|
||||
@@ -678,6 +688,10 @@ private:
|
||||
if (access.kind == TessellationAttribute::ControlInput && (access.address.dynamic || access.address.vertex == 0) && access.address.relative > 0 && access.address.relative % inputs == 0) agree(localStride, access.address.relative / inputs, access.pc);
|
||||
if (access.kind == TessellationAttribute::ControlOutput && access.address.vertex != 0) agree(controlStride, access.address.vertex, access.pc);
|
||||
if (access.kind == TessellationAttribute::ControlOutput && soleControlPoint(access)) agree(controlStride, access.address.relative, access.pc);
|
||||
if (access.kind == TessellationAttribute::ControlShared) {
|
||||
if (access.address.relative <= 0 || access.address.relative % inputs != 0) Fail("the merged hull program accesses LDS outside its patch", access.pc);
|
||||
agree(sharedStride, access.address.relative / inputs, access.pc);
|
||||
}
|
||||
if (access.kind == TessellationAttribute::EvaluationInput) {
|
||||
const auto end = access.address.constant + 4 * static_cast<std::int64_t>(access.components);
|
||||
if (access.address.relative > 0 && access.address.relative % outputs == 0 && access.address.constant >= 0 && end <= access.address.relative) {
|
||||
@@ -734,6 +748,13 @@ private:
|
||||
if (evaluator.Lowest(address) < static_cast<std::int64_t>(tessellation.patchesPerGroup) * outputs * controlStride) Fail("the domain shader reads off-chip memory that is neither its patch's control points nor a patch-constant record", access.pc);
|
||||
record(access, end);
|
||||
break;
|
||||
case TessellationAttribute::ControlShared:
|
||||
if (access.write) {
|
||||
if (address.vertex != sharedStride || address.constant < 0 || end > sharedStride) Fail("the merged hull program writes LDS outside its own control point's record", access.pc);
|
||||
} else if ((address.vertex != 0 && address.vertex != sharedStride) || address.constant < 0 || end + address.vertex * (outputs - 1) > inputs * sharedStride) {
|
||||
Fail("the merged hull program reads LDS outside its patch's control point records", access.pc);
|
||||
}
|
||||
break;
|
||||
case TessellationAttribute::PatchOutput:
|
||||
break;
|
||||
}
|
||||
@@ -841,6 +862,66 @@ private:
|
||||
}
|
||||
}
|
||||
|
||||
void requireUniformBarriers() const {
|
||||
const auto& blocks = program.BlockOrder();
|
||||
for (IrBlock* block : blocks) {
|
||||
if (std::none_of(block->Instructions().begin(), block->Instructions().end(), [](const IrValue* inst) { return inst->Opcode() == IrOpcode::Barrier; })) continue;
|
||||
std::unordered_set<IrBlock*> seen{block};
|
||||
std::vector<IrBlock*> pending;
|
||||
if (blocks.front() != block) pending.push_back(blocks.front());
|
||||
while (!pending.empty()) {
|
||||
IrBlock* current = pending.back();
|
||||
pending.pop_back();
|
||||
if (!seen.insert(current).second) continue;
|
||||
if (current->Successors().empty()) Fail("the merged hull program has an s_barrier that some invocations do not reach", 0u);
|
||||
pending.insert(pending.end(), current->Successors().begin(), current->Successors().end());
|
||||
}
|
||||
seen = {};
|
||||
pending.assign(block->Successors().begin(), block->Successors().end());
|
||||
while (!pending.empty()) {
|
||||
IrBlock* current = pending.back();
|
||||
pending.pop_back();
|
||||
if (current == block) Fail("the merged hull program has an s_barrier inside a loop", 0u);
|
||||
if (!seen.insert(current).second) continue;
|
||||
pending.insert(pending.end(), current->Successors().begin(), current->Successors().end());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void requireOwnLanes() const {
|
||||
for (IrBlock* block : program.BlockOrder()) {
|
||||
for (const IrValue* inst : block->Instructions()) {
|
||||
switch (inst->Opcode()) {
|
||||
case IrOpcode::ReadLane:
|
||||
case IrOpcode::WriteLane:
|
||||
case IrOpcode::DppMoveU32:
|
||||
case IrOpcode::DppUpdateU32:
|
||||
case IrOpcode::Permlane16U32:
|
||||
case IrOpcode::SwizzleU32:
|
||||
case IrOpcode::PermuteU32:
|
||||
case IrOpcode::BpermuteU32:
|
||||
Fail("the merged hull program reads or writes another lane's register, which its hull stage does not hold at one lane per invocation", 0u);
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void requireMergedLimits() const {
|
||||
if (limits == nullptr) Fail("the target has no tessellation limits for a merged hull program", 0u);
|
||||
const auto vectors = [](std::uint32_t bytes) { return (bytes + 15u) / 16u; };
|
||||
const auto patch = vectors(patchStride);
|
||||
const auto perVertex = vectors(controlStride) + vectors(sharedStride);
|
||||
const auto locations = vectors(controlStride) + patch + vectors(sharedStride);
|
||||
const auto total = 4ull * (static_cast<std::uint64_t>(tessellation.outputControlPoints) * perVertex + patch);
|
||||
if (4u * locations > limits->maxControlPerVertexOutputComponents || 4u * patch > limits->maxControlPerPatchOutputComponents || total > limits->maxControlTotalOutputComponents || 4u > limits->maxControlPerVertexInputComponents) {
|
||||
char text[256];
|
||||
std::snprintf(text, sizeof(text), "the merged hull program's outputs exceed the device limits (%u per-vertex locations, %u patch vectors, %llu components)", locations, patch, static_cast<unsigned long long>(total));
|
||||
Fail(text, 0u);
|
||||
}
|
||||
}
|
||||
|
||||
void requireFirstInvocation(const Access& access) {
|
||||
if (tessellation.outputControlPoints == 1u) return;
|
||||
std::unordered_set<IrValue*> path;
|
||||
@@ -895,7 +976,8 @@ private:
|
||||
auto address = access.address.constant + 4 * static_cast<std::int64_t>(component);
|
||||
if (access.kind == TessellationAttribute::PatchOutput || access.kind == TessellationAttribute::PatchInput) address -= FloorDiv(address, patchStride) * patchStride;
|
||||
IrValue& offset = constant(static_cast<std::uint32_t>(address));
|
||||
if (access.kind != TessellationAttribute::ControlInput || access.address.vertex == 0) return offset;
|
||||
const bool indexed = access.kind == TessellationAttribute::ControlInput || (access.kind == TessellationAttribute::ControlShared && !access.write);
|
||||
if (!indexed || access.address.vertex == 0) return offset;
|
||||
IrValue& invocation = insert(access, IrOpcode::GetBuiltin, IrType::U32, {&constant(static_cast<std::uint32_t>(StageInputKind::InvocationId)), &constant(0u)});
|
||||
IrValue& scaled = insert(access, IrOpcode::IMul32, IrType::U32, {&invocation, &constant(static_cast<std::uint32_t>(access.address.vertex))});
|
||||
return insert(access, IrOpcode::IAdd32, IrType::U32, {&scaled, &offset});
|
||||
@@ -954,20 +1036,23 @@ private:
|
||||
|
||||
IrProgram& program;
|
||||
const ShaderTessellationInputInfo& tessellation;
|
||||
const TessellationTargetLimits* limits;
|
||||
IrShaderStage stage;
|
||||
bool merged;
|
||||
AddressEvaluator evaluator;
|
||||
std::vector<Access> accesses;
|
||||
std::uint32_t localStride = 0;
|
||||
std::uint32_t localStride;
|
||||
std::uint32_t controlStride = 0;
|
||||
std::uint32_t patchStride = 0;
|
||||
std::uint32_t sharedStride = 0;
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
void TessellationLowering::Lower(IrProgram& program, const ShaderTessellationInputInfo& tessellation) const {
|
||||
void TessellationLowering::Lower(IrProgram& program, const ShaderTessellationInputInfo& tessellation, const TessellationTargetLimits* limits) const {
|
||||
const auto stage = program.Resources().stage;
|
||||
if (stage != IrShaderStage::Local && stage != IrShaderStage::TessellationControl && stage != IrShaderStage::TessellationEvaluation) return;
|
||||
Lowering(program, tessellation).Run();
|
||||
Lowering(program, tessellation, limits).Run();
|
||||
}
|
||||
|
||||
}
|
||||
@@ -86,11 +86,17 @@ ShaderStageKind toShaderStageKind(ShaderStage stage) {
|
||||
|
||||
namespace {
|
||||
|
||||
bool MergedHullStage(const RecompileRequest& request) {
|
||||
const auto stage = request.shader.stage;
|
||||
return (stage == ShaderStage::Local || stage == ShaderStage::TessellationControl) && request.graphics && request.graphics->tessellation && request.graphics->tessellation->mergedHull;
|
||||
}
|
||||
|
||||
// The host subgroup width wave64 programs are laid out for. Debug aid: APS5_SINGLE_LANE=<hex code
|
||||
// addresses, comma separated, or "all"> keeps the listed programs at one guest lane per invocation.
|
||||
std::uint32_t HostSubgroupSize(const RecompileRequest& request) {
|
||||
const bool vertexStage = request.shader.stage == ShaderStage::Vertex || request.shader.stage == ShaderStage::Local;
|
||||
if (vertexStage && !request.target.vertexSubgroups) return 1u;
|
||||
if (request.shader.stage == ShaderStage::TessellationControl && MergedHullStage(request)) return 1u;
|
||||
static const std::string list = [] { const char* text = std::getenv("APS5_SINGLE_LANE"); return text ? std::string(text) : std::string(); }();
|
||||
if (!list.empty()) {
|
||||
if (list == "all") return 64u;
|
||||
@@ -101,10 +107,16 @@ std::uint32_t HostSubgroupSize(const RecompileRequest& request) {
|
||||
return request.target.subgroupSize;
|
||||
}
|
||||
|
||||
RdnaProgram DecodeStageProgram(ShaderStageKind stage, std::span<const std::uint32_t> code) {
|
||||
RdnaProgram DecodeStageProgram(ShaderStageKind stage, std::span<const std::uint32_t> code, bool mergedHull) {
|
||||
static constexpr std::array<std::uint32_t, 1> EndProgram{0xbf810000u};
|
||||
if (stage == ShaderStageKind::Local && mergedHull) {
|
||||
RdnaProgram program;
|
||||
program.code = EndProgram;
|
||||
program.instructions.push_back(DecodeRdnaInstruction(0u, EndProgram, 0u));
|
||||
return program;
|
||||
}
|
||||
if (stage != ShaderStageKind::Local) return RdnaInstructionDecoder{}.Decode(code);
|
||||
auto program = DecodeRdnaFrontProgram(code);
|
||||
static constexpr std::array<std::uint32_t, 1> EndProgram{0xbf810000u};
|
||||
program.instructions.back() = DecodeRdnaInstruction(program.instructions.back().programCounter, EndProgram, 0u);
|
||||
return program;
|
||||
}
|
||||
@@ -134,8 +146,8 @@ std::atomic<std::uint64_t> frontEndBuilds{0};
|
||||
|
||||
struct PreparedControlFlow {
|
||||
PreparedControlFlow(const RecompileRequest& request, const SwappcInfo& swappcInfo)
|
||||
: stage(request.shader.stage), fallbacks(swappcInfo.linkedCallFallbacks.begin(), swappcInfo.linkedCallFallbacks.end()), swappc(WithFallbacks(swappcInfo, fallbacks)), code(request.shader.code.begin(), request.shader.code.end()),
|
||||
decoded(DecodeStageProgram(toShaderStageKind(request.shader.stage), code)), cfg(GraphBuilder{}.Build(decoded, &swappc)) {
|
||||
: stage(request.shader.stage), mergedHull(MergedHullStage(request)), fallbacks(swappcInfo.linkedCallFallbacks.begin(), swappcInfo.linkedCallFallbacks.end()), swappc(WithFallbacks(swappcInfo, fallbacks)), code(request.shader.code.begin(), request.shader.code.end()),
|
||||
decoded(DecodeStageProgram(toShaderStageKind(request.shader.stage), code, mergedHull)), cfg(GraphBuilder{}.Build(decoded, &swappc)) {
|
||||
Structurizer{}.Structurize(cfg);
|
||||
}
|
||||
|
||||
@@ -143,7 +155,7 @@ struct PreparedControlFlow {
|
||||
PreparedControlFlow& operator=(const PreparedControlFlow&) = delete;
|
||||
|
||||
bool Matches(const RecompileRequest& request, const SwappcInfo& swappcInfo) const {
|
||||
return stage == request.shader.stage && swappc.fetchCallAllowed == swappcInfo.fetchCallAllowed &&
|
||||
return stage == request.shader.stage && mergedHull == MergedHullStage(request) && swappc.fetchCallAllowed == swappcInfo.fetchCallAllowed &&
|
||||
swappc.userDataBaseRegister == swappcInfo.userDataBaseRegister && swappc.userDataCount == swappcInfo.userDataCount &&
|
||||
std::ranges::equal(fallbacks, swappcInfo.linkedCallFallbacks) && std::ranges::equal(code, request.shader.code);
|
||||
}
|
||||
@@ -154,6 +166,7 @@ struct PreparedControlFlow {
|
||||
}
|
||||
|
||||
ShaderStage stage;
|
||||
bool mergedHull;
|
||||
std::vector<std::uint32_t> fallbacks;
|
||||
SwappcInfo swappc;
|
||||
std::vector<std::uint32_t> code;
|
||||
@@ -218,8 +231,9 @@ IrProgram PrepareResourceProgram(const RecompileRequest& request, ShaderPreparat
|
||||
|
||||
constexpr InstructionTranslator translator;
|
||||
|
||||
if (frontend->mergedHull && cfg.hasFetchCall) throw std::runtime_error("a merged hull program that calls a fetch subroutine is unsupported");
|
||||
EmbeddedFetchPlan embeddedFetch;
|
||||
if ((stageKind == ShaderStageKind::Vertex || stageKind == ShaderStageKind::Local) && inputInfo.vertex != nullptr && inputInfo.vertex->fetchEmbedded) {
|
||||
if ((stageKind == ShaderStageKind::Vertex || stageKind == ShaderStageKind::Local) && !frontend->mergedHull && inputInfo.vertex != nullptr && inputInfo.vertex->fetchEmbedded) {
|
||||
embeddedFetch = EmbeddedVertexFetchAnalyzer{}.Analyze(decoded, inputInfo.vertex->fetchAttribReg, inputInfo.vertex->fetchBufferReg, request.context.userDataBaseRegister, static_cast<std::uint32_t>(request.context.userData.size()), request.context.waveSize);
|
||||
translateOptions.embeddedFetch = &embeddedFetch;
|
||||
}
|
||||
@@ -290,7 +304,7 @@ IrProgram PrepareResourceProgram(const RecompileRequest& request, ShaderPreparat
|
||||
|
||||
if (inputInfo.vertex != nullptr) {
|
||||
constexpr TessellationLowering tessellationLowering;
|
||||
tessellationLowering.Lower(program, inputInfo.vertex->tess);
|
||||
tessellationLowering.Lower(program, inputInfo.vertex->tess, request.target.tessellation ? &*request.target.tessellation : nullptr);
|
||||
}
|
||||
|
||||
constexpr SrtWalker srtWalker;
|
||||
|
||||
@@ -340,6 +340,7 @@ struct TessellationConfiguration {
|
||||
bool breakAtInstance = false;
|
||||
std::uint32_t instancePatches = 0;
|
||||
bool drawParameters = false;
|
||||
bool mergedHull = false;
|
||||
};
|
||||
|
||||
inline constexpr std::uint32_t MeshDrawPushOffsetBytes = 104;
|
||||
|
||||
@@ -94,7 +94,7 @@ struct SpirvEmitterState {
|
||||
SpirvModule module;
|
||||
const IrProgram& program;
|
||||
ShaderStageInputInfo inputInfo;
|
||||
std::array<std::uint32_t, 9> tessVariables {};
|
||||
std::array<std::uint32_t, 10> tessVariables {};
|
||||
std::uint32_t tessInnerVariable = 0;
|
||||
|
||||
SpirvRequirements requirements;
|
||||
|
||||
@@ -793,7 +793,7 @@ std::uint32_t TessellationPointer(SpirvValueEmitContext& ctx, const IrValue& ins
|
||||
const auto address = ctx.Arg(inst, 1);
|
||||
const bool patch = kind == TessellationAttribute::PatchOutput || kind == TessellationAttribute::PatchInput;
|
||||
const bool local = kind == TessellationAttribute::LocalOutput || kind == TessellationAttribute::ControlInput;
|
||||
const auto stride = local ? state.program.Metadata().tessellationLocalStride : state.program.Metadata().tessellationControlStride;
|
||||
const auto stride = local ? state.program.Metadata().tessellationLocalStride : kind == TessellationAttribute::ControlShared ? state.program.Metadata().tessellationSharedStride : state.program.Metadata().tessellationControlStride;
|
||||
const auto type = TypePointer(state, storage, TypeU32(state));
|
||||
const auto inPatch = [&](std::uint32_t vertex) {
|
||||
if (kind != TessellationAttribute::ControlInput) return vertex;
|
||||
@@ -801,9 +801,10 @@ std::uint32_t TessellationPointer(SpirvValueEmitContext& ctx, const IrValue& ins
|
||||
state.module.AddFunction(spv::OpExtInst, TypeU32(state), clamped, GlslStd450(state), GLSLstd450UMin, vertex, ConstantU32(state, state.inputInfo.vertex->tess.inputControlPoints - 1u));
|
||||
return clamped;
|
||||
};
|
||||
const bool sharedRead = kind == TessellationAttribute::ControlShared && inst.Opcode() == IrOpcode::GetTessellationAttribute;
|
||||
const auto* sum = inst.Argument(1);
|
||||
const auto* scaled = sum->Opcode() == IrOpcode::IAdd32 ? sum->Argument(0) : nullptr;
|
||||
if (kind == TessellationAttribute::ControlInput && scaled != nullptr && sum->Argument(1)->HasImmediate() && scaled->Opcode() == IrOpcode::IMul32 && scaled->Argument(1)->HasImmediate() && scaled->Argument(1)->ImmediateU32() == stride) {
|
||||
if ((kind == TessellationAttribute::ControlInput || sharedRead) && scaled != nullptr && sum->Argument(1)->HasImmediate() && scaled->Opcode() == IrOpcode::IMul32 && scaled->Argument(1)->HasImmediate() && scaled->Argument(1)->ImmediateU32() == stride) {
|
||||
const auto constant = sum->Argument(1)->ImmediateU32();
|
||||
const auto vertex = inPatch(EmitBinaryU32(state, spv::OpIAdd, ctx.Arg(*scaled, 0), ConstantU32(state, constant / stride)));
|
||||
state.module.AddFunction(spv::OpAccessChain, type, pointer, variable, vertex, ConstantU32(state, constant % stride / 16u), ConstantU32(state, constant % stride % 16u / 4u));
|
||||
@@ -815,7 +816,8 @@ std::uint32_t TessellationPointer(SpirvValueEmitContext& ctx, const IrValue& ins
|
||||
if (kind == TessellationAttribute::LocalOutput || patch) {
|
||||
state.module.AddFunction(spv::OpAccessChain, type, pointer, variable, attribute, component);
|
||||
} else {
|
||||
const auto vertex = kind == TessellationAttribute::ControlOutput ? EmitLaneId(state) : inPatch(EmitBinaryU32(state, spv::OpUDiv, address, ConstantU32(state, stride)));
|
||||
const bool own = kind == TessellationAttribute::ControlOutput || (kind == TessellationAttribute::ControlShared && !sharedRead);
|
||||
const auto vertex = own ? EmitLaneId(state) : inPatch(EmitBinaryU32(state, spv::OpUDiv, address, ConstantU32(state, stride)));
|
||||
state.module.AddFunction(spv::OpAccessChain, type, pointer, variable, vertex, attribute, component);
|
||||
}
|
||||
return pointer;
|
||||
@@ -824,7 +826,7 @@ std::uint32_t TessellationPointer(SpirvValueEmitContext& ctx, const IrValue& ins
|
||||
}
|
||||
|
||||
void DefineTessellationInterfaces(SpirvEmitterState& state) {
|
||||
std::array<bool, 9> used {};
|
||||
std::array<bool, 10> used {};
|
||||
for (const IrBlock* block : state.program.BlockOrder()) {
|
||||
for (const IrValue* inst : block->Instructions()) {
|
||||
if (inst->Opcode() != IrOpcode::GetTessellationAttribute && inst->Opcode() != IrOpcode::SetTessellationAttribute) {
|
||||
@@ -858,7 +860,7 @@ void DefineTessellationInterfaces(SpirvEmitterState& state) {
|
||||
type = array(TypeU32Vector(state, 4u), (state.program.Metadata().tessellationPatchStride + 15u) / 16u);
|
||||
} else {
|
||||
const bool local = kind == TessellationAttribute::LocalOutput || kind == TessellationAttribute::ControlInput;
|
||||
const auto stride = local ? state.program.Metadata().tessellationLocalStride : state.program.Metadata().tessellationControlStride;
|
||||
const auto stride = local ? state.program.Metadata().tessellationLocalStride : kind == TessellationAttribute::ControlShared ? state.program.Metadata().tessellationSharedStride : state.program.Metadata().tessellationControlStride;
|
||||
type = array(TypeU32Vector(state, 4u), (stride + 15u) / 16u);
|
||||
if (kind != TessellationAttribute::LocalOutput) {
|
||||
type = array(type, local ? tess.inputControlPoints : tess.outputControlPoints);
|
||||
@@ -874,7 +876,11 @@ void DefineTessellationInterfaces(SpirvEmitterState& state) {
|
||||
state.module.AddAnnotation(spv::OpDecorate, state.tessInnerVariable, spv::DecorationBuiltIn, spv::BuiltInTessLevelInner);
|
||||
state.module.AddAnnotation(spv::OpDecorate, state.tessInnerVariable, spv::DecorationPatch);
|
||||
} else {
|
||||
state.module.AddAnnotation(spv::OpDecorate, variable, spv::DecorationLocation, patch ? (state.program.Metadata().tessellationControlStride + 15u) / 16u : 0u);
|
||||
const auto controlLocations = (state.program.Metadata().tessellationControlStride + 15u) / 16u;
|
||||
const bool patchUsed = used[static_cast<std::uint32_t>(TessellationAttribute::PatchOutput)] || used[static_cast<std::uint32_t>(TessellationAttribute::PatchInput)];
|
||||
const auto patchLocations = patchUsed ? (state.program.Metadata().tessellationPatchStride + 15u) / 16u : 0u;
|
||||
const auto location = patch ? controlLocations : kind == TessellationAttribute::ControlShared ? controlLocations + patchLocations : 0u;
|
||||
state.module.AddAnnotation(spv::OpDecorate, variable, spv::DecorationLocation, location);
|
||||
}
|
||||
if (kind == TessellationAttribute::Factor || patch) {
|
||||
state.module.AddAnnotation(spv::OpDecorate, variable, spv::DecorationPatch);
|
||||
|
||||
@@ -188,6 +188,10 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO
|
||||
const auto tessellationBase = [&entryIr](TessellationBaseKind kind) -> IrValue& {
|
||||
return entryIr.Emit(IrOpcode::TessellationBase, IrOpcodeType(IrOpcode::TessellationBase), {&entryIr.Constant(static_cast<std::uint32_t>(kind))});
|
||||
};
|
||||
constexpr std::uint32_t MergedHullRecordBytes = 16u;
|
||||
const auto mergedHullRecord = [&entryIr](IrValue& address) -> IrValue& {
|
||||
return entryIr.Emit(IrOpcode::GetTessellationAttribute, IrOpcodeType(IrOpcode::GetTessellationAttribute), {&entryIr.Constant(static_cast<std::uint32_t>(TessellationAttribute::ControlInput)), &address, &entryIr.ConstantBool(true)});
|
||||
};
|
||||
|
||||
for (std::uint32_t index = 0; index < options.userDataCount; index++) {
|
||||
const auto reg = static_cast<ScalarReg>(options.userDataBaseRegister + index);
|
||||
@@ -325,6 +329,13 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO
|
||||
setVector(static_cast<VectorReg>(2), drawIndex(StageInputKind::VertexIndex));
|
||||
setVector(static_cast<VectorReg>(3), tessellationBase(TessellationBaseKind::RelativeIndex));
|
||||
setVector(static_cast<VectorReg>(5), drawIndex(StageInputKind::InstanceIndex));
|
||||
if (options.inputInfo.vertex->tess.mergedHull) {
|
||||
const auto kind = static_cast<std::uint32_t>(TessellationAttribute::LocalOutput);
|
||||
for (const auto [offset, index] : {std::pair{0u, StageInputKind::VertexIndex}, std::pair{4u, StageInputKind::InstanceIndex}}) {
|
||||
static_cast<void>(entryIr.Emit(IrOpcode::SetTessellationAttribute, IrType::Void, {&entryIr.Constant(kind), &entryIr.Constant(offset), &builtin(index), &entryIr.ConstantBool(true)}));
|
||||
}
|
||||
program.Metadata().tessellationLocalStride = MergedHullRecordBytes;
|
||||
}
|
||||
} else if (options.stage == ShaderStageKind::TessellationControl) {
|
||||
const auto& tess = options.inputInfo.vertex->tess;
|
||||
entryIr.SetScalarReg(static_cast<ScalarReg>(2), tessellationBase(TessellationBaseKind::OffChip));
|
||||
@@ -332,6 +343,14 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO
|
||||
entryIr.SetScalarReg(static_cast<ScalarReg>(3), entryIr.Constant(0x81010000u | tess.inputControlPoints | (tess.outputControlPoints << 8u)));
|
||||
setVector(static_cast<VectorReg>(0), builtin(StageInputKind::PrimitiveId));
|
||||
setVector(static_cast<VectorReg>(1), entryIr.BitwiseOr(entryIr.ShiftLeftLogical(builtin(StageInputKind::InvocationId), entryIr.Constant(8u)), tessellationBase(TessellationBaseKind::RelativeIndex)));
|
||||
if (tess.mergedHull) {
|
||||
IrValue& point = entryIr.Emit(IrOpcode::UMin32, IrOpcodeType(IrOpcode::UMin32), {&builtin(StageInputKind::InvocationId), &entryIr.Constant(tess.inputControlPoints - 1u)});
|
||||
IrValue& record = entryIr.IMul(point, entryIr.Constant(MergedHullRecordBytes));
|
||||
setVector(static_cast<VectorReg>(2), mergedHullRecord(record));
|
||||
setVector(static_cast<VectorReg>(3), entryIr.IAdd(entryIr.IMul(tessellationBase(TessellationBaseKind::RelativeIndex), entryIr.Constant(tess.inputControlPoints)), builtin(StageInputKind::InvocationId)));
|
||||
setVector(static_cast<VectorReg>(5), mergedHullRecord(entryIr.IAdd(record, entryIr.Constant(4u))));
|
||||
program.Metadata().tessellationLocalStride = MergedHullRecordBytes;
|
||||
}
|
||||
} else if (options.stage == ShaderStageKind::TessellationEvaluation) {
|
||||
entryIr.SetScalarReg(static_cast<ScalarReg>(3), entryIr.Constant(64u));
|
||||
entryIr.SetScalarReg(static_cast<ScalarReg>(4), tessellationBase(TessellationBaseKind::OffChip));
|
||||
|
||||
@@ -206,6 +206,7 @@ ShaderStageInputInfo BuildShaderStageInputInfo(ShaderStageKind stage, const Gues
|
||||
target.breakAtInstance = tessellation->breakAtInstance;
|
||||
target.instancePatches = tessellation->instancePatches;
|
||||
target.drawParameters = tessellation->drawParameters;
|
||||
target.mergedHull = tessellation->mergedHull;
|
||||
}
|
||||
if (stage == ShaderStageKind::Mesh) {
|
||||
if (mesh == nullptr) throw std::runtime_error("ShaderInputInfoBuilder: a mesh-stage program has no mesh configuration");
|
||||
|
||||
@@ -214,11 +214,11 @@ int main() {
|
||||
Require(request.graphics->draw.indexAddress == 0xabcdef3400u && request.graphics->draw.indexCount == 57u && request.graphics->draw.indexElementBytes == 4u && request.graphics->draw.instanceCount == 9u);
|
||||
Require(request.target.float16DenormPreserve == preserve);
|
||||
auto upgraded = bytes;
|
||||
upgraded[4] = 38u;
|
||||
upgraded[4] = 39u;
|
||||
InsertMeshBarycentricFlag(upgraded);
|
||||
if (fixture.pixel) upgraded.insert(upgraded.begin() + 258u, {0u, 0u, 0u});
|
||||
if (fixture.pixel) InsertPixelDefaultsAfterFrontFace(upgraded);
|
||||
if (fixture.tessellation) upgraded.insert(upgraded.end() - 35u, 5u, 0u);
|
||||
if (fixture.tessellation) upgraded.insert(upgraded.end() - 35u, 6u, 0u);
|
||||
upgraded.insert(upgraded.end(), sizeof(std::uint64_t) + 1u + 5u + 1u, 0u);
|
||||
Require(serializer.Serialize(request) == EncodeBase64(upgraded));
|
||||
auto truncated = bytes;
|
||||
@@ -254,11 +254,11 @@ int main() {
|
||||
layerRejected = std::string_view(error.what()) == "RequestSerializer: truncated data";
|
||||
}
|
||||
Require(layerRejected);
|
||||
layered[4] = 38u;
|
||||
layered[4] = 39u;
|
||||
InsertMeshBarycentricFlag(layered);
|
||||
layered.insert(layered.begin() + 259u, {0u, 0u});
|
||||
InsertPixelDefaultsAfterFrontFace(layered);
|
||||
if (fixture.tessellation) layered.insert(layered.end() - 35u, 5u, 0u);
|
||||
if (fixture.tessellation) layered.insert(layered.end() - 35u, 6u, 0u);
|
||||
layered.insert(layered.end(), sizeof(std::uint64_t) + 1u + 5u + 1u, 0u);
|
||||
Require(serializer.Serialize(layerReplay.request) == EncodeBase64(layered));
|
||||
}
|
||||
@@ -280,7 +280,7 @@ int main() {
|
||||
Require(replay.request.context.pixel->rasterPrimitive == ShaderRecompiler::RasterPrimitive::Rectangle);
|
||||
Require(replay.request.context.vertex->paClVsOutCntl == 0x00050a05u);
|
||||
Require(replay.request.target.float16DenormPreserve == preserve);
|
||||
bytes[4] = 38u;
|
||||
bytes[4] = 39u;
|
||||
InsertMeshBarycentricFlag(bytes);
|
||||
bytes.insert(bytes.begin() + 259u, {0u, 0u});
|
||||
InsertPixelDefaultsAfterFrontFace(bytes);
|
||||
@@ -404,13 +404,23 @@ int main() {
|
||||
Require(!fixture.tessellation || (request.graphics->tessellation->patchesPerGroup == 63u && request.graphics->tessellation->primitiveGroupSize == 37u && request.graphics->tessellation->breakAtInstance));
|
||||
Require(request.graphics->draw.indexAddress == 0xabcdef3400u && request.graphics->draw.indexCount == 57u && request.graphics->draw.indexElementBytes == 4u && request.graphics->draw.instanceCount == 9u);
|
||||
Require(request.target.float16DenormPreserve == preserve);
|
||||
auto upgraded = bytes;
|
||||
upgraded[4] = 38u;
|
||||
InsertMeshBarycentricFlag(upgraded);
|
||||
if (fixture.pixel) upgraded.insert(upgraded.begin() + 259u, {0u, 0u});
|
||||
if (fixture.pixel) InsertPixelDefaultsAfterFrontFace(upgraded);
|
||||
auto main38 = bytes;
|
||||
main38[4] = 38u;
|
||||
InsertMeshBarycentricFlag(main38);
|
||||
if (fixture.pixel) main38.insert(main38.begin() + 259u, {0u, 0u});
|
||||
if (fixture.pixel) InsertPixelDefaultsAfterFrontFace(main38);
|
||||
auto upgraded = main38;
|
||||
upgraded[4] = 39u;
|
||||
if (fixture.tessellation) upgraded.insert(upgraded.end() - 35u, 1u, 0u);
|
||||
main38.insert(main38.end(), sizeof(std::uint64_t) + 1u + 5u + 1u, 0u);
|
||||
upgraded.insert(upgraded.end(), sizeof(std::uint64_t) + 1u + 5u + 1u, 0u);
|
||||
Require(serializer.Serialize(request) == EncodeBase64(upgraded));
|
||||
const auto replay38 = serializer.Deserialize(EncodeBase64(main38));
|
||||
Require(replay38.request.graphics.has_value() && replay38.request.graphics->tessellation.has_value() == fixture.tessellation);
|
||||
Require(!fixture.tessellation || (!replay38.request.graphics->tessellation->mergedHull && replay38.request.graphics->tessellation->primitiveGroupSize == 37u && replay38.request.graphics->tessellation->breakAtInstance));
|
||||
Require(replay38.request.graphics->draw.indexAddress == 0xabcdef3400u && replay38.request.graphics->draw.indexCount == 57u && replay38.request.graphics->draw.indexElementBytes == 4u && replay38.request.graphics->draw.instanceCount == 9u);
|
||||
Require(replay38.request.target.float16DenormPreserve == preserve);
|
||||
Require(serializer.Serialize(replay38.request) == EncodeBase64(upgraded));
|
||||
for (const bool dither : {false, true}) {
|
||||
if (!fixture.pixel && dither) continue;
|
||||
auto main25 = bytes;
|
||||
@@ -419,12 +429,13 @@ int main() {
|
||||
const auto replay25 = serializer.Deserialize(EncodeBase64(main25));
|
||||
Require(!fixture.pixel || (replay25.request.context.pixel->layerExported == layer && replay25.request.context.pixel->ditheredAlphaToCoverage == dither && replay25.request.context.pixel->reversedBlend == 0u));
|
||||
Require(replay25.request.target.float16DenormPreserve == preserve && (!fixture.tessellation || replay25.request.graphics->tessellation->primitiveGroupSize == 37u));
|
||||
main25[4] = 38u;
|
||||
main25[4] = 39u;
|
||||
InsertMeshBarycentricFlag(main25);
|
||||
if (fixture.pixel) {
|
||||
main25.insert(main25.begin() + 260u, 0u);
|
||||
InsertPixelDefaultsAfterFrontFace(main25);
|
||||
}
|
||||
if (fixture.tessellation) main25.insert(main25.end() - 35u, 1u, 0u);
|
||||
main25.insert(main25.end(), sizeof(std::uint64_t) + 1u + 5u + 1u, 0u);
|
||||
Require(serializer.Serialize(replay25.request) == EncodeBase64(main25));
|
||||
}
|
||||
@@ -440,9 +451,9 @@ int main() {
|
||||
const auto replay = serializer.Deserialize(encoded);
|
||||
Require(replay.request.target.float16DenormPreserve == preserve && replay.request.target.float32FlushToZero == !preserve);
|
||||
auto bytes = DecodeBase64(encoded);
|
||||
Require(bytes.size() >= 8u && bytes[4] == 38u && bytes[5] == 0u && bytes[6] == 0u && bytes[7] == 0u);
|
||||
Require(bytes.size() >= 8u && bytes[4] == 39u && bytes[5] == 0u && bytes[6] == 0u && bytes[7] == 0u);
|
||||
auto future = bytes;
|
||||
future[4] = 39u;
|
||||
future[4] = 40u;
|
||||
bool futureRejected = false;
|
||||
try {
|
||||
static_cast<void>(serializer.Deserialize(EncodeBase64(future)));
|
||||
|
||||
@@ -169,3 +169,4 @@
|
||||
- [Scalar buffer descriptors](../../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) use only the base address, stride and record count for resources accessed exclusively by scalar loads; their fourth word is normalized to zero, following AMD RDNA2 ISA section 7.2.2. Vector and mixed accesses retain descriptor validation. Native Vulkan readbacks cover ignored fourth-word fields on Windows/NVIDIA; their console behavior has not been measured.
|
||||
- [Spanned wave64 compute](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvSubgroup.cpp) (shader recompiler) - `APS5_SPANNED_WAVE=1`, off by default, runs a single-wave wave64 compute program at one lane per invocation on a 32-lane host subgroup and exchanges ballots, shuffles and reductions between the two host subgroups through workgroup memory and barriers. The driver turns it on only where compute subgroups can be pinned at 32 full lanes (`subgroupSizeControl` and `computeFullSubgroups`), passes it as `SpirvTarget::spannedWaves` (part of the cache key and the serialized request), and creates the pipelines of spanned programs with `REQUIRE_FULL_SUBGROUPS`, since the lane number `SubgroupId * 32 + SubgroupLocalInvocationId` needs two full subgroups. Those barriers are correct only where every invocation of the workgroup reaches them, which holds for the guest's wave-uniform control flow. The emulated 64-bit LDS atomics ([LockedLdsUpdate](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp)) run only on the lanes the exec mask keeps, so their lock loop ballots within its own host subgroup, as its elect already does. Any other cross-lane exchange emitted inside per-lane control flow would hang a spanned wave in the same way, and the emitter does not reject one: there is no uniformity check. The only check so far is manual: disassemble the module (`spirv-dis`), list the selection and loop constructs that enclose each `OpControlBarrier`, and read the condition of each enclosing selection. In Ghost of Yōtei's program 0x80003af600 (PPSA26344), 108 of 111 barriers sat only in loops on wave-uniform conditions and 3 inside per-invocation conditionals, all in the lock loop. Other programs and titles have not been scanned. An earlier, cheaper ballot (one barrier instead of three) also lost the device for a reason that is not understood, which may be another barrier placement problem. Measured on an RTX 4090 (Windows): an offline replay of that program and two Yōtei runs that previously lost the device (NVIDIA TDR) now run past it, and [LdsMaskWaterfall.cpp](../../../core/libs/prx/libSceAgcDriver/tests/execution/LdsMaskWaterfall.cpp) runs both modes and covers a half-inactive wave. Stray (PPSA02101) on Intel ANV and a Demon's Souls (PPSA01342) request replay showed no fault, before this fix
|
||||
- [Guest memory load guards](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (shader recompiler) - a guest load whose lane is inactive or whose address is out of bounds is still performed and its result replaced by 0 with `OpSelect`, instead of a selection per guard, when the load can be pointed at a location nothing writes: scratch reads its own element 0; storage buffers (direct buffers, GDS) read element 0xfffffffe, which `robustBufferAccess2` turns into a zero load with no memory access, so the extra read cannot race with a write the guarded program would not have raced with ([Vulkan memory model](https://docs.vulkan.org/spec/latest/appendices/memorymodel.html#data-race)). Without `robustBufferAccess2` ([VulkanDevice.cpp](../../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) enables it when the device has it) those loads keep their selections; LDS loads, stores, atomics, formatted and physical-address (BDA) accesses always do. LDS keeps the branch because NVIDIA's compiler predicates a branched LDS load at no cost, while the select form costs about 2.5 instructions per dword and splits a `ds_read_b128` into four loads (RTX 4060 measurements by @karanvora2599 in #4048); Intel's static counts favour the select only for wide reads, and run time was not measured on either. The aligned high word of a misaligned-capable buffer read re-reads the low word's own element. A selection guard costs four blocks per lane half and the host compiler's time grows with the block count: PPSA11142's merged PSP vertex program (3176 ISA words, wave64) emitted 16,983 blocks and 1,710 conditional branches, 5,907 and 868 with the selects. `agc_guarded_memory_loads` traces the emitted loads in both device modes. AMD's proprietary driver does not get `robustBufferAccess2` ([RobustBufferAccess.hpp](../../../core/libs/prx/libSceAgcDriver/Execution/include/RobustBufferAccess.hpp)), so its loads keep their selections. On an RX 6700 XT with driver 32.0.21045.5002, enabling the feature broke `agc_driver_layered_volume_draw` and `agc_driver_layered_volume_vertex`: the vertex draw left the volume untouched even with the selections kept. `agc_driver_subvector_loop` failed too, with the selected loads: the driver compiled the begin instruction's `exec_lo == 0` as a test of the whole 64-bit mask, so the high half of a wave64 skipped its loop. The begin instruction now tests for an empty mask as `(exec_lo | exec_hi) == 0` rather than `exec_lo == 0 && exec_hi == 0`, and that driver compiles it correctly. The cause of the layered-volume failures was not found.
|
||||
- [Merged hull programs](../../../core/shader/recompiler/Optimization/src/TessellationLowering.cpp) (registered binary type 3) run whole in the hull stage, one invocation per output control point. On the console hull lane i keeps local lane i's registers, which with fewer input than output control points stop matching the patch's control points from i = inputs on; here invocation i holds control point min(i, inputs - 1) of its own patch. A wave-mask branch (`s_cbranch_exec[n]z`, `s_cbranch_vcc[n]z`) tests the invocation's own lane, so it can skip a block, scalar instructions included, that the console runs for the wave; `v_readfirstlane_b32` returns the invocation's own value, not the first active lane's, and another lane's register throws. The register layout (`s3`, `v0`-`v5`, as for the separate halves) is not measured on hardware. s[0:1] hold the hull user-data address (SH 0x102/0x103) when a draw sets it and zero otherwise, so a program reading it unset faults at the null address.
|
||||
Reference in new issue
Block a user