fix(libSceAudio3d): let open ex check like open in a PS4 process

In firmware 13.42 sceAudio3dPortOpenEx only sets the flag that skips
the PS4-process AudioOut2 advances, which this library never makes, so
it now opens like sceAudio3dPortOpen. A PS4-process open reports an
open port before an unsupported buffer mode or bed count, as the
firmware does, and the debt entry records the advances after an open
and a push for a PS4 SDK of 0x11000000 or later.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

core/libs/prx/libSceAudio3d/Export.cpp

@@ -200,9 +200,9 @@ int OpenPs4Port(const char* function, int user_id, const Audio3dOpenParameters*
     if (max_objects == 0 || queue_depth == 0) return AUDIO3D_ERROR_INVALID_PARAMETER;
     if (buffer_mode > AUDIO3D_BUFFER_ADVANCE_AND_PUSH) return AUDIO3D_ERROR_INVALID_PARAMETER;
     if (num_beds != 2 && num_beds != 3) return AUDIO3D_ERROR_INVALID_PARAMETER;
+    if (g_port.open) return AUDIO3D_ERROR_OUT_OF_RESOURCES;
     if (buffer_mode != AUDIO3D_BUFFER_ADVANCE_AND_PUSH) Unsupported(function, "buffer mode " + std::to_string(buffer_mode));
     if (num_beds != 2) Unsupported(function, "3 beds");
-    if (g_port.open) return AUDIO3D_ERROR_OUT_OF_RESOURCES;
     StorePort(queue_depth, parameters->granularity, buffer_mode, max_objects, num_beds, id);
     return 0;
 }
@@ -667,7 +667,6 @@ const char* APS5_VABI sceAudio3dStrError(std::int32_t error) {
 }
 
 int APS5_VABI sceAudio3dPortOpenEx(int user_id, const Audio3dOpenParameters* parameters, uint32_t* id) {
-    if (Ps4Process()) Unsupported(__func__, "PS4 process");
     return sceAudio3dPortOpen(user_id, parameters, id);
 }
 

core/libs/tests/GuestAudio3dPort.cpp

@@ -618,7 +618,11 @@ void CheckPs4Process() {
     for (const std::uint32_t beds : {0u, 1u, 4u}) {
         Require(open([&](auto& p) { p.size_this = 0x28; p.num_beds = beds; }) == INVALID_PARAMETER && id == 7, "a PS4 process needs 2 or 3 beds");
     }
-    RequireThrows([&] { sceAudio3dPortOpenEx(SYSTEM_USER, &defaults, &id); }, "open ex in a PS4 process is unsupported");
+    Require(sceAudio3dPortOpenEx(1, &defaults, &id) == INVALID_PARAMETER && id == 7, "open ex in a PS4 process checks like open");
+    Require(sceAudio3dPortOpenEx(SYSTEM_USER, &defaults, &id) == 0 && id == 0, "open ex in a PS4 process opens the port");
+    id = 7;
+    Require(open([](auto& p) { p.buffer_mode = 0; }) == OUT_OF_RESOURCES && id == 7, "a PS4 process reports the open port before an unsupported buffer mode");
+    Require(sceAudio3dPortClose(0) == 0, "close after open ex in a PS4 process");
     Require(sceAudio3dPortCreate(0x100, 0, 1, &id) == INVALID_PARAMETER && id == 7, "create in a PS4 process needs a zero reserved argument");
     RequireThrows([&] { sceAudio3dPortCreate(0x100, 0, 0, &id); }, "create in a PS4 process selects the unsupported buffer mode 0");
     Require(open([](auto& p) { p.size_this = 0x28; p.num_beds = 2; }) == 0 && id == 0, "a PS4 process opens with 2 beds");

docs/dev/debt/libSceAudio3d.md

@@ -2,7 +2,7 @@
 
 ### Silent stubs
 
-- [libSceAudio3d](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - `sceAudio3dObjectSetAttributes` makes the firmware 13.42 checks (the count and the array, then the port, attribute ids 1 to 11, the format and sample count of a `PCM` block, the reserved object, then a value for every id except `RESET_STATE`) but applies nothing, so an object has no samples, no gain and no position; the value checks AudioOut2 makes when it applies them are not modelled. `sceAudio3dPortSetAttribute` checks the late reverb level (0 to 1), the downmix spread radius (0.1 to 2) and the height-aware value as firmware 13.42 does and ignores them, and `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output
+- [libSceAudio3d](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - `sceAudio3dObjectSetAttributes` makes the firmware 13.42 checks (the count and the array, then the port, attribute ids 1 to 11, the format and sample count of a `PCM` block, the reserved object, then a value for every id except `RESET_STATE`) but applies nothing, so an object has no samples, no gain and no position; the value checks AudioOut2 makes when it applies them are not modelled. `sceAudio3dPortSetAttribute` checks the late reverb level (0 to 1), the downmix spread radius (0.1 to 2) and the height-aware value as firmware 13.42's AudioOut2 (libSceAudioOut.sprx) does when Audio3d passes them on, and ignores them, and `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output
 
 ### Functional
 
@@ -16,7 +16,7 @@
 - [sceAudio3dReportRegisterHandler](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - the 20 call reports reach the type 0 handlers after the call's own lock is released, while firmware 13.42 calls them under its report lock, where a handler that registers or unregisters a handler never returns. Here a report already under way on another thread can still call a handler after `sceAudio3dReportUnregisterHandler` returned, which the console's lock prevents
 - [sceAudio3dPortDestroy](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - closes port 0 through `sceAudio3dPortClose`. Firmware 13.42 passes the port's AudioOut2 context number, not the port id, to `sceAudio3dPortClose`, which accepts only 0, so the console closes the port only when that number is 0. The number comes from the audio service and cannot be read from the module; this project's port is context 0
 - [sceAudio3dPortOpen](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - a failed open leaves the open port's settings alone, while firmware 13.42 stores a parameter block of a known size before it checks the user, the values or the open port, so on the console a failed open while a port is open changes the granularity and buffer mode that `sceAudio3dAudioOutOpen`, `sceAudio3dPortAdvance` and `sceAudio3dPortPush` check afterwards
-- [sceAudio3dPortOpen, sceAudio3dPortOpenEx, sceAudio3dBedWrite2 and sceAudio3dAudioOutOpen in a PS4 process](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - firmware 13.42 stores `sceKernelIsPs4Process` at `sceAudio3dInitialize` and takes a PS4 branch in these calls. Its answers are followed where this project can give them: buffer mode 0 to 2 and 2 or 3 beds are valid, route 1 is rejected on a 2-bed port, and type 3 with `param` bit 17 is 0x80260007. These throw: buffer modes 0 and 1 and 3 beds (not modelled), `sceAudio3dPortOpenEx` (it also skips AudioOut2 advances for SDK 0x11000000 and later), and formats 6 and 7, which the firmware rewrites to 2 and 5 for a PS4 SDK up to 0x3FFFFFF while `sceKernelGetPs4CompiledSdkVersion` throws here. Routes 1 and 2 play on both outputs
+- [sceAudio3dPortOpen, sceAudio3dPortPush, sceAudio3dBedWrite2 and sceAudio3dAudioOutOpen in a PS4 process](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - firmware 13.42 stores `sceKernelIsPs4Process` at `sceAudio3dInitialize` and takes a PS4 branch in these calls. Its answers are followed where this project can give them: buffer mode 0 to 2 and 2 or 3 beds are valid, route 1 is rejected on a 2-bed port, and type 3 with `param` bit 17 is 0x80260007. Buffer modes 0 and 1 and 3 beds throw (not modelled), and so do formats 6 and 7, which the firmware rewrites to 2 and 5 for a PS4 SDK up to 0x3FFFFFF while `sceKernelGetPs4CompiledSdkVersion` throws here. For a PS4 SDK of 0x11000000 or later the firmware also advances the AudioOut2 context after a successful open and after a push that leaves the queue empty, unless `sceAudio3dPortOpenEx` was called; neither advance happens here. Routes 1 and 2 play on both outputs
 - [sceAudio3dBedWrite2](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - 24 channels return `INVALID_PARAMETER`, while firmware 13.42 also accepts 24 (besides 2, 6 and 8) and passes the bed to AudioOut2, whose handling of it is unknown
 - [sceAudio3dCreateSpeakerArray](../../../core/libs/prx/libSceAudio3d/Export.cpp) (libSceAudio3d) - the speaker-array calls return the AudioOut2 result unchanged: the console translates AudioOut2 speaker-array errors into Audio3d codes, but this project's AudioOut2 speaker-array calls throw instead of returning error codes, so there is nothing to translate yet
 - [Audio3d bed downmix coefficients](../../../core/libs/prx/libSceAudio3d/Mixer.hpp) (libSceAudio3d) - the centre and the surrounds fold into stereo with a 0.7071 gain and the LFE channel is dropped, which follows [shadPS4](https://github.com/shadps4-emu/shadPS4) (PS4): its `DOWNMIX_CENTER` and `DOWNMIX_SURROUND` are 0.7071 and its `DOWNMIX_LFE` is 0, so the `if constexpr` that adds the LFE is compiled out. [prosper](https://github.com/mattias800/prosper) disagrees on the LFE only: it folds it into both sides with a 0.5 gain, in its AudioOut2 main bed path, which is not an Audio3d path. KytyPS5 has no Audio3d mixing at all. The absolute level is unmeasured: fpPS4 scales the front by 1 + 3/sqrt(2) instead of 1