fix(libkernel): clear the thread handle when scePthreadCreate rejects a long name as measured on a PS5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

core/libs/prx/libkernel/Pthread/src/Thread.cpp

@@ -955,7 +955,10 @@ extern "C" {
 
 int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name) {
     if (!thread || !entry) throw std::runtime_error("scePthreadCreate: null arg");
-    if (name && std::strlen(name) >= GUEST_THREAD_NAME_CAPACITY) return SCE_KERNEL_ERROR_ENAMETOOLONG;
+    if (name && std::strlen(name) >= GUEST_THREAD_NAME_CAPACITY) {
+        *thread = nullptr;
+        return SCE_KERNEL_ERROR_ENAMETOOLONG;
+    }
     if (attr && !*attr) attr = nullptr;
     auto p = std::make_unique<PthreadPrivate>(attr ? *attr : nullptr);
     bool detached = false;

core/libs/tests/GuestPthreadSecondJoin.cpp

@@ -62,9 +62,11 @@ int main() {
     Require(scePthreadCreate(&named, nullptr, Finish, nullptr, "0123456789abcdefghijklmnopqrstu") == SCE_OK);
     Require(scePthreadGetname(named, name) == SCE_OK && std::strcmp(name, "0123456789abcdefghijklmnopqrstu") == 0);
     Require(scePthreadJoin(named, nullptr) == SCE_OK);
-    Pthread untouched = reinterpret_cast<Pthread>(std::uintptr_t{0x77});
-    Require(scePthreadCreate(&untouched, nullptr, Finish, nullptr, "0123456789abcdefghijklmnopqrstuv") == SCE_KERNEL_ERROR_ENAMETOOLONG);
-    Require(untouched == reinterpret_cast<Pthread>(std::uintptr_t{0x77}));
+    for (const char* tooLong : {"0123456789abcdefghijklmnopqrstuv", "0123456789abcdefghijklmnopqrstuvw"}) {
+        Pthread cleared = reinterpret_cast<Pthread>(std::uintptr_t{0x77});
+        Require(scePthreadCreate(&cleared, nullptr, Finish, nullptr, tooLong) == SCE_KERNEL_ERROR_ENAMETOOLONG);
+        Require(cleared == nullptr);
+    }
     Pthread unnamed = nullptr;
     std::memset(name, 'x', sizeof(name));
     Require(scePthreadCreate(&unnamed, nullptr, Finish, nullptr, nullptr) == SCE_OK);

docs/dev/debt/libkernel.md

@@ -190,7 +190,7 @@
 - [sigaltstack](../../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records each thread's alternate signal stack and reports it back, following FreeBSD's `kern_sigaltstack` (`EINVAL` for flags other than `SS_DISABLE`, `ENOMEM` below `MINSIGSTKSZ` 2048), but handlers never run on it, so `SS_ONSTACK` and `EPERM` are never reported
 - [sigpending](../../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) reports the intersection of the process-wide pending set of `sigtimedwait` and the blocked mask of `_sigprocmask`, as FreeBSD's `sys_sigpending` intersects the pending list with `td_sigmask`, and returns -1 with `EFAULT` for a null set. PPSA99420 imports it
 - [scePthreadJoin, scePthreadCreate](../../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - a joined thread is kept until `scePthreadCreate` publishes over the same handle storage, so one host object and its handle are retained per distinct guest storage location for the life of the process; a guest that joins into a fresh location every time keeps every one. A handle copied to a second storage and joined there retains the object under the original storage key, so a later `scePthreadCreate` over the original storage releases an object the copy still points at, and the copy reads freed memory. A detached thread is released when it ends, so joining it after it ended reads freed memory. A null entry crashed the payload and throws. Joining the calling thread returns `EDEADLK` and a zeroed attribute creates a default thread, as on the console. PPSA34015 once stopped with `Joining guest thread: Access is denied` before the join and detach checks followed the console
-- [scePthreadCreate and scePthreadDetach, measured](../../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - as in a payload process on a PS5 (firmware 13.60): a name of 32 bytes or more returns `ENAMETOOLONG` (`0x8002003f`) without creating a thread, 31 is kept whole, and a null name gives an empty one; detaching a joined thread returns `ESRCH`; detaching a thread another thread is joining, or a detached one, returns `EINVAL`, as does joining a detached one; a second concurrent join returns `EOPNOTSUPP`; a null thread returns `EINVAL` from both; a null handle pointer to `scePthreadCreate` faulted and throws here. Whether the handle is written before the name check was not measured; here it is left alone
+- [scePthreadCreate and scePthreadDetach, measured](../../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - as in a payload process on a PS5 (firmware 13.60): a name of 32 bytes or more returns `ENAMETOOLONG` (`0x8002003f`) without creating a thread and sets the handle to 0 (32 and 33 tried), 31 is kept whole, and a null name gives an empty one; detaching a joined thread returns `ESRCH`; detaching a thread another thread is joining, or a detached one, returns `EINVAL`, as does joining a detached one; a second concurrent join returns `EOPNOTSUPP`; a null thread returns `EINVAL` from both; a null handle pointer to `scePthreadCreate` faulted and throws here
 - [Thread cancellation](../../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) follows FreeBSD libthr (states and types 0/1 and 0/2, `PTHREAD_CANCELED` 1, the `_pthread_cleanup_info` node layout), not measured on the console. A request is acted on only in `scePthreadTestcancel`, the condition variable, `sem` and `scePthreadSem` waits, `scePthreadJoin` and the sleeps, and their POSIX names; file and socket calls are not cancellation points. An enabled cancellation request wakes POSIX `sleep` and exits the thread. `usleep`, `sceKernelUsleep`, `nanosleep`, `_nanosleep`, `sceKernelNanosleep` and `sceKernelSleep` check cancellation only before and after the host sleep, so an enabled request during those waits is delivered when the sleep ends. The asynchronous type acts like the deferred one, except at the two set calls.
 - [cpuset_setaffinity](../../../core/libs/prx/libkernel/Pthread/Posix/Thread.cpp) (libkernel) - the calling thread, another live guest thread selected by its guest TID, or the calling process is handled; other levels (ROOT/CPUSET) and other scopes (CPUSET/JAIL) throw as unmodelled, invalid levels or scopes return `EINVAL` and unknown thread or process IDs return `ESRCH`, following [FreeBSD 11.4](https://github.com/freebsd/freebsd-src/blob/d106002967eff712894ab8b5337bbbff106ce5e0/sys/kern/kern_cpuset.c#L1281-L1286) thread selection. Bits outside `sceKernelGetAvailableCpumask` are dropped, and a mask with none of those bits fails with `EDEADLK`, as FreeBSD does when the new set has no CPU in common with the parent set. Guest-TID lookup and the update hold the thread registry lock so a join cannot release the target during access. Affinity is recorded for guest CPU reporting and child inheritance, not applied to host scheduling; PS5 behaviour has not been measured
 - [cpuset_getaffinity](../../../core/libs/prx/libkernel/Pthread/Posix/Thread.cpp) (libkernel) - the calling thread, another live guest thread selected by its guest TID, or the calling process is handled; other levels (ROOT/CPUSET) and other scopes (CPUSET/JAIL) throw as unmodelled, invalid levels or scopes return `EINVAL` and unknown thread or process IDs return `ESRCH`, following [FreeBSD 11.4](https://github.com/freebsd/freebsd-src/blob/d106002967eff712894ab8b5337bbbff106ce5e0/sys/kern/kern_cpuset.c#L426-L437) guest-TID selection, not measured on PS5. For the process the mask is the union of the affinities of the live guest threads, as [FreeBSD 11.4](https://github.com/freebsd/freebsd-src/blob/d106002967eff712894ab8b5337bbbff106ce5e0/sys/kern/kern_cpuset.c#L1150-L1156) ORs the mask of every thread in the process; threads the guest did not create through libkernel are not included, and the console result is unverified