Merge pull request #11846 from heditrigui/feat/libc-reallocarray

feat(libc): implement reallocarray

core/libs/prx/libc/include/Api.hpp

@@ -253,6 +253,7 @@ void* APS5_VABI malloc_nid_postfix(size_t size);
 void APS5_VABI free_nid_postfix(void* ptr);
 void* APS5_VABI realloc_nid_postfix(void* ptr, size_t newSize);
 void* APS5_VABI reallocf_nid_postfix(void* ptr, size_t newSize);
+void* APS5_VABI reallocarray_nid_postfix(void* ptr, size_t nmemb, size_t size);
 void* APS5_VABI memalign_nid_postfix(size_t alignment, size_t size);
 void* APS5_VABI aligned_alloc_nid_postfix(size_t alignment, size_t size);
 void* APS5_VABI reallocalign_nid_postfix(void* ptr, size_t size, size_t alignment);

core/libs/prx/libc/src/FileAndHeap.cpp

@@ -649,6 +649,14 @@ void* APS5_VABI reallocf_nid_postfix(void* ptr, size_t newSize) {
     return result;
 }
 
+void* APS5_VABI reallocarray_nid_postfix(void* ptr, size_t nmemb, size_t size) {
+    if (size != 0 && nmemb > std::numeric_limits<size_t>::max() / size) {
+        errno = 12;
+        return nullptr;
+    }
+    return realloc_nid_postfix(ptr, nmemb * size);
+}
+
 void* APS5_VABI memalign_nid_postfix(size_t alignment, size_t size) {
     if (!ApplicationHeapDefault_nid_no_patch()) {
         if (!validAlignment(alignment)) {

core/libs/tests/ApplicationHeap.cpp

@@ -333,6 +333,15 @@ int main(int argc, char** argv) {
         empty = ApplicationHeapReallocate_nid_no_patch(nullptr, 0);
         require(empty != nullptr);
         ApplicationHeapFree_nid_no_patch(empty);
+        pointer = static_cast<unsigned char*>(reallocarray_nid_postfix(nullptr, 4, 10));
+        require(pointer != nullptr && ApplicationHeapUsableSize_nid_no_patch(pointer) == 40);
+        std::memset(pointer, 0x5a, 40);
+        pointer = static_cast<unsigned char*>(reallocarray_nid_postfix(pointer, 5, 20));
+        require(pointer != nullptr && ApplicationHeapUsableSize_nid_no_patch(pointer) == 100);
+        for (unsigned i = 0; i < 40; ++i) require(pointer[i] == 0x5a);
+        *__error_nid_postfix() = 0;
+        require(reallocarray_nid_postfix(pointer, 2, SIZE_MAX) == nullptr && *__error_nid_postfix() == 12);
+        ApplicationHeapFree_nid_no_patch(pointer);
         for (std::size_t alignment : {16, 64, 4096}) {
             auto* aligned = ApplicationHeapAlign_nid_no_patch(alignment, 37);
             require(reinterpret_cast<std::uintptr_t>(aligned) % alignment == 0);

docs/dev/debt/libc.md

@@ -41,7 +41,7 @@
 - [sceLibcPafMspaceCreate, Destroy, Malloc, Memalign, PosixMemalign, Free, Calloc, Realloc, Reallocalign, MallocStats and MallocStatsFast](../../../core/libs/prx/libc/src/Mspace.cpp) - PAF exports delegate to `sceLibcMspace` without malloc hooks; `Memalign` and `PosixMemalign` forward to their `Mspace` counterparts with owner ignored and are unverified on console. `Create`'s options block (version, type 1 or 2 at +2, field at +0x10) throws when non-null as its meaning is unknown. Block footers are not modelled, so owners are ignored: on PS5 firmware 13.60 mismatched owners in `Reallocalign` or in-place `Realloc` kill the process and `MallocStats` returns 1 if non-heap-owned blocks exist, which is not reproduced. `MallocUsableSize`, `Trim`, `IsHeapEmpty`, `CheckMemoryBounds`, `ReportMemoryBlocks` and `GetFooterValue` are not implemented.
 - [malloc_usable_size](../../../core/libs/prx/libc/src/ApplicationHeap.cpp) (libc) - with the default allocator it returns the size the block was requested with (the range registered for it), not the rounded-up chunk size Sony's allocator reports; both satisfy "at least the requested size", and bytes past the request would not be write-watched. With an application allocator it calls entry 9 of the replacement table (`malloc_usable_size` in KytyPS5's `MallocReplace` order) and throws when the table leaves it null, like `malloc_stats_fast`
 - [malloc_stats_fast](../../../core/libs/prx/libc/src/GuestHeap.cpp) (libc) - with the default allocator it fills the 0x28-byte managed-size block as `sceLibcMspaceMallocStatsFast` does, from the default heap's own accounting, not Sony's allocator: the system sizes count the 1 MiB spans of its small-block classes and its committed large blocks, cached free ones included, and the in-use sizes count each handed-out block at its size-class or 64 KiB-granule size, header included. A block size below 0x28 or a null pointer returns 22, as in the mspace version; the version field is not checked. Ship of Harkinian (PPSA99320) calls it about 10 s after start-up.
-- [malloc, calloc, realloc, memalign, aligned_alloc, reallocalign and posix_memalign](../../../core/libs/prx/libc/src/FileAndHeap.cpp) (libc) - the console's own heap was measured in a payload process on firmware 13.60, not in a title. A heap a title replaces was not measured and keeps the FreeBSD jemalloc behaviour: `ENOMEM` on failure, `EINVAL` for an alignment that is zero or not a power of two, and `realloc(ptr, 0)` frees the block. With the console's heap, failures set `EINVAL`, and `errno` is left alone for a size above `SIZE_MAX - 0x1008f` (`SIZE_MAX - (A + 0x100af)` for an alignment over 32 rounded up to A), a `calloc` overflow or an alignment of 2^63 or more. Alignments round up to a power of two, and `realloc(ptr, 0)` keeps the block with 0 usable bytes.
+- [malloc, calloc, realloc, reallocarray, memalign, aligned_alloc, reallocalign and posix_memalign](../../../core/libs/prx/libc/src/FileAndHeap.cpp) (libc) - the console's own heap was measured in a payload process on firmware 13.60, not in a title. A heap a title replaces was not measured and keeps the FreeBSD jemalloc behaviour: `ENOMEM` on failure, `EINVAL` for an alignment that is zero or not a power of two, and `realloc(ptr, 0)` frees the block. With the console's heap, failures set `EINVAL`, and `errno` is left alone for a size above `SIZE_MAX - 0x1008f` (`SIZE_MAX - (A + 0x100af)` for an alignment over 32 rounded up to A), a `calloc` overflow or an alignment of 2^63 or more. Alignments round up to a power of two, and `realloc(ptr, 0)` keeps the block with 0 usable bytes.
 - [reallocalign](../../../core/libs/prx/libc/src/FileAndHeap.cpp) (libc) - `reallocalign(ptr, 0, alignment)` for a block without that alignment throws, where the console's heap in a payload process on PS5 firmware 13.60 returned a different block with that alignment and 0 usable bytes, `errno` untouched (a 64-aligned block given 256); whether it frees the old block is not known. There, as here, `reallocalign(ptr, 0, 16)` on a 256-aligned block returned it unchanged, `reallocalign(NULL, 0, 256)` a 256-aligned block with 0 usable bytes, and alignment 3 a moved block that kept its bytes
 - [reallocf](../../../core/libs/prx/libc/src/FileAndHeap.cpp) (libSceLibcInternal, implemented in libc) - follows FreeBSD 11 `lib/libc/stdlib/reallocf.c` on top of this libc's `realloc`, as a payload process on a PS5 (firmware 13.60) does with the default heap (null and 0, size 0, 2^40 with `errno` 22 and the block freed, `SIZE_MAX` with `errno` untouched): size 0 does what `realloc` does here, keeping the block with 0 usable bytes with the default heap and freeing it and returning null with an application allocator, so the block is not freed twice; when `realloc` fails, the block is freed through the same allocator and null is returned with the `errno` that `realloc` left. Other failures (a commit failure that is not out of memory, an unknown pointer) throw, as they do for `realloc`
 - [sceLibcMspaceReallocalign](../../../core/libs/prx/libc/src/Mspace.cpp) (libSceLibcInternal, implemented in libc) takes `(void* heap, void* block, size_t alignment, size_t size)`, the order of the console's `libSceLibcInternal.sprx` (SDK 4.5x build of `J03950001`, 0x13880, whose internal routine at 0x12390 treats the third argument as the alignment and, for a null block, passes it to the routine `sceLibcMspaceMemalign` uses). Measured on a PS5 (firmware 13.60) from a homebrew payload: a null block with alignment 4096 and size 100 returns a 4096-aligned block; an alignment of 0 or 3 succeeds and 100 gives a 128-aligned block, as the routine raises it to at least 32 and to the next power of two; an alignment of 2^63 or more returns null, keeps the block and leaves `errno` alone; and a size of 0 returns a block (the same one when it is already suitable) instead of freeing it. A block that is aligned and large enough is kept where it is, without giving back the bytes past the new size as the console's in-place resize does