This reverts commit 780b33aea3, restoring the 13824 MiB direct memory pool.
With the pool at 12288 MiB, Astro Bot (PPSA21564) can no longer load its
level: DirectMemoryAllocator asserts "Can't allocate direct memory" for a
64 MB VideoPlayerTexture with 12228 of 12288 MB in use, then the process
faults about 26 s in (2 of 2 runs, RTX 4090, Linux, main cccdaa2734).
The same tree with only this merge reverted runs the full 150 s with no
direct memory assert (2 of 2 runs).
A resident direct memory mapping created a SharedPage, a Physical, a
physical map node and an aliases buffer for every 16 KiB page, about
18 MiB of heap per GiB mapped, although a page mapped once holds no
state there. Resident pages now keep only their View, with pins and the
compare-when-shared mark in it; an index from (section, offset) to the
resident run finds a live resident mapping of a physical page, and the
shared state is created when a run is demoted, or when it is evicted
with data or a compare mark. ForgetSection drops the index entries of
the section's runs.
guest_memory on Windows 11 26200 (Ryzen 5 5600X): private memory per
GiB mapped at map time drops from 30.9 MiB to 13.4 MiB.
Co-Authored-By: Claude <[email protected]>
PS5_PayloadSDK measured in a title (firmware 12.09) that
sceKernelMapDirectMemory with read-execute or read-write-execute returns
0x80020016, and that sceKernelMprotect then grants execute to a
read-write mapping. The map granted execute directly.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The same-range/same-protection fast path in RemapFixedIntoRegistered
returned without touching host pages, but DoMprotect/DoMtypeprotect
change host protection without updating the recorded nativeProtection,
so a stale record could leave guest pages with the wrong host
permissions. The shortcut now reissues the per-block backing map with
records skipped, keeping the bookkeeping optimization while reapplying
the requested host protection.
CreateDirectMemoryBacking registered g_physBlocks before building the
per-page map, so a bad_alloc during page construction left an orphaned
physical block that poisoned later allocations; the block now publishes
after the page map is built.
CheckFixedRemapReprotectsAfterMprotect covers both shortcut directions
with stale records and verifies the host protection via VirtualQuery /
proc maps, and the existing remap test gains the current uint64 Munmap
call.
CheckMlock reads VmLck and expects ENOMEM from mlock of reserved and unmapped ranges. ThreadSanitizer, like AddressSanitizer, replaces mlock and munlock with interceptors that return 0 without locking anything, so skip the same checks that ANYPS5_ENABLE_ASAN already skips. The guest arena range check stays as is: a ThreadSanitizer build keeps the default range.
Prospero Win measured on firmware 12.02 that the PS5 kernel accepts
MAP_EXCL but ignores it: MAP_FIXED | MAP_EXCL over an existing mapping
replaces it without error. Fixed anonymous mmap now does the same instead
of FreeBSD's ENOMEM. MAP_EXCL without MAP_FIXED stays EINVAL.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
CheckRemappedDirectMemoryIsChargedOnce compared the system-wide commit
total (K32GetPerformanceInfo) before and after mapping, unmapping and
remapping 256 MiB of direct memory, and required the growth to stay under
384 MiB in one of three attempts. Every other process on the host changes
that total, so guest_memory failed at GuestMemory.cpp:2164 whenever a
parallel test or a compiler committed and freed more than 128 MiB during
each attempt.
Measure the private and shared commit of the test process instead
(PrivateUsage and SharedCommitUsage of PROCESS_MEMORY_COUNTERS_EX2), which
covers both the private resident run and the direct memory section. The
remap grows it by 265 MiB on every run, so the attempts are dropped.
The unchecked release passed its whole range to ForgetDirectMemory,
which throws on any page that is not allocated, so releasing a range
that is partly or wholly free aborted the process. Release only the
allocated runs, as shadPS4's unchecked free does; the checked variant
still returns ENOENT for them.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
munmap failed with EINVAL for an address that is not page aligned.
FreeBSD 11's sys_munmap rejects only a zero length and an address wrap;
it moves the address down to its page and grows the length by the same
offset before rounding it up, so the pages the range touches are
unmapped.
Resolve ImportMappedRanges onto the lazy lease from #7465: ranges of
0x10000 bytes or less still take no lease, the first larger range takes
the narrow lease over the mapped runs instead of a full one, and an import
that would replace another (with the GPU mutex held) re-acquires a full
lease. The adopting path keeps its full lease for refreshImports.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Measured on firmware 13.60: an address or length aligned to 4 KiB but
not to 16 KiB returns EINVAL.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Measured on firmware 13.60: the released range stays mapped with its
protection and contents, and a fixed no-overwrite map over it fails
with ENOMEM; a null or unaligned address, an unaligned length or a
range that wraps returns EINVAL, checked before a zero length, which
returns 0, as does a range with nothing mapped.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Measured on firmware 13.60: submits take 1 to 128 requests at priority
1 to 3, the id lists of poll, wait, cancel and delete take 1 to 128 ids,
wait takes mode 1 or 2, and sceKernelAioSetParam takes a window and a
delayed count of 0 to 128, a split flag of 0 or 1 and, with split on,
split and chunk sizes of 1 to 0x1000000; everything else returns EINVAL
without writing. sceKernelAioInitializeParam writes the console's three
default scheduling structs.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A clean resident run was made read-only so that the next collect
could skip GetWriteWatch, and the first guest store into it was
resumed on the guest thread. The exception frame Windows builds for
that fault overwrites the System V red zone below rsp (#361), which
is what #5596 removed for singly mapped direct memory. Resident runs
are now never armed and are collected with GetWriteWatch only.
Refs #9136
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Measured on firmware 13.60: a start whose rounding to 16 KiB overflows
finds nothing, and of two equal free ranges the lowest is returned.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
harisibrar1