feat(libkernel): guest memory protection and VirtualQuery on macOS hosts

- mprotect: the guest rounds a range to its 16 KiB pages. A Mach-O image
  is loaded at a multiple of the host page size only (4 KiB under
  Rosetta), so the pages are counted from where the ELF address 0 of the
  image holding the range was loaded, and a rounded edge smaller than a
  guest page that lies in the gap between two segments, where no host page
  exists, is left out. The write watch is told the new protection.
- sceKernelVirtualQuery reads the region with mach_vm_region.
- sceKernelMunlockall unlocks the ranges libkernel locked, one by one:
  macOS has no munlockall, which fails there with ENOSYS and would
  overwrite errno.
- The stack protect test reads page protections with mach_vm_region.

This is mugurc's f66a4df741 from his macOS series, without the mlock
check that #10889 adds.

Co-authored-by: mugurc <[email protected]>

core/libs/prx/libc/include/GuestAllocations.hpp

@@ -62,6 +62,8 @@ void GuestAllocationsAdd_nid_postfix(void* mutation, void* pointer, std::size_t
 void GuestAllocationsRequireUnpinned_nid_postfix(void* mutation, const void* pointer, std::size_t bytes);
 void GuestAllocationsRequireAvailable_nid_postfix(void* mutation, const void* pointer, std::size_t bytes);
 bool GuestAllocationsCovers_nid_postfix(void* mutation, const void* pointer, std::size_t bytes);
+void GuestAllocationsTrimToRegistered_nid_postfix(void* mutation, const void** pointer, std::size_t* bytes);
+void GuestAllocationsGuestPages_nid_postfix(std::uintptr_t address, std::size_t bytes, std::size_t page, std::uintptr_t* first, std::uintptr_t* end);
 bool GuestAllocationsOverlaps_nid_postfix(void* mutation, const void* pointer, std::size_t bytes);
 bool GuestAllocationsForRegistered_nid_postfix(void* mutation, const void* pointer, std::size_t bytes, const std::function<void(const void*, std::size_t)>& visit);
 Range GuestAllocationsFind_nid_postfix(void* mutation, const void* pointer);
@@ -101,6 +103,7 @@ public:
     void RequireUnpinned(const void* pointer, std::size_t bytes) const { GuestAllocationsRequireUnpinned_nid_postfix(handle, pointer, bytes); }
     void RequireAvailable(const void* pointer, std::size_t bytes) const { GuestAllocationsRequireAvailable_nid_postfix(handle, pointer, bytes); }
     bool Covers(const void* pointer, std::size_t bytes) const { return GuestAllocationsCovers_nid_postfix(handle, pointer, bytes); }
+    void TrimToRegistered(const void*& pointer, std::size_t& bytes) const { GuestAllocationsTrimToRegistered_nid_postfix(handle, &pointer, &bytes); }
     bool Overlaps(const void* pointer, std::size_t bytes) const { return GuestAllocationsOverlaps_nid_postfix(handle, pointer, bytes); }
     bool ForRegistered(const void* pointer, std::size_t bytes, const std::function<void(const void*, std::size_t)>& visit) const { return GuestAllocationsForRegistered_nid_postfix(handle, pointer, bytes, visit); }
     Range Find(const void* pointer) const { return GuestAllocationsFind_nid_postfix(handle, pointer); }

core/libs/prx/libc/src/GuestAllocations.cpp

@@ -15,6 +15,11 @@
 #define NOMINMAX
 #endif
 #include <windows.h>
+#elif defined(__APPLE__)
+#include "prx/libc/include/specifics/linux/ElfTypes.hpp"
+#include <mach-o/dyld.h>
+#include <mach-o/loader.h>
+#include <unistd.h>
 #else
 #include <cerrno>
 #include <cstring>
@@ -524,6 +529,54 @@ bool GuestAllocationsForRegistered_nid_postfix(void*, const void* pointer, std::
     return any;
 }
 
+void GuestAllocationsGuestPages_nid_postfix(std::uintptr_t address, std::size_t bytes, std::size_t page, std::uintptr_t* first, std::uintptr_t* end) {
+    std::uintptr_t bias = 0;
+#ifdef __APPLE__
+    struct Search {
+        std::uintptr_t address;
+        std::uintptr_t bias;
+    } search{address, 0};
+    dl_iterate_phdr([](dl_phdr_info* image, std::size_t, void* data) {
+        auto& found = *static_cast<Search*>(data);
+        for (unsigned index = 0; index < image->dlpi_phnum; ++index) {
+            const auto& header = image->dlpi_phdr[index];
+            const auto start = static_cast<std::uintptr_t>(image->dlpi_addr + header.p_vaddr);
+            if (header.p_type == PT_LOAD && found.address >= start && found.address - start < header.p_memsz) {
+                found.bias = static_cast<std::uintptr_t>(image->dlpi_addr);
+                return 1;
+            }
+        }
+        return 0;
+    }, &search);
+    bias = search.bias;
+#endif
+    const auto mask = static_cast<std::uintptr_t>(page - 1);
+    *first = bias + ((address - bias) & ~mask);
+    *end = bias + ((address + bytes - bias + mask) & ~mask);
+}
+
+void GuestAllocationsTrimToRegistered_nid_postfix(void*, const void** pointer, std::size_t* bytes) {
+#ifdef __APPLE__
+    constexpr std::uint64_t guestPage = 0x4000;
+    auto start = reinterpret_cast<std::uintptr_t>(*pointer);
+    auto end = start + *bytes;
+    const auto& ranges = registry().ranges;
+    const auto following = ranges.upper_bound(start);
+    const bool startCovered = following != ranges.begin() && std::prev(following)->first + std::prev(following)->second->bytes > start;
+    if (!startCovered && following != ranges.end() && following->first < end && following->first - start < guestPage) start = following->first;
+    const auto beyond = ranges.lower_bound(end);
+    if (beyond != ranges.begin()) {
+        const auto finish = std::prev(beyond)->first + std::prev(beyond)->second->bytes;
+        if (finish < end && finish > start && end - finish < guestPage) end = finish;
+    }
+    *pointer = reinterpret_cast<const void*>(start);
+    *bytes = end - start;
+#else
+    static_cast<void>(pointer);
+    static_cast<void>(bytes);
+#endif
+}
+
 bool GuestAllocationsCovers_nid_postfix(void*, const void* pointer, std::size_t bytes) {
     const auto address = reinterpret_cast<std::uintptr_t>(pointer);
     require(address != 0 && bytes != 0 && bytes <= std::numeric_limits<std::uint64_t>::max() - address, "invalid guest allocation range");

core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp

@@ -1325,10 +1325,13 @@ int DoMprotect(const void* addr, size_t len, int prot) {
     constexpr auto pageMask = static_cast<std::uintptr_t>(PS5_PAGE_SIZE - 1);
     const auto limit = std::numeric_limits<std::uintptr_t>::max();
     if (address == 0 || len == 0 || len > limit - address || address + len > limit - pageMask) throw std::invalid_argument("Invalid guest memory protection range");
-    const auto first = address & ~pageMask;
-    const auto end = (address + len + pageMask) & ~pageMask;
-    const auto bytes = static_cast<std::size_t>(end - first);
-    const auto* pointer = reinterpret_cast<const void*>(first);
+    auto first = address & ~pageMask;
+    auto end = (address + len + pageMask) & ~pageMask;
+#ifdef __APPLE__
+    GuestAllocations::GuestAllocationsGuestPages_nid_postfix(address, len, PS5_PAGE_SIZE, &first, &end);
+#endif
+    auto bytes = static_cast<std::size_t>(end - first);
+    const void* pointer = reinterpret_cast<const void*>(first);
     const auto nativeProtection = LinuxProtFromSce(prot);
     if (GuestThreadStackHostRangeOverlaps(address, address + len))
         throw std::system_error(EACCES, std::generic_category(), "mprotect overlaps host-only guest thread stack pages");
@@ -1395,14 +1398,18 @@ int DoMprotect(const void* addr, size_t len, int prot) {
         if (!image.guestModule) throw std::invalid_argument("Memory protection of a foreign image is not supported");
         mutation.RegisterImage(reinterpret_cast<const void*>(image.base));
     }
+    mutation.TrimToRegistered(pointer, bytes);
 #endif
     std::vector<std::pair<const void*, std::size_t>> spans;
     mutation.ForRegistered(pointer, bytes, [&](const void* span, std::size_t spanBytes) { spans.emplace_back(span, spanBytes); });
-    if (spans.empty() || reinterpret_cast<std::uintptr_t>(spans.front().first) != first) return SCE_KERNEL_ERROR_EINVAL;
+    if (spans.empty() || spans.front().first != pointer) return SCE_KERNEL_ERROR_EINVAL;
     for (const auto& [span, spanBytes] : spans) mutation.RequireUnpinned(span, spanBytes);
     for (const auto& [span, spanBytes] : spans) {
         mutation.Protect(span, spanBytes, (prot & 3) != 0, (prot & 2) != 0, (prot & GuestProtGpuReadWrite) != 0, [&] {
             if (mprotect(const_cast<void*>(span), spanBytes, nativeProtection) != 0) throw std::system_error(errno, std::generic_category(), "mprotect failed");
+#ifdef __APPLE__
+            GuestWriteWatch::GuestWriteWatchProtectionChanged_nid_postfix(span, spanBytes, nativeProtection);
+#endif
         });
         RecordProtection(span, spanBytes, prot);
     }

core/libs/prx/libkernel/DirectMemory/Export.cpp

@@ -389,6 +389,17 @@ int APS5_VABI sceKernelVirtualQuery(const void* addr, int flags, VirtualQueryInf
  info->end = info->start + host.RegionSize;
  const bool writable = (host.Protect & (PAGE_READWRITE | PAGE_EXECUTE_READWRITE | PAGE_WRITECOPY | PAGE_EXECUTE_WRITECOPY)) != 0;
  const bool executable = (host.Protect & (PAGE_EXECUTE | PAGE_EXECUTE_READ | PAGE_EXECUTE_READWRITE | PAGE_EXECUTE_WRITECOPY)) != 0;
+ #elif defined(__APPLE__)
+ mach_vm_address_t region = address;
+ mach_vm_size_t regionSize = 0;
+ vm_region_basic_info_data_64_t host{};
+ mach_msg_type_number_t count = VM_REGION_BASIC_INFO_COUNT_64;
+ mach_port_t object = MACH_PORT_NULL;
+ if (mach_vm_region(mach_task_self(), &region, &regionSize, VM_REGION_BASIC_INFO_64, reinterpret_cast<vm_region_info_t>(&host), &count, &object) != KERN_SUCCESS || region > address || (host.protection & VM_PROT_READ) == 0) return SCE_KERNEL_ERROR_EACCES;
+ info->start = region;
+ info->end = region + regionSize;
+ const bool writable = (host.protection & VM_PROT_WRITE) != 0;
+ const bool executable = (host.protection & VM_PROT_EXECUTE) != 0;
  #else
  std::ifstream maps("/proc/self/maps");
  std::string line;
@@ -765,7 +776,7 @@ int APS5_VABI sceKernelMunlock_nid_postfix(void* address, std::uint64_t length)
 }
 
 int APS5_VABI sceKernelMunlockall_nid_postfix() {
-#ifndef _WIN32
+#if !defined(_WIN32) && !defined(__APPLE__)
     ::munlockall();
 #endif
     std::map<uintptr_t, size_t> ranges;
@@ -791,6 +802,10 @@ int APS5_VABI sceKernelMunlockall_nid_postfix() {
             cursor = next;
         }
     }
+#elif defined(__APPLE__)
+    const int saved = errno;
+    for (const auto& [start, len] : ranges) ::munlock(reinterpret_cast<void*>(start), len);
+    errno = saved;
 #endif
     return 0;
 }

core/libs/tests/GuestKernelStackProtect.cpp

@@ -13,6 +13,10 @@
 #define NOMINMAX
 #endif
 #include <windows.h>
+#elif defined(__APPLE__)
+#include <mach/mach.h>
+#include <mach/mach_vm.h>
+#include <sys/mman.h>
 #else
 #include <sys/mman.h>
 #include <sys/resource.h>
@@ -39,6 +43,16 @@ static PageState QueryPage(std::uintptr_t address) {
     Require(VirtualQuery(reinterpret_cast<const void*>(address), &memory, sizeof(memory)) == sizeof(memory));
     const bool mapped = memory.State == MEM_COMMIT;
     return {mapped, mapped && (memory.Protect & (PAGE_READWRITE | PAGE_EXECUTE_READWRITE)) != 0, (static_cast<std::uint64_t>(memory.State) << 32) | memory.Protect};
+#elif defined(__APPLE__)
+    mach_vm_address_t region = address;
+    mach_vm_size_t size = 0;
+    vm_region_basic_info_data_64_t info{};
+    mach_msg_type_number_t count = VM_REGION_BASIC_INFO_COUNT_64;
+    mach_port_t object = MACH_PORT_NULL;
+    if (mach_vm_region(mach_task_self(), &region, &size, VM_REGION_BASIC_INFO_64, reinterpret_cast<vm_region_info_t>(&info), &count, &object) != KERN_SUCCESS || region > address) return {false, false, 0};
+    const bool readable = (info.protection & VM_PROT_READ) != 0;
+    const bool writable = (info.protection & VM_PROT_WRITE) != 0;
+    return {true, readable && writable, 1ull | (readable ? 2ull : 0) | (writable ? 4ull : 0) | ((info.protection & VM_PROT_EXECUTE) != 0 ? 8ull : 0)};
 #else
     std::FILE* maps = std::fopen("/proc/self/maps", "r");
     Require(maps != nullptr);

docs/dev/debt/libkernel.md

@@ -259,6 +259,7 @@
 - [connect to AF_UNSPEC](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) dissolves a datagram association as FreeBSD 11.4's `udp_disconnect()` does, which resets the local address to the wildcard address and keeps the port. On Linux and on Windows a socket bound to an explicit port keeps its local address as well (`127.0.0.1:<port>` stays `127.0.0.1:<port>`), because a bound socket cannot be bound again, and `getsockname` reports that host address. On Windows a `sendto` to a non-loopback address from such a socket then fails with `ENETUNREACH` (Winsock 10051, measured on Windows 10), where FreeBSD sends it from the wildcard address. One bound to port 0 is bound again to the wildcard address with the port it had
 - [connect](../../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) - `connect` to port 0 on an `AF_INET` or `AF_INET6` stream or datagram socket fails with `EADDRNOTAVAIL` (49), and on a connected datagram socket it disconnects first, as on a PS5 (firmware 13.60) and in FreeBSD 11.4's `soconnect()`. A connected stream socket keeps its host `EISCONN` answer. Known gap: `sendto` and `sendmsg` to port 0 on an `AF_INET` datagram socket fail with 49 (measured for an unbound socket) unless the socket is bound to a specific address and port and the destination is neither `INADDR_ANY` nor `INADDR_BROADCAST`, which is the path where `udp_output()` skips `in_pcbconnect_setup()` and FreeBSD sends to port 0. There the call returns the length without the host, as Linux's `udp_sendmsg` refuses port 0 with `EINVAL`; an `AF_INET6` socket still goes to the host. [sceNetConnect](../../../core/libs/prx/libSceNet/Export.cpp) is not changed
 - [msync](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) follows FreeBSD 11's `kern_msync` and `vm_map_sync`: the address is rounded down to the page, a range that wraps is `EINVAL`, as is `MS_ASYNC` with `MS_INVALIDATE`, and a range whose start or any page is not mapped (as `sceKernelVirtualQuery` reports it, reservations included) is `ENOMEM`; a page unmapped from inside a larger mapping still counts as mapped there, since `sceKernelVirtualQuery` keeps reporting the whole mapping. The guest has no file mappings, so a successful call writes nothing back. `MS_INVALIDATE` on a mapped range fails with `EBUSY`, as it did in a payload process on a PS5 (firmware 13.60) for anonymous memory, locked or not, also with a hole unmapped inside it; a title process, reservations, direct memory and flexible memory were not measured
+- Guest memory protection on macOS ([mprotect](../../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp), [sceKernelVirtualQuery](../../../core/libs/prx/libkernel/DirectMemory/Export.cpp)): the guest rounds a range to its 16 KiB pages, counted from where the ELF address 0 of the image holding it was loaded, which is a multiple of the host page size only (4 KiB under Rosetta). A rounded edge smaller than a guest page that lies in the gap between two segments of an image, where no host page exists, is left out of the request. `sceKernelVirtualQuery` reports the region `mach_vm_region` returns, so its extent is the host's. After a protection change the write watch is told the protection, which the host reports for a watched page as its own
 - [madvise](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) accepts `MADV_PROTECT` (10) as a no-op hint without modifying process protection. FreeBSD 11's `kern_madvise` (`sys/vm/vm_mmap.c`) handles it before the address check, ignores the address and length, and calls `kern_procctl(..., PROC_SPROTECT, PPROT_SET)`, which requires `PRIV_VM_MADV_PROTECT` and fails with `EPERM` without it. Returning 0 unconditionally assumes the calling process is privileged. Console behaviour is unverified
 - [mincore](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) follows FreeBSD 11's `sys_mincore` (`sys/vm/vm_mmap.c`), not measured on the console: the address is rounded down to the page, length 0 returns 0, a null vector over a mapped range returns `EFAULT`, an overflowing range or any unmapped page in the range (as reported by `sceKernelVirtualQuery`) returns `ENOMEM`. For valid ranges, a page reports `MINCORE_INCORE` (0x1) when `sceKernelVirtualQuery` reports it committed and 0 when it is only reserved; actual residency, referenced and modified bits are not modelled. Length 0 returns 0 when the start page is mapped and `ENOMEM` otherwise, and the range and mapping checks come before the null vector `EFAULT`, as in `kern_mincore` Java 25 LAN / DNS (PPSA99869) imports it
 - [mlockall, munlockall](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) - `mlockall` returns `EINVAL` for zero or unknown flags. Valid `MCL_CURRENT` throws because it would lock host stacks, heaps, modules and driver mappings along with guest memory; `MCL_FUTURE` throws because later mappings are not tracked for locking. FreeBSD 11.4's `sys_munlockall` (`sys/vm/vm_mmap.c`:1133-1165) checks `priv_check(td, PRIV_VM_MUNLOCK)` first (returning `EPERM` without it) and unwires the user map (`vm_map_unwire` over `vm_map_min`...`vm_map_max` with `VM_MAP_WIRE_HOLESOK`). Returning 0 unconditionally assumes the calling process is privileged. `munlockall` unlocks memory locked by `mlock` (`VirtualUnlock` on Windows, tolerating `ERROR_NOT_LOCKED` for pages never locked, and `munlockall` on Linux). XPSemu (PPSA97358) imports `mlockall` and `munlockall`; their calls in QEMU `os-posix.c` execute only when `-overcommit mem-lock=on` is enabled. The call has not been observed in a run with this implementation.