A private collect read from the allocation map makes no query, so it
can't see memory that code bypassing WindowsMappings changed inside a
recorded allocation. A page decommitted in place there is the case it
can't catch: GetWriteWatch still succeeds over it, so the collect
reports no writes where a query would have refused the range.
With APS5_VERIFY_DIRECT_COLLECT set, each such collect also queries its
span's regions, and Collect throws, naming the span and the switch, when
GetWriteWatch succeeded over memory that is no longer committed
private. ENVIRONMENT.md lists the switch.
windows_write_watch_verify_direct (the same test binary with
--verify-direct, the switch set) collects a recorded private allocation,
checks its write is still reported, decommits one page in place and
requires the next collect not to succeed. Without the check the collect
succeeds.
Adapts fable-void's 2760ad808.
Collecting private guest writes from the allocation map trusts the
extent the mappings recorded and calls GetWriteWatch without a query.
When memory in that extent was released by code that bypasses
WindowsMappings, the call fails and Collect threw "collect
private guest writes". Before the map, the query found the memory no
longer committed private and Collect returned false, which its callers
already handle.
A failed GetWriteWatch now queries the region at the cursor. If it is
no longer committed private memory, Collect returns false as before;
otherwise the failure is still fatal.
windows_write_watch releases a collected private allocation with
VirtualFree behind the mappings, then collects it again: it must return
false without throwing. Without the query it throws.
Adapts fable-void's 2760ad808.
Mapping shared direct memory while shared maps are imported placed one
view per 16 KiB page: a placeholder split, a map and a protect for every
page. A 1 GiB map made about 200,000 system calls, held the mapping
thread for about 0.4 s and took and dropped the mappings mutex on every
page, so threads taking write faults queued behind it.
Map now maps each run of shared pages that need a view as one view: one
placeholder split, one map at read-write-execute (the cap later guest
protections need) and one protect per run, and builds the per-page
bookkeeping under the mutex while yielding to waiting faults. The views
move the run's page references instead of copying them, so releasing the
run does not hold the mutex. Protections and write tracking change
subpages inside the view as before. A view is split only where a release
or a remap cuts it: reset unmaps the view, maps the two pieces again over
the same section offsets and restores every 4 KiB protection, so armed
write tracking and guest protections survive the split. An alias of a
page elsewhere and a per-page protection leave the view whole.
Tested: windows_write_watch gains a case that maps 1 GiB of shared
memory (65536 views before, one after), then protects one page, aliases
one page, arms write tracking on one page, releases one page and remaps
another: the run splits only at the released and remapped pages (2 and
4 views), every other page keeps its bytes and protection, and the armed
page still faults and is collected. It fails before ("a shared gibibyte
was mapped as 65536 views instead of one") and passes after.
agc_driver_committed_imports: the 1 GiB shared map and import takes
190-196 ms on the mapping thread instead of 418-427 ms, and the longest
write fault on another thread stays at 46-97 us. Built; the write watch,
committed import, guest memory, write guard, host and window import,
residency and mapping tests pass (58 tests).
Mapping shared guest memory page by page queried the address space
three times for every 16 KiB page: twice to find the placeholder the
page splits from, and once more to see whether the section page still
had to be committed, which it then committed on its own. For a range of
fresh direct memory that is seven system calls a page, and with shared
imports attached every direct mapping takes this path on the guest
thread that maps it, under the mapping lock.
A mapping that goes page by page now commits its section range once
through one window, and splits each page off the placeholder the
previous page left, whose extent it already knows; only a page after a
resident run or a demotion looks the placeholder up again. A page now
costs a placeholder split, the view and its protection.
Tested: built; windows_write_watch gains a case that maps 256 shared
pages and counts the address-space queries the mapping makes, then
stores to and reads back every page. With the per-page lookups it fails
(775 queries for 256 pages) and with this change it passes.
agc_driver_committed_imports, agc_write_watch_coverage,
agc_private_collect, agc_write_guards, agc_driver_dcc_keys_first_write,
agc_driver_residency_dirty_mirrors, agc_driver_residency_driver,
guest_memory and the host import, window import and alias store tests
pass.
WindowsMappings keys each physical page's state (moved into the section,
saved bytes of an unmapped resident page) by the section handle value.
sceKernelReleaseDirectMemory frees the backing before it unmaps the views,
so ForgetSection ran while those views still existed and kept their slots.
After the views went, the slots stayed under a handle value that Windows
hands out again, and the next direct allocation whose section got that
value started with the old page state: its first mapping went to the
section instead of private memory, or received the old allocation's bytes.
Views now share ownership of their slot, ForgetSection drops every slot
of the section at once, and JIT shared memory forgets its section before
closing it too.
Yunus Shaikh