Commit Graph
160 Commits
Author SHA1 Message Date
Dean 396b282315 Merge pull request #339 from DotDebian/feat/cpu-time-clocks
feat(libkernel): support the CPU-time clocks in clock_gettime and clock_getres on Windows
2026-10-04 14:21:40 +13:00
Dean 1964aad443 Merge pull request #428 from RomainROCH/codex/fix-posix-semaphore-limits
fix(libkernel): enforce POSIX semaphore count limits
2026-10-04 14:03:06 +13:00
Dean d0bec2e448 Merge pull request #378 from DotDebian/fix/posix-file-errno
fix(libkernel): return -1 and set errno from the POSIX open, stat, unlink and rmdir
2026-10-04 14:01:57 +13:00
Dean e5676b3f0e Merge pull request #315 from RomainROCH/codex/fix-semaphore-overflow
fix(libkernel): avoid semaphore count overflow
2026-10-04 14:01:19 +13:00
Alexey 87f016b0da Merge pull request #370 from DotDebian/fix/pthread-attr-get
fix(libkernel): report the thread's priority and affinity from scePthreadAttrGet
2026-10-03 11:02:46 +03:00
Alexey e1bfb3973f Merge pull request #364 from DotDebian/feat/pthread-barrierattr
feat(libkernel): implement the pthread_barrierattr functions
2026-10-03 11:01:01 +03:00
Alexey a818beedf5 Merge pull request #341 from DotDebian/feat/kernel-cancel-sema
feat(libkernel): implement sceKernelCancelSema
2026-10-03 10:58:13 +03:00
Alexey 9f7e6499bd Merge pull request #335 from DotDebian/feat/kernel-event-accessors
feat(libkernel): implement the sceKernelGetEvent* field accessors
2026-10-03 10:50:28 +03:00
RomainROCH eae05782af fix(libkernel): enforce POSIX semaphore count limits 2026-10-03 04:03:25 +02:00
Pouare514 1c724c0228 feat(libs): Cyberpunk 2077 loading support (REDGalaxy + stubs)
# Conflicts:
#	core/libs/prx/libSceContentExport/Export.cpp
#	core/libs/prx/libScePlayGoDialog/Export.cpp
2026-10-02 23:14:04 +03:00
boykopovar 5026351b3f Merge pull request #274 from RomainROCH/codex/fix-directory-buffer-errors
fix(libkernel): report undersized Windows directory buffers
2026-10-02 22:39:37 +03:00
boykopovar 4181fbd521 Merge pull request #279 from Pouare514/fix/dreaming-sarah-boot
fix(libkernel): boot Dreaming Sarah past unwind and debug raise
2026-10-02 22:38:49 +03:00
boykopovar 11814ad660 Merge pull request #349 from DotDebian/feat/sync-on-address
feat(libkernel): implement sceKernelSyncOnAddressWait and sceKernelSyncOnAddressWake
2026-10-02 22:31:45 +03:00
boykopovar d399b4e04c Merge pull request #245 from RomainROCH/codex/fix-windows-directory-close
fix(libkernel): clear directory metadata on close
2026-10-02 22:21:30 +03:00
boykopovar 74bdce6f04 Merge pull request #232 from MichonGoddijn231849/fix/guest-read-destination
fix(libkernel): read, pread, AIO and APR write into guest memory as host writes
2026-10-02 22:19:46 +03:00
DotDebianandClaude Opus 5.5 1817d83fec fix(libkernel): return -1 and set errno from the POSIX open, stat, unlink and rmdir
open, _open, stat, unlink and rmdir forwarded the result of their sceKernel
twin, so a failure returned 0x8002xxxx instead of -1 and errno was never set
to the guest error. They now return -1 with errno set to the code carried by
the sce error, and report a null pointer as EFAULT and an empty path as
ENOENT.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:17:12 +02:00
DotDebianandClaude Opus 5.5 f1be5b8ba5 fix(libkernel): report the thread's priority and affinity from scePthreadAttrGet
scePthreadAttrGet wrote priority 700 and left the affinity of the
destination untouched, whatever the thread was created with or set to
afterwards. It now reports the values the thread holds, as
pthread_attr_get_np already does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:03:51 +02:00
DotDebianandClaude Opus 5.5 b279848c12 feat(libkernel): implement the pthread_barrierattr functions
pthread_barrier_init threw for any non-null attributes object, and no
function could create one. Add pthread_barrierattr_init, _destroy,
_getpshared and _setpshared after FreeBSD libthr, and let
pthread_barrier_init take the object.

PTHREAD_PROCESS_SHARED still throws in pthread_barrierattr_setpshared:
FreeBSD 9.0 rejects it and 11.0 accepts it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 17:08:07 +02:00
DotDebian ba512363b1 feat(libkernel): implement sceKernelSyncOnAddressWait and sceKernelSyncOnAddressWake
sceKernelSyncOnAddressWait parks the calling thread while the 32-bit word
at the address still holds the expected value, until
sceKernelSyncOnAddressWake releases it or the timeout expires.
sceKernelSyncOnAddressWake releases up to the requested number of threads
parked on the address, in arrival order.

The libc.prx bundled with SDK 9 titles imports both for its once
primitive, so those titles stopped at load with an unresolved import.

# Conflicts:
#	docs/dev/TechnicalDebt.md
2026-10-02 16:49:44 +03:00
boykopovar a4157bbeee Merge pull request #355 from DotDebian/feat/map-named-flexible-memory-internal
feat(libkernel): implement sceKernelMapNamedFlexibleMemoryInternal
2026-10-02 16:45:58 +03:00
boykopovar e00b491465 Merge pull request #286 from SP4C3B4R-8/feat/windows-mapping-hints
feat(libkernel): support mapping address hints and no-overwrite fixed mappings on Windows
2026-10-02 16:41:43 +03:00
DotDebianandClaude Opus 5.5 74fcea27f8 feat(libkernel): implement sceKernelMapNamedFlexibleMemoryInternal
libkernel did not export sceKernelMapNamedFlexibleMemoryInternal, which
the libc bundled with SDK 9 titles imports. It now forwards to
sceKernelMapNamedFlexibleMemory: the mapping counts against the flexible
budget, gets the range name and is released by sceKernelMunmap. Older
libc builds call the public function at the same sites with the same
arguments and result handling.

The known callers pass flag 0x8000, whose meaning is unknown. It is
still rejected as an unsupported mapping flag and is recorded in
TechnicalDebt.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 14:08:58 +02:00
Pouare514 f1a3d60b15 fix(libkernel): boot Dreaming Sarah past unwind and debug raise 2026-10-02 13:34:35 +02:00
DotDebianandClaude Opus 5.5 dab1110556 feat(libkernel): implement sceKernelCancelSema
sceKernelCancelSema was a throwing stub. It now releases every thread
blocked in sceKernelWaitSema with SCE_KERNEL_ERROR_ECANCELED without
consuming tokens, sets the count to setCount (the initial count when
setCount is negative) and reports how many threads were waiting. A
setCount above the maximum returns SCE_KERNEL_ERROR_EINVAL and changes
nothing.

Waiters are counted separately from waiterCount, which is only
decremented when a waiter leaves, so two cancels in a row do not report
the same thread twice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 11:57:00 +02:00
DotDebianandClaude Opus 5.5 d6cca762f4 feat(libkernel): support the CPU-time clocks in clock_gettime and clock_getres on Windows
CLOCK_THREAD_CPUTIME_ID (14) and CLOCK_PROCESS_CPUTIME_ID (15) threw
"unsupported clock_id" on Windows while the Linux branch already maps
them to the native clocks. clock_gettime now returns the user + kernel
time from GetThreadTimes / GetProcessTimes, and clock_getres reports the
clock interrupt period from GetSystemTimeAdjustment, which is the step
those counters actually advance by.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 11:40:22 +02:00
DotDebianandClaude Opus 5.5 40d63b1b4c feat(libkernel): implement the sceKernelGetEvent* field accessors
sceKernelGetEventId, sceKernelGetEventFilter, sceKernelGetEventFflags,
sceKernelGetEventData and sceKernelGetEventUserData return the matching
field of the KernelEvent filled by sceKernelWaitEqueue and throw on a
null event. sceKernelGetEventError still throws NotImplemented.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 10:46:15 +02:00
RomainROCH 74586bab57 fix(libkernel): avoid semaphore count overflow 2026-10-02 06:19:39 +02:00
SP4C3B4R-8andClaude Opus 5.5 4099ffb7d3 feat(libkernel): accept MAP_NO_OVERWRITE fixed mappings on Windows
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Bn7CrhG7pTPR6DT13tN6Me
2026-10-01 23:44:31 +02:00
SP4C3B4R-8andClaude Opus 5.5 3fc3b993da feat(libkernel): honour non-fixed mapping address hints on Windows
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Bn7CrhG7pTPR6DT13tN6Me
2026-10-01 23:44:30 +02:00
RomainROCH 021eb65b51 fix(libkernel): report undersized Windows directory buffers 2026-10-01 23:24:41 +02:00
MichonandClaude Opus 5.5 04691c0797 feat(libs): offline services GTA V reaches in Story Mode
Sign-in, web browser and player dialogs finish cancelled; the message dialog finishes at once;
NpAuth, entitlement consumption, reachability, account age and the NpUtility bandwidth test
report signed out; sceKernelMtypeprotect changes the protection (the memory type has no host
equivalent); the APR ForEach resolvers resolve every path and report each failure in its own
result entry. vswprintf's stub is dropped in favor of the existing wide-formatter
implementation, which already covers it on every platform.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-10-01 23:21:54 +02:00
MichonandClaude Opus 5.5 676bbea348 fix(libkernel): read, pread, AIO and APR write into guest memory as host writes
A file read into guest memory is a host write: before the I/O it pins the shared pages it
covers, which reopens any page armed for write tracking and keeps write tracking from arming
it again until the read ends, when the pages count as written. A destination the guest cannot
write fails with EFAULT before any I/O; read and pread throw as for any failed read, AIO
reports SCE_KERNEL_ERROR_EFAULT, and APR throws. APR also throws when a read ends before the
requested size.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-01 18:56:36 +02:00
RomainROCH 91dd508c85 fix(libkernel): clear directory metadata on close 2026-10-01 16:48:10 +02:00
boykopovar 61ba4d661a refactor(cmake): place all test executables in build tests directory 2026-10-01 12:28:06 +03:00
boykopovar b22ede6c93 refactor(cmake): move some libs test definitions to local cmakelists 2026-10-01 12:21:46 +03:00
SamG-Coder 7f834fc415 feat(libc): add C locale conversion and sample runtime exports 2026-10-01 11:58:51 +03:00
boykopovar e57ee5120d Merge pull request #206 from Praetorian0420/main
Implement network control and socket functionality improvements
2026-10-01 11:41:41 +03:00
boykopovar 33e11c7ea2 Merge pull request #203 from oneandonlydean/fix/libkernel-direct-memory-memfd
fix(libkernel): back direct memory with a memfd on Linux
2026-10-01 11:31:43 +03:00
Praetorian0420 e3eeac4a81 Implement network control and socket functionality improvements
- Enhanced the network control library to check for host addresses and update connection states accordingly.
- Added support for socket operations including connect, listen, accept, send, and recv in the UDP and TCP socket implementations.
- Introduced error handling for various socket operations, ensuring proper feedback for unsupported actions.
- Created comprehensive tests for both TCP and UDP socket functionalities, validating the expected behavior of the network stack.
- Refactored existing code to improve clarity and maintainability, including the removal of unused functions and the addition of necessary error checks.
2026-09-30 23:29:01 -05:00
Dean Galvin d544596283 fix(libkernel): back direct memory with a memfd on Linux
std::tmpfile() places the shared direct memory backing in /tmp, so a
title's direct memory counts against that filesystem's size (a tmpfs is
half of RAM by default): once /tmp fills, touching a backing page raises
SIGBUS. memfd_create gives the same shared memory pages without a
filesystem or its size limit.
2026-10-01 16:06:42 +13:00
MichonandClaude Opus 5.5 1d806a4688 fix(libkernel): honour MAP_FIXED in sceKernelReserveVirtualRange and sceKernelMemoryPoolReserve
Both ignored the flags and the requested address and reserved wherever the arena
had room, so a title that reserves a fixed range (Marvel's Wolverine passes
0x400010, MAP_FIXED | MAP_NO_COALESCE) and then uses the address it asked for
touched memory it never reserved. A fixed reservation now lands at the address,
after checking the range is free; other reservations are placed as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-01 00:00:22 +02:00
boykopovar c71be475fc Merge pull request #187 from MichonGoddijn231849/feat/demons-souls-exports
feat: export the AMPR command buffer waits and writes, sceAgcFuseShaderHalves, sceAgcSetSubmitMode, sceNpSessionSignalingCreateContext and sceNetEpollAbort
2026-09-30 22:03:54 +03:00
boykopovar 95f34bed3d Merge pull request #171 from asimplestray/feat/kernel-time-signal
feat(kernel): implement Sleep, sigprocmask and pthread finalize
2026-09-30 21:03:48 +03:00
boykopovar c3aed31639 Merge pull request #169 from asimplestray/feat/kernel-file-ops
feat(kernel): implement file metadata ops and address conversion
2026-09-30 21:03:08 +03:00
boykopovar ba1d32f450 Merge pull request #168 from asimplestray/feat/kernel-psem
feat(kernel): implement pthread semaphore lifecycle
2026-09-30 21:01:53 +03:00
boykopovar 6756edb91f Merge pull request #164 from MichonGoddijn231849/fix/munmap-across-mappings
fix(libkernel): unmap ranges that span several mappings
2026-09-30 20:55:43 +03:00
asimplestray a4e755ccb1 feat(kernel): implement file metadata ops and address conversion 2026-09-30 16:51:37 +00:00
MichonandClaude Opus 5.5 2b5f2d32d3 feat: export the AMPR command buffer waits and writes, sceAgcFuseShaderHalves, sceAgcSetSubmitMode, sceNpSessionSignalingCreateContext and sceNetEpollAbort
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-30 18:19:55 +02:00
boykopovar 077d4a487f Merge pull request #125 from oneandonlydean/feat/pthread-prio-inherit
feat(libkernel): accept priority-inheritance mutexes
2026-09-30 18:11:38 +03:00
Dean Galvin d8c7c0ccdf feat(agc): track guest memory writes on Linux with async userfaultfd write protection
GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH
and answered 0 on Linux, so every cached copy of guest memory (textures,
resident render targets, buffers, DCC key proofs) read as changed at every
check: resident targets were stored and re-uploaded at every refresh and
unit shadows stayed off. Astro Bot's intro ran at ~3 fps.

libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every
guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is
registered with one userfaultfd in asynchronous write-protect mode
(UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first
write without a handler thread, for user and kernel-mode writes alike), and
PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the
last scan of a range and protects them again in one call. Unmap forgets
the range. Direct memory is mapped from shared backings over its
reservation and a fixed remap replaces the mapping, so both register the
new mapping afresh. The watch is per mapping: a write through one mapping
of shared physical pages is not seen at another, so physical pages mapped
at more than one address leave the watch in every mapping of them.

A registered mapping reads as written until a collect covers it. The watch
keeps that itself instead of relying on how the kernel classifies pages
never faulted in: Linux 6.17 reports a range without page tables as not
written and leaves it unprotected, 7.2 reports it as written. The first
collect of a fresh range reports it written and write-protects it with
UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages
(UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report.

Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where
vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the
mechanism on scratch pages (a populated and an unpopulated page written
after protection report exactly, untouched ones do not). Without it (kernel
older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or
APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays
unwatched exactly as before: CollectWrites answers 0 and callers compare
bytes. A range with a page outside the registered mappings also answers 0,
and a protection or scan that fails reports its remaining pages as written
and answers 0.

The driver's tracker keeps its contract and generations. On Linux its
block stamps live in 4 GiB leaves over the user address space, made on
first stamp, instead of the arena's flat arrays; the collect memo now works
on both platforms. GuestMemory::WriteWatched and Watched replace the arena
queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run
on Linux over watched memory and stay off everywhere else. The Windows
arena path is unchanged apart from going through the same stamp helpers.

Host imports: a driver may lift the write protection of imported pages
itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host
import with a writable HMM fault on every submission after the pages were
protected again, so PAGEMAP_SCAN reports them as written with no CPU store,
and a resident display image read as written at every present (found by
Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA
pins the pages once at the import and stays exact from then on. A probe at
device setup decides per device: it maps and watches scratch pages, collects
them clean, imports them, collects, submits a GPU copy that reads the import,
waits, and collects again. Pages written after the submission mean imports
resolve write protection: every import from then on takes its range off the
watch (GuestMemory::Unwatch) before it is made, so Watched is false for it,
CollectWrites answers 0 and callers compare bytes; GPU writes to watched
memory keep their MarkWritten stamps. The result is logged once:
"[write-watch] host imports resolve write protection: yes/no". A probe that
cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch
overrides the probe (default probe; anything else throws), and
APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe
reads 0 of 4 pages written after the GPU read (4 after the import itself),
so imports stay watched there.

The import itself lifts the protection once where imports stay watched:
on the RTX 4090 the pinning at vkAllocateMemory reports every imported page
as written (4 of 4 in the probe), with no CPU store. Such a report would
read as a CPU write over any GPU results pending in the range and drop
them. So every import of a watched range is made inside an import window
(GuestMemory::ImportWatched), under the tracker lock: the range is
collected right before the import, so earlier CPU writes are stamped as
usual, and again right after it, and what the window reports is stamped
"maybe written". Readers without pending results see it as a change
(UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or
compare. Decisions about pending GPU results read it as untracked:
ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the
drop rule, the write-back's keep blocks, the alias checks, the unit
shadow's stale drop and the whole-image stale checks (through the new
WrittenSince, which also answers false for a range off the watch) store
the results instead of dropping them. A real CPU store racing an import
also races the pending GPU writes to the same memory, which has no
defined order on hardware either, so the GPU results winning is
acceptable. From the second collect on the range is protected again and
watched normally. A range the watch does not cover whole, or a collect of
the window that fails, is taken off the watch instead.

An unwatched import stays off the watch for the rest of its mapping: a
remap registers the range afresh, the import kept over it takes it off
again at the next reconcile, and a re-import after an unmap follows the
decision. Unwatch invalidates the collect memo, so a collect memoized
before the import answers 0. A range that leaves the watch while a storage
image holds a generation for it no longer reads as CPU-written:
ChangedBlocks now tells whether the range is watched, and the drop rule,
the write-back's keep blocks and the alias borrow treat an unwatched range
as untracked (stored, never dropped), as a generation of 0 is.

Tests: guest_memory checks the watch through libkernel mappings (a CPU
store, a read, a kernel write through read(), another thread, a memcpy
across pages, an mprotect cycle, a partial munmap and a fixed remap) and
over a fresh 2 MiB-aligned mapping without page tables, checks direct
memory (watched, an alias takes the shared pages off the watch in both
mappings, a fixed direct remap over flexible memory is watched afresh), and
names the failing line and the reported pages. The recorder tests allocate watched
memory on both platforms, so the UnchangedSince, key proof, close race and
unit shadow tests run on Linux, and the storage refresh test gains a
watched variant: results kept on the GPU reach a CPU reader, and a plain
CPU store into one unit with newer results pending is found by the watch
alone. The recorder tests now also run the real import probe, check the
decision with the result injected (unwatch: an import leaves the watch, a
memoized collect, a partly imported range and the unimported neighbour
answer as they should, MarkWritten stamps only watched memory, a remap and
a re-import after an unmap follow the decision; watch: the import stays
watched and stamped), and check that results pending when their memory
leaves the watch are stored rather than dropped. A further test imports the
memory of a storage image with results pending in watch mode: the
results survive the import window and reach guest memory, a cache
generation taken before the import sees a change, the window reads as
maybe written and not as a CPU store, and a CPU store after it reads as
one again. Unit shadows and the
watched storage refresh report themselves as not run when imports are
compared.
2026-10-01 00:08:08 +13:00