Commit Graph
133 Commits
Author SHA1 Message Date
mirkobrombin 79ceca417f chore: merge pull request #71 from rayzorben/feat/ntdll-critical-sections 2026-10-31 08:15:54 +00:00
Raymond Bennett 01577e440e feat: serve ntdll's critical sections from the guest RTL and give the PEB its lock
Zig's standard library calls RtlEnterCriticalSection and
RtlLeaveCriticalSection on the PEB's FastPebLock before it reads the process
parameters, and stopped at the first call. ntdll's critical section functions
now forward to the guest RTL, which already serves kernel32's (the Rtl
forms answer an NTSTATUS), and the PEB names a free critical section at
FastPebLock (PEB+0x38, PEB+0x1c for 32-bit programs) next to the loader lock.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the RTL exports, the ntdll forwarding rule and the PEB field
(cherry picked from commit bd07c49ed7df6c38fa76c4a27597811eb9b76ea6)
2026-10-30 08:34:15 -07:00
Raymond Bennett df78d476dc feat: poll sockets through the AFD device, as mio and wepoll do
mio, the I/O layer under Tokio, waits for sockets by opening \Device\Afd
with NtCreateFile and issuing IOCTL_AFD_POLL through NtDeviceIoControlFile,
with the completions delivered to an I/O completion port; wepoll and libuv
do the same. Every Tokio program that touches the network stopped there:
xh, Deno's fetch, Bun.

An AFD endpoint is now a handle that can join a completion port. A poll
reads AFD_POLL_INFO (the timeout, the sockets and the events wanted), waits
in the reactor on the sockets' host descriptors, and finishes as overlapped
I/O does: the output records with the events that occurred (receive,
expedited, send, disconnect, abort, local close, accept, connect failure),
the IO_STATUS_BLOCK, the event, and a packet carrying the APC context. The
native cancels end pending polls as cancelled. WSAIoctl answers
SIO_BASE_HANDLE and the SIO_BSP_HANDLE codes with the socket itself, since
no layered provider sits on it. The structures and codes are the ones those
open-source projects define; Microsoft does not document this interface.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the AFD endpoint, its poll and cancellation, and the base-handle codes
(cherry picked from commit 660a20304054e00ab94afef0322f9dbd0a8cbb8f)
2026-10-30 08:25:32 -07:00
mirkobrombin 8e6eb23925 chore: merge pull request #73 from rayzorben/feat/ntdll-kernel32-forms 2026-10-30 06:32:59 +00:00
mirkobrombin 38c47764ad chore: merge pull request #69 from rayzorben/fix/standard-handles-inheritable 2026-10-30 06:32:59 +00:00
mirkobrombin a020ed0b4d chore: merge pull request #55 from rayzorben/feat/nt-create-file 2026-10-30 06:32:59 +00:00
mirkobrombin da35d9f548 chore: merge pull request #43 from rayzorben/fix/module-handle-by-path 2026-10-30 06:32:58 +00:00
mirkobrombin 90c9ca4a7b chore: merge pull request #40 from rayzorben/feat/ntdll-string-functions 2026-10-30 06:32:58 +00:00
mirkobrombin ffb2e75959 chore: merge pull request #38 from rayzorben/feat/current-processor-number 2026-10-30 06:32:58 +00:00
mirkobrombin a6fd705da6 chore: merge pull request #27 from rayzorben/pr/app-local-dll-precedence 2026-10-30 06:32:58 +00:00
mirkobrombin 1d561e2ecd fix: give each window its own GPU surface, hidden ones included 2026-10-29 23:18:44 +00:00
mirkobrombin fd46ad1b5a feat: name failed delay loads and the thread of each traced call 2026-10-29 21:51:37 +00:00
mirkobrombin fab478bf8a feat: map the sysparams and UI Automation API sets and add PathCchRemoveFileSpec 2026-10-29 21:12:46 +00:00
mirkobrombin 0bb7dd6f04 feat: serve Windows Runtime system classes Office asks for at startup
Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Wrote the twinapi Windows Runtime system classes, their shell and netlist hooks and the twinapi client test, after Office stopped at startup on missing runtime classes.
2026-10-29 17:10:05 +00:00
mirkobrombin ba02478235 feat: compile HLSL through an own d3dcompiler_47 2026-10-29 11:10:04 +00:00
mirkobrombin be6e825ba8 fix: report display capabilities for every device context 2026-10-29 09:57:47 +00:00
Raymond Bennett eddfb6a619 feat: answer the ntdll forms of kernel32 functions programs call directly
Programs built without kernel32 imports, Zig's among them, call ntdll's
RtlUpcaseUnicodeChar, RtlQueryPerformanceCounter and
RtlQueryPerformanceFrequency, RtlAddVectoredExceptionHandler and its
relatives, and leave through RtlExitUserProcess. Each is the function
kernel32 has (CharUpper of one character, QueryPerformanceCounter and
QueryPerformanceFrequency, AddVectoredExceptionHandler and the others,
ExitProcess) and now runs the same handler.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the added names
(cherry picked from commit 6b9a401a7364d6aa1a360ee6217ef53bb3aa9456)
2026-10-29 01:30:24 -07:00
Raymond Bennett 4b98555fa6 fix: make a program's standard handles inheritable
A program's standard handles came from its creator by inheritance, so on
Windows they carry the inherit flag. Raven made them without it, and Dart's
`dart run`, which re-executes itself with its standard handles in a
PROC_THREAD_ATTRIBUTE_HANDLE_LIST, was refused with ERROR_INVALID_PARAMETER.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the change
(cherry picked from commit dc01c2417b95a583baec65bb82200323ba785f62)
2026-10-29 00:08:39 -07:00
Raymond Bennett 023fe78345 feat: open files and directories with NtCreateFile and NtOpenFile
Rust's standard library, uv, Zig and the event loops built on mio call
NtCreateFile and NtOpenFile directly, which Raven did not answer.

The object name becomes a Win32 path (\??\ and \DosDevices\ names, UNC
names, and names relative to an open directory), and the disposition and
create options become CreateFileW's: FILE_DIRECTORY_FILE and
FILE_NON_DIRECTORY_FILE check what was opened, a directory is created first
for FILE_CREATE or FILE_OPEN_IF, synchronous options decide whether the
handle is overlapped, and the write-through, sequential, random-access,
no-buffering, delete-on-close and reparse-point options become the matching
flags. The IO_STATUS_BLOCK reports FILE_OPENED, FILE_CREATED,
FILE_OVERWRITTEN or FILE_SUPERSEDED, and errors come back as the NTSTATUS
values the Win32 errors stand for.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the handlers, from the NtCreateFile and NtOpenFile documentation
(cherry picked from commit 1482289fad28953e3df44449973cd9ea168f7d5e)
2026-10-28 23:17:41 -07:00
Raymond Bennett 59b96ac618 fix: find a module named by its path in GetModuleHandleExW, as GetModuleHandleW does
GetModuleHandleExW looked modules up by file name only, so a full path never
matched, and on 64 bits GetModuleHandleW had no path lookup either. The .NET
host loads hostfxr.dll by its full path and then pins it with
GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_PIN, path), so PowerShell 7
stopped with "Failed to pin library".

Both now use GetModuleHandle's own lookup: by name, with API sets and the
trailing-dot rule, or by path, where the 64-bit loader matches the module of
that file name to the file it was loaded from (or, for the libraries Raven
serves, to the system folder GetModuleFileName reports for them).

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the shared lookup and the 64-bit path lookup
(cherry picked from commit 9ea61e7f3c61546be696103a103181cd6ed3e5c3)
2026-10-28 23:16:13 -07:00
Raymond Bennett 748341457a feat: add ntdll's strlen, strcmp and strncmp
eza's GNU build imports them from ntdll, which exports part of a C runtime.
The comparisons read no byte past the first difference, the terminator or
the count.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the handlers
(cherry picked from commit bb244301ebef9726e2e55012184d63c8bcb060a1)
2026-10-28 23:16:05 -07:00
Raymond Bennett 1f91a7038f feat: add GetCurrentProcessorNumber and its Ex, Nt and Rtl forms
DuckDB stops at start-up on GetCurrentProcessorNumber. The number is the host
processor the calling thread runs on, in group 0 and within the processor count
GetActiveProcessorCount reports; the Ex forms fill PROCESSOR_NUMBER.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: the handlers

(cherry picked from commit 7d0c81ae4019d0dbfb4d1b05604b225be92fd2a1)
2026-10-28 23:15:51 -07:00
Raymond Bennett fd3875f197 feat: replace only listed system libraries with a program's own copy
A copy in the program's folder now loads for d3d8, d3d9, d3d10core, d3d11,
dxgi, ddraw, dinput, dinput8, winmm, version, xinput1_3 and xinput1_4,
the libraries graphics wrappers and mod loaders ship. Every other system
library stays the runtime's whatever the program ships, as before this
series, so a bundled dbghelp.dll or winhttp.dll is still answered by
Raven. RAVEN_PREFER_LOCAL_DLLS=1 loads the program's copy of any system
library outside the core list instead.

The program's folder is no longer read file by file at each start: it is
listed once, on the first listed name the runtime asks about, and only a
file that matches such a name is mapped to check that it is a DLL of the
program's width. The C runtime ordering change in main.cpp is dropped, as
msvcrt.dll and ucrtbase.dll are again never looked for there.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Reworked the change to the maintainer's review: the allowlist,
 the RAVEN_PREFER_LOCAL_DLLS setting, the per-name ProgramLibraries
 lookup shared by the x64 and x86 paths, and the dependencies_test case
 and CTest entry for both modes.
2026-10-28 14:20:02 -04:00
Raymond Bennett 10515aecda feat: load a program's own copy of a non-core system library
On Windows only the KnownDLLs are protected from a copy in the program's
folder; graphics wrappers, overlays and plugin loaders ship their own
winmm.dll, version.dll, dinput8.dll, d3d9.dll or dxgi.dll there and
forward to the system copy. Raven served every system library itself.

A short core list (ntdll, kernel32, kernelbase, user32, gdi32, comctl32,
advapi32, msvcrt, ucrtbase, ole32, oleaut32, combase, rpcrt4, shell32,
shlwapi, setupapi, comdlg32) is still always Raven's. For any other
system library a copy beside the program loads, and Raven's own
implementation stays reachable under its System32 or SysWOW64 path and
through LOAD_LIBRARY_SEARCH_SYSTEM32. Copies an installer leaves in the
prefix's system directory are still ignored. The x86 path shares the
program-folder scan, which replaces the d3d9.dll special case.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Wrote the loader, provider and dependency changes and the new
 dependencies_test case, after a program's own winmm.dll was found to be
 replaced by Raven's. Prompt: find what stops the program's own DLL
 from loading and fix it in Raven.
2026-10-28 14:13:17 -04:00
Ray Bennett 4699a741c1 feat: give console programs the terminal as their console (#31)
* feat: give console programs the terminal as their console

A standard handle that is a terminal, and CONIN$, CONOUT$ and CON, now act as a console: modes map to terminal settings,
reads and writes convert between UTF-16 and the terminal's UTF-8, terminal input becomes key events, and Ctrl+C runs the
control handlers. Files and pipes still fail the console functions, as on Windows.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Wrote the console functions over the terminal, their pty test and the guest test program from the public console API documentation.

* test: let the console client run by hand, saying which step failed

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Made the exact size and cursor checks optional and gave each step an instruction and a failure number.

* feat: follow the console cursor through the output instead of asking the terminal

Programs that redraw their command line read and move the cursor on every key; asking the terminal each time was slow, and
wrong on terminals that never answer. The terminal is now asked only when the position may be stale.

Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Wrote the cursor tracker, WriteConsoleOutputCharacter, ScrollConsoleScreenBuffer, the resize events and their tests.
2026-10-28 17:11:25 +01:00
mirkobrombin 7da686f352 feat: name the module and function of a failed GetProcAddress in the trace
Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Added the module and function names to the failed GetProcAddress trace.
2026-10-28 10:45:26 +00:00
mirkobrombin bea2192ed6 fix: leave room for hooking libraries in front of every runtime export 2026-10-27 22:04:55 +00:00
mirkobrombin c1911963d1 fix: reserve the program's stack for threads that name only a commit size 2026-10-27 18:50:00 +00:00
mirkobrombin b387f9f6f2 feat: serve App-V package files from their virtual file system folders 2026-10-27 16:10:47 +00:00
mirkobrombin 2ee5d19e95 feat: publish the generic AVL tables of ntdll 2026-10-27 14:47:46 +00:00
mirkobrombin 7ebf58957c feat: add RegSetKeyValue and the key and value information classes of the native registry API 2026-10-27 14:46:45 +00:00
mirkobrombin c895d8e91c feat: add PathCchCanonicalize and the command line argument helpers 2026-10-27 14:32:31 +00:00
mirkobrombin 7a8f280f1f feat: add Windows Runtime strings and app package queries
Assisted-by: Claude Code:claude-opus-5-5
AI-Scope: Fixed the string header layout and wrote the activation factory registry, device family and package queries from the public headers.
2026-10-27 09:20:47 +00:00
mirkobrombin 5378f9b89b feat: fetch over HTTP and HTTPS with WinINet 2026-10-27 02:55:04 +00:00
mirkobrombin 44a593867c feat: run an SSH client's terminal and file transfer windows 2026-10-26 23:13:38 +00:00
mirkobrombin bcae123e00 feat: keep the runtime's libraries as files in the system directories
Closes #22
2026-10-26 14:17:44 +00:00
mirkobrombin 4bc7845391 feat: make the native OpenGL bridge the default for 64-bit programs
Assisted-by: Claude:claude-opus-5.5
AI-Scope: wrote the default selection of the OpenGL bridge and the fallback to a program's own opengl32.dll
2026-10-26 06:09:35 +00:00
mirkobrombin 137391a0c4 perf: answer tick counts inside parallel guests
Assisted-by: Claude:claude-opus-5.5
AI-Scope: found tick count calls trapping in parallel processes while profiling and enabled their in-guest stubs
2026-10-26 02:24:41 +00:00
mirkobrombin 9815ab8f6d perf: serve handle closes and event changes 2026-10-26 02:22:37 +00:00
mirkobrombin 41f3c1ba57 perf: trim the parallel host's per-call overhead 2026-10-26 00:17:53 +00:00
mirkobrombin 72ec37d544 perf: answer decided waits inside served calls 2026-10-26 00:17:53 +00:00
mirkobrombin 3957dc46ab perf: push, pop and flush singly linked lists inside x64 guests 2026-10-25 22:04:55 +00:00
mirkobrombin 5ca60c6e94 feat: add an experimental X11 desktop backend
Refs #10

Assisted-by: Claude Code:claude-opus-5.5
AI-Scope: wrote the XCB presentation, Vulkan and EGL surface support, the --desktop=x11 option and the Xvfb tests from the issue #10 plan
2026-10-25 16:17:49 +00:00
mirkobrombin 1fc462a544 fix: preserve handles and security tokens across processes 2026-10-25 08:52:34 +01:00
mirkobrombin 61927c1cc8 fix: handle shared waits and semaphore wakeups 2026-10-25 06:09:55 +01:00
mirkobrombin acddd36586 fix: wait for matching window messages without busy polling
Assisted-by: Codex:GPT-6
AI-Scope: Implement blocking message waits and cover message filters and wakeups.
2026-10-25 04:38:33 +01:00
mirkobrombin fbbb8fcdd9 feat: add opt-in per-thread graphics queues 2026-10-25 03:36:31 +01:00
mirkobrombin 14d44fb628 feat: add a native OpenGL bridge 2026-10-25 01:39:15 +02:00
mirkobrombin 4d0e2b4be1 fix: preserve thread context during served API calls 2026-10-24 23:29:27 +02:00
mirkobrombin 5cb1d1b313 fix: reserve TLS slot zero for executable storage 2026-10-24 16:24:28 +00:00