A handler that read the mask and put it back with SIG_SETMASK stored its
own signal, which sigprocmask reports from the hold, in the process mask
and cleared the hold, so the signal stayed blocked after the handler
returned. pselect, setcontext and a siglongjmp inside the handler replace
the mask the same way. When the handler's own thread releases a hold, the
signal now gets back its bit from before the handler when the handler
returns, as FreeBSD's sigreturn puts back the mask from before it. A mask
another thread sets meanwhile is kept.
Main's #10706 moves the declarations of exported functions and their types
into the libraries' public headers, and gives WaitForPendingInSet a
KernelTimespec timeout. The merge keeps this branch's DeliverUnblockedPending
and ChangeBlockedMask with main's KernelTimespec. It takes main's side of
the declarations in GuestSignalSets.cpp, whose setjmp and longjmp this
branch had added and libc's Api.hpp now declares. GuestSignalHandlerMask.cpp
includes the libkernel and libc public headers in place of its own
declarations and GuestSignalSet and GuestSigaction types, as the
public_headers audit requires.
A handler's blocks are now holds kept apart from the mask sigprocmask sets, and released when the handler returns, so a sigprocmask(SIG_BLOCK) from another thread meanwhile survives even for a signal in the handler's sa_mask. sigprocmask reports the holds; SIG_UNBLOCK of a held signal releases its hold and SIG_SETMASK, which longjmp uses, releases them all, while sigsuspend's own restore keeps them.
main now keeps all four words of the guest signal mask. A handler's sa_mask above signal 32 is added too, and SIGKILL and SIGSTOP stay unblocked, as sigprocmask leaves them.
A guest signal handler ran with the mask it was raised under, so a
handler that raised its own signal again re-entered itself at once,
sigprocmask inside the handler did not report the signal or the
handler's sa_mask as blocked, and SA_RESETHAND left the handler
installed for the next delivery.
FreeBSD 11.4 forms the handler's frame and then, in postsig_done()
(sys/kern/kern_sig.c), blocks the handler's sa_mask and, unless
SA_NODEFER is set, the signal itself, and resets the disposition to
SIG_DFL when SA_RESETHAND is set; sigreturn restores the previous mask,
so a signal raised meanwhile stays pending until the handler returns.
sigaction keeps reporting SA_RESETHAND afterwards, since sigdflt() does
not clear ps_sigreset.
The handler now runs with its signal (unless SA_NODEFER) and its mask
added to the guest mask, which is restored when it returns, and pending
signals that the restored mask unblocks are delivered when raise
returns. SA_RESETHAND resets the handler to the default before it runs.
Yunus Shaikh