A JUnit file without testcases, as ctest writes when it finds no tests in a build directory, gave '0 tests' and exit code 0, so a run that built nothing passed. Treat it as an unreadable report, with exit code 2.
ctest writes a test whose executable is missing, for example because it failed to link, as status="notrun" with <skipped message="Unable to find executable">, the same shape as a test that skipped itself. sanitizer_report counted both as skipped, which hid a test that did not build. A skipped element now means a skip only when its message is a ctest skip status (SKIP_RETURN_CODE=<n> or SKIP_REGULAR_EXPRESSION_MATCHED, as tools/check_skips.py reads it) or when the status is not a failing one; any other not-run test counts as failed. On a local ASan and UBSan run of main this moves atrac9_gradient, whose executable does not link, from the 721 skipped tests to the failed ones.
Read the system-out and system-err of each testcase and keep the first sanitizer line: the SUMMARY line of ASan, LSan or TSan, else their ERROR or WARNING line, else a UBSan runtime error line with its source location. When any unexpected failure has one, the failed test table gains a First sanitizer report column, shortened to 200 characters and escaped for Markdown.
List the known failures of the selected sanitizers whose test passed, so a fixed test can leave the list, and those that are not in the run at all, such as a renamed test. Neither affects the exit code.
sanitizer_report.py --known reads a text file with one known failure per line: the test name, the comma-separated sanitizers it fails under and the reason. With --sanitizer naming the sanitizers of the build, a failure of a listed test moves to a Known failure table with its reason and no longer fails the run. Blank lines are ignored; a line without a reason, an unknown sanitizer and a duplicate test and sanitizer pair are errors that name the line.
Add tools/sanitizer_report.py, which reads the JUnit file that ctest --output-junit writes, counts passed, skipped and failed tests, lists the failed ones in a Markdown summary and exits with 1 when any test failed. A test that ctest did not run counts as failed, and a disabled test as skipped.
Mohamed Malek Aifa