Files
AnyPS5/docs/dev/debt/libSceLibreSsl.md
T
Claude 24bbc6d5f8 fix(libSceLibreSsl): leave the AES key untouched when key setup fails
AES_set_encrypt_key zeroed the whole schedule and set rounds before it
checked the key length, so a refused call changed the AES_KEY. As in
LibreSSL 2.5.4's aes_core.c, it now returns -1 or -2 before writing
anything and writes only the 44, 52 or 60 words the key length needs.

The test fills the key with a sentinel and checks, for both setup
functions, that refused calls leave all 244 bytes unchanged, that
AES-128 and AES-192 keep the 16 and 8 unused words, and that every
written word matches the FIPS-197 appendix A schedules and the
decryption schedules computed independently and with OpenSSL 3.0.13.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-23 18:56:08 +00:00

5.5 KiB

libSceLibreSsl technical debt

Unknown function info

  • libSceLibreSsl pending exports - 357 named callable exports are listed in the PS5 3.20 export table and were cross-checked against the same module in PS5 firmware 13.60. Their signatures and behaviour are unverified; the declarations use the no-parameter unknown-signature convention and throw through NotImplemented_nid_no_patch.
  • MD4, MD5, SHA1, SHA224, SHA256, SHA384 and SHA512 Init/Update/Final and SHA256 (libSceLibreSsl) - signatures and context layouts follow the LibreSSL headers, and buffering and Final follow its sources; the module's LibreSSL version is unknown and nothing is checked on the console. Behaviour that differs between LibreSSL versions throws: SHA256 with a null digest buffer (older versions return a static buffer) and SHA512_Final with the SHA-512/224 or SHA-512/256 digest length. Null contexts, null input with a nonzero length, null digest buffers in the 32-bit Final functions and a buffered byte count that is not below the block size, which LibreSSL does not check, throw
  • DES_set_key, DES_ecb_encrypt, DES_set_odd_parity (libSceLibreSsl) - the 128-byte DES_key_schedule holds the 8-byte key followed by zeros instead of LibreSSL's precomputed subkeys, so a title reading its contents sees different bytes; DES_ecb_encrypt derives the subkeys through mbedtls on each call. Not compared with the console. Results follow LibreSSL des_key.c and des.c: DES_set_key skips the parity and weak-key checks and returns 0, as it does while DES_check_key is 0 (that variable is not in the PS5 export inventory), a nonzero enc encrypts, and DES_set_odd_parity sets each byte's low bit for odd parity. Null pointers throw where LibreSSL faults
  • X509_verify_cert_error_string - no console call is verified. The 64-bit argument, low-32-bit lookup, messages and shared numeric fallback follow LibreSSL 2.5.4, the version identified in published firmware 13.40 measurements. Console modifications and other firmware versions remain unverified. Unknown-code results share an overwritten static buffer and are not safe for concurrent use, as in that release; code 1 and codes above 54 use the numeric fallback.
  • RAND_add, RAND_bytes, RAND_status, RAND_load_file, RAND_file_name, CRYPTO_malloc, CRYPTO_free, CRYPTO_set_mem_functions (libSceLibreSsl) - follow LibreSSL's rand_lib.c, randfile.c and malloc-wrapper.c, which are the same in 2.6.5 (3.4.3 linked); nothing is checked on the console. RAND_bytes reads host entropy and CRYPTO_malloc and CRYPTO_free use the guest heap. CRYPTO_malloc returns null for a size of zero or less, as LibreSSL 2.5.4 to 3.4 do; later versions take a size_t size and call malloc. A null buffer with a positive length throws
  • AES_set_encrypt_key, AES_set_decrypt_key, AES_encrypt, AES_decrypt, AES_ecb_encrypt, AES_cbc_encrypt, AES_cfb128_encrypt, AES_ctr128_encrypt (libSceLibreSsl) - follow LibreSSL's aes.h and mode code. Nothing is checked on the console. Key setup follows LibreSSL 2.5.4's aes_core.c, the version the X509_verify_cert_error_string entry names: a null pointer returns -1 and a key length other than 128, 192 or 256 bits returns -2 before the key is written, and only the 44, 52 or 60 words the key length needs are written. The AES_KEY schedule uses that file's generic C layout (big-endian words, rounds 10, 12 or 14). The 2.5.4 amd64 build uses aes-x86_64 instead, which copies the key bytes without swapping, so each word is byte-reversed compared with the C layout (OpenSSL 3.0.13's x86-64 code gives the same schedules byte-reversed); the console's build options are unknown, so a schedule the title reads or fills is unverified. CTR leaves the last keystream block in ecount_buf even at a block boundary. Null pointers in the cipher and mode functions, other rounds values and a CFB or CTR position outside 0-15 throw