AES_set_encrypt_key zeroed the whole schedule and set rounds before it
checked the key length, so a refused call changed the AES_KEY. As in
LibreSSL 2.5.4's aes_core.c, it now returns -1 or -2 before writing
anything and writes only the 44, 52 or 60 words the key length needs.
The test fills the key with a sentinel and checks, for both setup
functions, that refused calls leave all 244 bytes unchanged, that
AES-128 and AES-192 keep the 16 and 8 unused words, and that every
written word matches the FIPS-197 appendix A schedules and the
decryption schedules computed independently and with OpenSSL 3.0.13.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
AES_set_encrypt_key, AES_set_decrypt_key, AES_encrypt, AES_decrypt,
AES_ecb_encrypt, AES_cbc_encrypt, AES_cfb128_encrypt and
AES_ctr128_encrypt threw. They now follow LibreSSL's aes.h and its CBC,
CFB-128 and CTR mode code on the caller's 244-byte AES_KEY: key setup
returns -1 for null arguments and -2 for a key length other than 128,
192 or 256 bits, CBC pads a partial final block with the IV and writes
the whole block, CFB and CTR resume from *num, and the CTR counter
carries across all 128 bits.
The test checks the FIPS-197 appendix A and C vectors and the SP 800-38A
CBC, CFB-128 and CTR vectors, split across calls, in place and with
partial blocks.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
CRYPTO_malloc threw for a size of zero or less. LibreSSL 2.5.4, 3.4.1
and 3.4.3 all take an int size and return null there, so it now does
the same, and the test checks for null instead of a throw.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
RAND_add, RAND_bytes, RAND_status, RAND_load_file, RAND_file_name,
CRYPTO_malloc, CRYPTO_free and CRYPTO_set_mem_functions threw. They now
behave as in LibreSSL 2.6.5 through 3.4.3, where these functions did not
change: RAND_bytes fills the buffer from host entropy and returns 1,
RAND_add ignores its input, RAND_status returns 1, RAND_load_file returns
123456 for -1 and the requested count otherwise, RAND_file_name copies
"/dev/urandom" with strlcpy semantics, CRYPTO_malloc and CRYPTO_free use
the guest heap, and CRYPTO_set_mem_functions refuses with 0.
CRYPTO_malloc of zero or a negative size throws because later LibreSSL
versions take a size_t and call malloc instead of returning null.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The DES block cipher and the parity helper come from mbedtls, which is
already a submodule; DES_set_key follows LibreSSL with DES_check_key
off and stores the key in the schedule.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FzhWRMauD3ouc8kSxT2NnT
MD4, MD5, SHA1, SHA224, SHA256, SHA384 and SHA512 Init/Update/Final and
the SHA256 one-shot threw. They now hash as LibreSSL does, on the
caller's MD4_CTX, MD5_CTX, SHA_CTX, SHA256_CTX and SHA512_CTX laid out as
in LibreSSL's md4.h, md5.h and sha.h, so a context copied or inspected
by the title stays valid: the bit count in Nl/Nh, the partial block in
data with its length in num, and md_len selecting the truncated digest.
Final pads and clears the buffer as LibreSSL's sources do.
Cases that differ between LibreSSL versions throw: SHA256 with a null
digest buffer and SHA512_Final with the SHA-512/224 or SHA-512/256
length. The test checks the RFC 1320, RFC 1321 and FIPS 180 vectors over
several chunk sizes, the context fields, copied contexts and those
cases.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Dean Galvin