RSA_padding_check_PKCS1_type_1 refuses a block with fewer than eight
0xFF bytes, but no case reached that check: lowering it to one byte
still passed. RSA_public_decrypt now gets a type 1 block with seven
0xFF bytes, which must be refused, and one with eight, which must give
its payload. The RSA_flags entry also drops its sentence that no export
creates a key: RSA_new and d2i_RSA_PUBKEY do now.
The RSA test pinned features that aren't implemented yet as throwing:
RSA_free with an ENGINE, ex_data, blinding or Montgomery context set,
the BER forms of d2i_RSA_PUBKEY and of a signature's DigestInfo, SSLv23
and X9.31 padding, moduli over 8192 bits, a custom bn_mod_exp, and the
digest types with no object. A test shouldn't lock in a missing
feature, so these checks and the OtherModExp fixture go. The calls still
refuse them, as the debt entries say; the null-pointer, malformed-input
and invalid-length checks stay.
Asked for in oneandonlydean's review.
Yunus Shaikh