Fix T9050 accidentally valid padding issue

Usually happens at least once per 2000 runs when using public RSA
encryption with OAEP padding followed by private RSA decryption with
PKCS1 v1.5 padding. The OAEP is quite unpredictable and can produce a
valid PKCS1 v1.5 padding from time to time.

Valid PKCS1 v1.5 padded message looks as follows:
0x00 || 0x02 || PS || 0x00 || M
where M is a decrypted message and PS is 8+ non-zero octets.

Fix by checking the unpadded message length if above case occurs.

Change-Id: I9991730f5e5cc895dfbfbaf6a6c757dd15f7a313
Hint: Use only 512-bit keys to speed up testing.

diff --git a/src/yaca/yaca-test-rsa.cpp b/src/yaca/yaca-test-rsa.cpp
index 3fcd2a5..4bc7b21 100644 (file)
--- a/src/yaca/yaca-test-rsa.cpp
+++ b/src/yaca/yaca-test-rsa.cpp
@@ -1,5 +1,5 @@
 /*
- *  Copyright (c) 2016 Samsung Electronics Co., Ltd All Rights Reserved
+ *  Copyright (c) 2016 - 2021 Samsung Electronics Co., Ltd All Rights Reserved
  *
  *  Contact: Dariusz Michaluk (d.michaluk@samsung.com)
  *
@@ -197,9 +197,14 @@ void test_rsa_padding(const KeyPair& kp, const PaddingInfo& pi, EncryptionType e
                               (p.padding == padding))))
             expected = YACA_ERROR_NONE;
 
-        YACA_RESULT(expected, decrypt(p.padding, dec_key.get(),
-                                      ciphertext.get(), ciphertext_len,
-                                      &tmp, &plaintext_len));
+        int ret = decrypt(p.padding, dec_key.get(),
+                          ciphertext.get(), ciphertext_len,
+                          &tmp, &plaintext_len);
+        if (ret != expected && expected == YACA_ERROR_INVALID_PARAMETER) {
+            YACA_ASSERT_MSG(ret == YACA_ERROR_NONE, "Got unexpected error " << ret);
+            YACA_ASSERT_MSG(plaintext_len != max_len,
+                            "Message unpadded with invalid padding has correct length");
+        }
     }
 
     /* decryption with SSLV23 will fail if it was used during encryption */