Fix T9050 accidentally valid padding issue

Usually happens at least once per 2000 runs when using public RSA
encryption with OAEP padding followed by private RSA decryption with
PKCS1 v1.5 padding. The OAEP is quite unpredictable and can produce a
valid PKCS1 v1.5 padding from time to time.

Valid PKCS1 v1.5 padded message looks as follows:
0x00 || 0x02 || PS || 0x00 || M
where M is a decrypted message and PS is 8+ non-zero octets.

Fix by checking the unpadded message length if above case occurs.

Change-Id: I9991730f5e5cc895dfbfbaf6a6c757dd15f7a313
Hint: Use only 512-bit keys to speed up testing.

diff --git a/src/yaca/yaca-test-rsa.cpp b/src/yaca/yaca-test-rsa.cpp
index 3fcd2a5497227682c2d86ae699b661b7e3f4d02a..4bc7b21e0e5a9225da99bbcd1e6d06fe1a1d14fe 100644 (file)
--- a/src/yaca/yaca-test-rsa.cpp
+++ b/src/yaca/yaca-test-rsa.cpp
@@ -1,5 +1,5 @@
 /*
- *  Copyright (c) 2016 Samsung Electronics Co., Ltd All Rights Reserved
+ *  Copyright (c) 2016 - 2021 Samsung Electronics Co., Ltd All Rights Reserved
  *
  *  Contact: Dariusz Michaluk (d.michaluk@samsung.com)
  *
@@ -197,9 +197,14 @@ void test_rsa_padding(const KeyPair& kp, const PaddingInfo& pi, EncryptionType e
                               (p.padding == padding))))
             expected = YACA_ERROR_NONE;
 
-        YACA_RESULT(expected, decrypt(p.padding, dec_key.get(),
-                                      ciphertext.get(), ciphertext_len,
-                                      &tmp, &plaintext_len));
+        int ret = decrypt(p.padding, dec_key.get(),
+                          ciphertext.get(), ciphertext_len,
+                          &tmp, &plaintext_len);
+        if (ret != expected && expected == YACA_ERROR_INVALID_PARAMETER) {
+            YACA_ASSERT_MSG(ret == YACA_ERROR_NONE, "Got unexpected error " << ret);
+            YACA_ASSERT_MSG(plaintext_len != max_len,
+                            "Message unpadded with invalid padding has correct length");
+        }
     }
 
     /* decryption with SSLV23 will fail if it was used during encryption */