Fix segfault found by fuzzer.

Unsigned int(input_len) is casted to int(flen), this can lead to using negative value,
unfortunately openssl doesn't check it.

According to openssl documentation, input_len is limited by RSA key size,
let's validate it in yaca to avoid segfault.

Change-Id: I8e821b94794f1b5d7231df16c591fe88c12c84e2

diff --git a/src/rsa.c b/src/rsa.c
index 7fbb6e4..336d1f7 100644 (file)
--- a/src/rsa.c
+++ b/src/rsa.c
@@ -1,5 +1,5 @@
 /*
- *  Copyright (c) 2016 Samsung Electronics Co., Ltd All Rights Reserved
+ *  Copyright (c) 2016-2021 Samsung Electronics Co., Ltd All Rights Reserved
  *
  *  Contact: Krzysztof Jackiewicz <k.jackiewicz@samsung.com>
  *
@@ -89,6 +89,9 @@ static int encrypt_decrypt(yaca_padding_e padding,
 
        max_len = ret;
 
+       if (input_len > max_len)
+               return YACA_ERROR_INVALID_PARAMETER;
+
        ret = yaca_zalloc(max_len, (void**)&loutput);
        if (ret != YACA_ERROR_NONE)
                return ret;