From: Xiahong Bao <xiahong.bao@nxp.com>
Date: Fri, 27 Oct 2023 23:52:04 +0000 (+0900)
Subject: calc_pframe_target_size: fix integer overflow
X-Git-Tag: accepted/tizen/7.0/unified/20240521.012539~1^2~52^2
X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=61c927a4ede8b43b29091e3ee9f993eea3b3156a;p=platform%2Fupstream%2Flibvpx.git

calc_pframe_target_size: fix integer overflow

The intermediate value in the target bandwidth
calculation may exceed integer bounds.

Bug: 308007926

Change-Id: I8288c5820db06a550d88bf91fccc86106996deaa
Signed-off-by: Xiahong Bao <xiahong.bao@nxp.com>
---

diff --git a/vp8/encoder/ratectrl.c b/vp8/encoder/ratectrl.c
index 6f14322..fcd4eb0 100644
--- a/vp8/encoder/ratectrl.c
+++ b/vp8/encoder/ratectrl.c
@@ -719,7 +719,8 @@ static void calc_pframe_target_size(VP8_COMP *cpi) {
         }
 
         /* lower the target bandwidth for this frame. */
-        cpi->this_frame_target -= (cpi->this_frame_target * percent_low) / 200;
+        cpi->this_frame_target -=
+            (int)(((int64_t)cpi->this_frame_target * percent_low) / 200);
 
         /* Are we using allowing control of active_worst_allowed_q
          * according to buffer level.