libs: encoder: h264_fei: fix potential overflow before widen
authorU. Artie Eoff <ullysses.a.eoff@intel.com>
Thu, 19 Sep 2019 18:17:24 +0000 (11:17 -0700)
committerU. Artie Eoff <ullysses.a.eoff@intel.com>
Thu, 19 Sep 2019 18:17:27 +0000 (11:17 -0700)
commit1ce66e2ed5eeecf0b6c28118dccdd83fdc4f6afb
treee7d937810588aae781495e3798bb9dc1b8f3fcae
parent2bfef5a4548a1b1f9cd2ec6a0670db6460b47dca
libs: encoder: h264_fei: fix potential overflow before widen

Found by static analysis. encoder->mb_width * encoder->mb_height
is evaluated using 32-bit arithmetic before widen.  Thus, cast
at least one of these to guint64 to avoid overflow.
gst-libs/gst/vaapi/gstvaapiencoder_h264_fei.c