projects / platform / upstream / libvpx.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: bd53cee) | patch
vp9_quantize_avx2,highbd_get_max_lane_eob: fix mask
authorJames Zern <jzern@google.com>
Mon, 10 Apr 2023 20:29:02 +0000 (13:29 -0700)
committerJames Zern <jzern@google.com>
Mon, 10 Apr 2023 21:47:06 +0000 (14:47 -0700)
commit61709a177aa8ef60dbc52e4409beb0d486095d55
tree5f96291e7216ee6f4dfb0eb1cd58d0d770c92b4dtree | snapshot
parentbd53ceef3aa0ee9c8661cfe341e60aa4acac7ab1commit | diff
vp9_quantize_avx2,highbd_get_max_lane_eob: fix mask

Pack nz_mask with zero. After the result is permuted this has the effect
of ignoring the upper half of the iscan register which is only loaded
with 128-bits. Depending on the optimization level and the load used the
upper half of the ymm register may contain undefined values which can
produce an incorrect eob. If this is large enough it can cause a crash.

Bug: chromium:1431729
Change-Id: I4ebae9fa39f228bdd29dcc19935f3f07759d75f5
vp9/encoder/x86/vp9_quantize_avx2.c diff | blob | history
Domain: Multimedia / Codec;