drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes
authorAlex Hung <alex.hung@amd.com>
Wed, 11 Jan 2023 16:54:11 +0000 (09:54 -0700)
committerAlex Deucher <alexander.deucher@amd.com>
Tue, 28 Feb 2023 19:29:35 +0000 (14:29 -0500)
commitb4ceeffd13870b641a284ffb0f6fb4ffe19b0b14
tree6fd827aaae9b9bd8948cf2c8047c0d9d1cd345a4
parent1e5d4d8eb8c0f15d90c50e7abd686c980e54e42e
drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes

[WHY]
When PTEBufferSizeInRequests is zero, UBSAN reports the following
warning because dml_log2 returns an unexpected negative value:

  shift exponent 4294966273 is too large for 32-bit type 'int'

[HOW]

In the case PTEBufferSizeInRequests is zero, skip the dml_log2() and
assign the result directly.

Reviewed-by: Jun Lei <Jun.Lei@amd.com>
Acked-by: Qingqing Zhuo <qingqing.zhuo@amd.com>
Signed-off-by: Alex Hung <alex.hung@amd.com>
Tested-by: Daniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c