drm/amd/display: Fix DCN32 DSC delay calculation
authorGeorge Shen <george.shen@amd.com>
Fri, 7 Oct 2022 16:43:26 +0000 (12:43 -0400)
committerAlex Deucher <alexander.deucher@amd.com>
Tue, 1 Nov 2022 15:47:50 +0000 (11:47 -0400)
[Why]
DCN32 DSC delay calculation had an unintentional integer division,
resulting in a mismatch against the DML spreadsheet.

[How]
Cast numerator to double before performing the division.

Reviewed-by: Alvin Lee <Alvin.Lee2@amd.com>
Acked-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: George Shen <george.shen@amd.com>
Tested-by: Mark Broadworth <mark.broadworth@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.c

index ad66e24..4a3e7a5 100644 (file)
@@ -1746,7 +1746,7 @@ unsigned int dml32_DSCDelayRequirement(bool DSCEnabled,
                }
 
                DSCDelayRequirement_val = DSCDelayRequirement_val + (HTotal - HActive) *
-                               dml_ceil(DSCDelayRequirement_val / HActive, 1);
+                               dml_ceil((double)DSCDelayRequirement_val / HActive, 1);
 
                DSCDelayRequirement_val = DSCDelayRequirement_val * PixelClock / PixelClockBackEnd;