drm/format-helper: Fix endianness in drm_fb_*_to_*() conversion helpers
authorGeert Uytterhoeven <geert@linux-m68k.org>
Fri, 8 Jul 2022 18:21:08 +0000 (20:21 +0200)
committerNoralf Trønnes <noralf@tronnes.org>
Tue, 19 Jul 2022 14:15:50 +0000 (16:15 +0200)
commit4d9db10576ff51afa8cf7727fbad55ada299359b
treea2ce8b5a3d1ea13212cb169035469b555f78d7fe
parent14e7157afb055248ed34901fcd6fbf54201cfea1
drm/format-helper: Fix endianness in drm_fb_*_to_*() conversion helpers

DRM formats are defined to be little-endian, unless the
DRM_FORMAT_BIG_ENDIAN flag is set.  Hence when converting from one
format to another, multi-byte pixel values loaded from memory must be
converted from little-endian to host-endian.  Conversely, multi-byte
pixel values written to memory must be converted from host-endian to
little-endian.  Currently only drm_fb_xrgb8888_to_rgb332_line() includes
endianness handling.

Fix this by adding endianness handling to all conversion functions that
process multi-byte pixel values.

Note that the conversion to RGB565 is special, as there are two
versions: with and without byteswapping of the RGB565 pixel data.

Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Noralf Trønnes <noralf@tronnes.org>
Link: https://patchwork.freedesktop.org/patch/msgid/33f390d3bae2c5a45c0050097dc95f2e17644f2f.1657300532.git.geert@linux-m68k.org
drivers/gpu/drm/drm_format_helper.c