drm/bochs: fix DRM_FORMAT_* handling for big endian machines.
authorGerd Hoffmann <kraxel@redhat.com>
Fri, 21 Sep 2018 13:47:01 +0000 (15:47 +0200)
committerGerd Hoffmann <kraxel@redhat.com>
Tue, 25 Sep 2018 12:49:49 +0000 (14:49 +0200)
commit48b442238250bd27edfef13ab314b295b2734cc4
tree6b26575d83c1f75d57e67283e7d15423ce9d95c3
parent184bef8924c95d43c1476d21034c14c8582f487f
drm/bochs: fix DRM_FORMAT_* handling for big endian machines.

Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code
on bigendian machines.  Also set the quirk_addfb_prefer_host_byte_order
mode_config bit so drm_mode_addfb() asks for the correct format code.

Create our own plane and use drm_crtc_init_with_planes() instead of
depending on the default created by drm_crtc_init().  That way the plane
format list is correct on bigendian machines.

Also re-add the framebuffer format check dropped by "df2052cc92 bochs:
convert to drm_fb_helper_fbdev_setup/teardown".

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in
the bochs-drm.ko driver on big endian machines.  Without the patch only
ADDFB (which still seems to be used by the majority of userspace) works
correctly.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20180921134704.12826-4-kraxel@redhat.com
drivers/gpu/drm/bochs/bochs_fbdev.c
drivers/gpu/drm/bochs/bochs_kms.c