Re: [PATCH 3/5] drm: fix drm_mode_addfb() on big endian machines.

From: Gerd Hoffmann
Date: Wed Sep 05 2018 - 02:10:48 EST


Hi,

> >>> As explained before, that would break radeon userspace on big endian hosts.
> >>
> >> We last discussed this about a year ago, so I hope you'll forgive my
> >> lapse in memory...
> >>
> >> There's userspace that uses ADDFB2 with DRM_FORMAT_XRGB8888 but
> >> expects it to be host-endian?
> >
> > ADDFB, not ADDFB2. The latter probably didn't even exist yet when this
> > was made to work. :)
>
> Right, but ADDFB doesn't know or care about DRM_FORMAT_*. That's what
> I'm saying -- keep ADDFB working, and fix up the DRM_FORMAT_*
> underneath it both in the conversion and in the driver. Gerd's patch
> allows us to do this incrementally, eventually truing up the
> DRM_FORMAT_* in the driver, enabling ADDFB2 to work as expected.

If it is that simple then yes, we should be able to fix the radeon kms
driver, then drop the quirk once all kms drivers are fixed.

But IIRC there are some radeon-sepcific calls used by the radeon xorg
driver affected too (thats why the commit message says "... both xorg and
kernel drivers ..."), so fixing it for radeon isn't that easy ...

cheers,
Gerd