Re: [PATCH v2] drm/virtio: Add suppport for non-native buffer formats

[Christian Zigotzky]

On 19 November 2023 at 07:33 pm, Geert Uytterhoeven wrote:
> Hi Christian,
>
> On Sun, Nov 19, 2023 at 5:28 PM Christian Zigotzky
> <chzigotzky@xxxxxxxxxxx> wrote:
> > On 16 November 2023 at 03:44 pm, Gerd Hoffmann wrote:
> > > On Thu, Nov 16, 2023 at 02:16:54PM +0100, Geert Uytterhoeven wrote:
> > > > When using virtgpu on a big-endian machine, e.g. powerpc QEMU:
> > > >
> > > >       virtio-pci 0000:00:02.0: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-2)
> > > >
> > > > or m68k/virt:
> > > >
> > > >       virtio-mmio virtio-mmio.125: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-2)
> > > >
> > > > and the graphical display fails to come up.
> > > >
> > > > Before, the call to drm_mode_addfb() caused a translation from a fourcc
> > > > format (XR24) to a bpp/depth pair (32/24) to a potentially different fourcc
> > > > format (BX24 on big-endian), due to the quirk processing in
> > > > drm_driver_legacy_fb_format().  After, the original fourcc format (XR24)
> > > > is passed unmodified.
> > > >
> > > > However, the virtgpu DRM driver supports only a single format for its
> > > > main plane: DRM_FORMAT_HOST_XRGB8888, which is XR24 on little-endian,
> > > > and BX24 on big-endian.  I.e. on big-endian, virtgpu does not support
> > > > XR24, which is the default DRM format, and must be supported by all
> > > > drivers.  Before, this was reported, but didn't lead to a failure:
> > > >
> > > >       virtio-mmio virtio-mmio.125: [drm] bpp/depth value of 32/24 not supported
> > > >       virtio-mmio virtio-mmio.125: [drm] No compatible format found
> > > >
> > > > As the core virtgpu driver and device support both XR24 and BX24 on both
> > > > little-endian and big-endian just fine, fix this extending the list of
> > > > supported formats for main plane and cursor plane to XR24/BX24 resp.
> > > > AR24/BA24.
> > > >
> > > > Fixes: 6ae2ff23aa43a0c4 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")
> > > > Reported-by: Christian Zigotzky <chzigotzky@xxxxxxxxxxx>
> > > > Closes: https://lore.kernel.org/r/c47fba21-3ae9-4021-9f4a-09c2670ebdbc@xxxxxxxxxxx
> > > > Suggested-by: Gerd Hoffmann <kraxel@xxxxxxxxxx>
> > > > Signed-off-by: Geert Uytterhoeven <geert@xxxxxxxxxxxxxx>
> > > Reviewed-by: Gerd Hoffmann <kraxel@xxxxxxxxxx>
> > The new patch works but I don't see the virtio-mouse-pci pointer
> > anymore. I see the pointer with -device usb-tablet. Please check the
> > second patch. I will use the first patch for the RC2 of kernel 6.7.
> That's probably partially explained by commit 99748ab64fcc8578 ("drm:
> virtio: fix virtio_gpu_cursor_formats"), which forced BA24 for the
> cursor plane on big-endian, but unfortunately linked thread doesn't
> contain the full picture.
>
> Where is the default cursor format defined?
> I see virtio_gpu_mode_dumb_create() still defaults to
> DRM_FORMAT_HOST_XRGB8888.  However, that can't be the cause, as the
> cursor formats require an alpha channel.
>
> Gr{oetje,eeting}s,
>
>                          Geert
Hi Geert,

Could you please revert the v2 patch because of the issue with the 
virtio-mouse-pci cursor? I will try to use the v1 patch for the RC3 of 
kernel 6.7.

Thanks,
Christian